A knowledge credibility measurement method based on knowledge graph

Abstract

The present invention discloses a knowledge credibility measurement method based on knowledge graph, which comprises the following steps: first, using knowledge representation learning algorithm to vectorize entities and relationships in knowledge graph, and then using energy function and sigmoid function to calculate the credibility of relationship; then, according to attribute standard set, the attribute value of entity in knowledge graph is evaluated, and attribute value credibility is calculated by combining word forest similarity and edit distance; then, a question-answering system is constructed, and the question-answering system and question-answering standard set are used to calculate the credibility of knowledge in knowledge graph for problem solving; finally, combining the credibility of relationship, the credibility of attribute value and the credibility of problem solving, and performing weighted summation according to different weights to obtain the credibility of knowledge graph. The invention is helpful to correct the erroneous knowledge in knowledge graph and improve the accuracy of knowledge in knowledge graph.

Description

Technical Field

[0001] The present invention relates to knowledge credibility measurement of knowledge graphs in the field of knowledge graph improvement, and specifically to a knowledge credibility measurement method based on knowledge graphs. Background Art

[0002] Since Google proposed the concept of "knowledge graph" in 2012, knowledge graph has been widely used in semantic search, intelligent question and answer, and decision support. For example, knowledge graph is used for risk control in the financial field and for recommending search results in the search field.

[0003] In the early days, knowledge graphs were basically constructed manually. This construction method is not only very labor-intensive, but also unable to update the new knowledge in the real world in a timely manner. Therefore, many researchers have been committed to studying how to automatically extract knowledge from unstructured data, so that they can continuously obtain knowledge from the network to complete the existing knowledge graph. However, although the automated construction technology saves a lot of manpower and can supplement the new knowledge in a timely manner, this approach will inevitably introduce a lot of erroneous knowledge. At present, knowledge graphs are widely used in fields such as question answering, recommendation, and search. These applications based on knowledge graphs all assume that the knowledge in the knowledge graph is correct, which is bound to have some potential problems. For example, in decision support, wrong knowledge will lead to wrong decisions, resulting in serious consequences. Therefore, the knowledge credibility measurement of knowledge graphs is of great significance.

[0004] Due to the complexity of knowledge graphs, how to better and more accurately measure the knowledge credibility of knowledge graphs is the focus of current research. To address this problem, this paper conducts an in-depth study on the knowledge credibility measurement method based on knowledge graphs, and proposes a method for measuring the credibility of knowledge in knowledge graphs from three dimensions: relationships, attribute values, and problem solving. Summary of the invention

[0005] In order to determine whether the knowledge in the knowledge graph is credible, the present invention proposes a knowledge credibility measurement method based on the knowledge graph, which aims to quantify the knowledge credibility of the knowledge graph and help improve the knowledge graph. In the process of constructing the domain knowledge graph, the knowledge graph can be improved according to the evaluation results, the erroneous knowledge can be corrected, and the missing knowledge can be supplemented, so that the application based on the knowledge graph can accurately feedback the knowledge to the user.

[0006] To achieve the above objectives, the technical solution to achieve the objectives of the present invention is: a knowledge credibility measurement method based on knowledge graph, comprising the following steps:

[0007] The first step is to measure the credibility of the relationship. First, the vector representation of entities and relationships in the knowledge graph is trained to embed the entities and relationships into a low-dimensional vector space. Then the energy function and sigmoid function are used to calculate the credibility of the relationship.

[0008] The translation model in knowledge representation learning regards the relationship vector in the triple as the translation from the head entity vector to the tail entity vector. Therefore, the knowledge representation learning algorithm can be used to train the vector representation of entities and relationships in the knowledge graph, and then the energy function can be used to calculate the credibility of the relationship. Ideally, the value of the energy function calculated for a credible triple should be infinitely close to 0, but in actual situations, the calculation result of the energy function is subject to the training effect of the entity and relationship vector representation. Even if the triple is correct, the value of its energy function often cannot be close to 0. Therefore, the credibility of the relationship can only be judged by the size of the energy value, that is, the smaller the energy value, the more credible the relationship of the triple.

[0009] The second step is to measure the credibility of entity attribute values. The credibility of entity attribute values ​​is measured according to the attribute standard set. First, the credibility of the attribute value of each attribute of an entity is calculated using the word forest similarity and edit distance. There are different credibility calculation methods depending on the attribute value type. Then, the attribute value credibility at the entity level is calculated based on the attribute value credibility of all attributes of each entity. Finally, the attribute value credibility of each entity is weighted averaged to calculate the attribute value credibility of the entire knowledge graph.

[0010] The third step is to measure the credibility of problem solving. First, we build a question-answering system based on the knowledge graph. Then we build a standard set of question-answering. Each line in the standard set contains a natural language question and the standard answer corresponding to the question.

[0011] After the question-answering system and the question-answering standard set are built, the credibility of the problem solving can be measured. The question-answering standard set is read, and each natural language question is input into the question-answering system. The credibility of the problem solving is calculated based on the difference between the standard answer and the answer output by the question-answering system. The credibility calculation method is based on ExactMatch.

[0012] Exact Match is an important indicator to measure the difference between the predicted answer and the standard value, and is applied to different tasks such as classification and regression. Exact Match calculates the proportion of predicted answers that completely match the standard answer in all samples, while the present invention chooses to use cosine similarity to measure the credibility between the actual answer returned by the question-answering system and the given standard answer. The calculation formula is as follows:

[0013]

[0014] Among them, Q is the set of all question sentences, PA i is the predicted answer to the question with id i, GA i is the standard answer to the given i-th question. i , G.A. i ) is PA i with GA i The cosine similarity of .

[0015] The fourth step is to measure the credibility of the knowledge graph. The credibility of the knowledge graph is calculated based on the relationship credibility, attribute credibility and problem-solving credibility obtained in the previous steps. The calculation formula is as follows:

[0016] tw(KG)=γ1tw(relation)+γ2tw(property)+γ3tw(QA)

[0017] Among them, γ1, γ2, and γ3 represent the weight of relationship credibility, the weight of attribute credibility, and the weight of problem-solving credibility, respectively. The values ​​of parameters γ1, γ2, and γ3 are limited to γ1+γ2+γ3=1. The weighted sum of relationship credibility, attribute credibility, and problem-solving credibility is taken to obtain the credibility of the knowledge graph.

[0018] Compared with the prior art, the significant advantages of the present invention are: the technical solution measures the credibility of knowledge in the knowledge graph from three dimensions: relationship, attribute value and problem solving, so as to accurately reflect the credibility of knowledge in the knowledge graph; (1) At the relationship layer, the present invention uses the knowledge representation algorithm to vectorize entities and relationships, and then uses the value of the energy function to represent the credibility of the relationship; (2) At the attribute layer, the present invention adopts different methods to calculate the credibility of attribute values ​​according to the different attribute value types. In the string attribute value, a combination of word forest and edit distance is introduced to calculate the credibility, while the credibility of the numerical type is measured according to the difference between the value in the knowledge graph and the true value; (3) At the problem solving layer, the present invention proposes a method to measure the credibility of knowledge in the knowledge graph from the perspective of problem solving. After constructing a question-answering system based on the knowledge graph, cosine similarity is introduced to measure the credibility of problem solving. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Flowchart of knowledge credibility measurement based on knowledge graph.

[0020] Figure 2 Flowchart of relationship credibility measurement.

[0021] Figure 3 Flowchart of entity attribute value credibility measurement.

[0022] Figure 4 The tree-like organizational structure of the synonym dictionary.

[0023] Figure 5 The path weights of each layer of the synonym dictionary.

[0024] Figure 6 Question and answer process based on knowledge graph. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in combination with specific implementation methods and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention.

[0026] Embodiment 1: Combination Figure 1 The present invention provides a knowledge credibility measurement method based on knowledge graph, the method comprising the following steps:

[0027] The first step is to measure the credibility of the relationship. There are many knowledge representation learning algorithms. In a specific embodiment of the present invention, the TransE algorithm is used to obtain the vector representation of entities and relationships. The calculation process of the credibility of the relationship is shown in the attached figure. Figure 2 As shown, it includes the following steps:

[0028] Step 1: Artificially construct a standard set of relationships, use the TransE algorithm to train entities and relationships, and embed the entities and relationships in the knowledge graph into a low-dimensional vector space.

[0029] Step 2: Use the energy function to calculate the credibility of each triple relationship. The calculation formula is as follows:

[0030] E(h, r, t) = ||h+rt||

[0031] Among them, h is the vector representation of the head node in the triple, r is the vector representation of the tail node in the triple, and t is the vector representation of the tail node in the triple.

[0032] Step 3: Use the sigmoid function to calculate the credibility of each relationship. The calculation formula is as follows:

[0033]

[0034] Among them, α is the threshold associated with the relation r.

[0035] Step 4: Calculate the relationship credibility of the knowledge graph. The calculation formula is as follows:

[0036]

[0037] Among them, R is the set of all triples.

[0038] The second step is to measure the credibility of entity attribute values, using the attribute standard set to conduct a credibility assessment of entity attribute values ​​in the knowledge graph. The process is as follows Figure 3 As shown, the specific steps are as follows:

[0039] Step 1: Use the attribute standard set to calculate the credibility of each attribute in the entity. In a specific example of the present invention, a radar-related attribute standard set is constructed by consulting the data. The calculation method uses different methods according to the attribute value type. Assume that x and y are the actual values ​​of attribute p in the attribute standard set and the knowledge graph, respectively.

[0040] 1) String type attribute value

[0041] The attribute value of string type uses the combination of Cilin and edit distance to calculate the similarity. The "Tongyi Cilin" was originally compiled by Mei Jiaju and others in 1983. However, due to the long history of the original version, the Information Retrieval Laboratory of Harbin Institute of Technology integrated a large number of modern Chinese words and eliminated some rare and uncommon words that appeared less frequently in newspapers and magazines, and compiled the "HIT Information Retrieval Laboratory Tongyi Cilin Extended Edition". Figure 4 As shown in the figure, the extended version of Tongyi Cilin organizes all the entries together according to the tree-like hierarchical structure, and uses 6 layers of coding to encode the entries. The first level is represented by uppercase English letters, the second level is represented by lowercase English letters, the third level is represented by two-digit decimal integers, the fourth level is represented by uppercase English letters, and the fifth level is represented by two-digit decimal integers. The sixth layer of coding "=" indicates that the meanings of the words are equal and synonymous; "#" indicates the same type but different meanings, which are related words. "@" means "self-enclosed" and "independent", which has neither synonyms nor related words in the dictionary.

[0042] If both x and y exist in the synonym dictionary, the synonym dictionary is used to calculate their credibility, otherwise the edit distance is used to calculate their credibility. The formula for calculating the similarity of the dictionary is as follows:

[0043] tw(p)=sim(C1,C2)=1.0-0.05dis(C1,C2)

[0044] Among them, dis(C1, C2) is the distance between word codes C1 and C2 in the tree structure. The distance is obtained by summing the weights from C1 to C2. The weights of each layer in Cilin are shown in the attached figure. Figure 5 shown.

[0045] There are several special cases in the similarity calculation of CiLin. When the codes of two words are the same and the suffix is ​​"=", the similarity is treated as 1, and when the suffix is ​​"#", the similarity is treated as 0.5. When two words are not in a large category, the distance between the words is treated as 18.

[0046] The edit distance similarity calculation formula is as follows:

[0047]

[0048] Among them, Length(x) and Length(y) respectively represent the character lengths of x and y, and LD(x, y) is the edit distance between the two strings x and y.

[0049] 2) Attribute values of numerical types

[0050] The credibility of the attribute value of the numerical type is measured according to its distance from the true value, and the calculation formula is as follows:

[0051]

[0052] Step 2, calculate the credibility of the attribute value of each entity. Assume that P is the set of attributes of entity e, then the calculation formula is as follows:

[0053]

[0054] Step 3, calculate the overall credibility of the attribute value. Assume that E is the set of entities, then the calculation formula is as follows:

[0055]

[0056] The third step, credibility measurement of problem solving. The first step in measuring the credibility of problem-solving knowledge is to construct a question-answering system based on the knowledge graph. In a specific example of the present invention, the template matching method is selected to construct a question-answering system based on the knowledge graph of the radar field. The specific steps are as follows:

[0057] 1) Construct templates. Manually construct a series of templates, each of which contains multiple ways of asking a question and corresponding example graph database query statements.

[0058] 2) Construct a vocabulary. The words that appear in the natural language questions in the templates are used as the vocabulary. Read each question and perform word segmentation on it. The word segmentation tool is selected as hanlp, and stop words such as "de" and "nali" are removed to form the vocabulary vocabulary.txt;

[0059] 3) Convert natural language questions into vectors. Read the template file and use the bag-of-words model to convert each natural language question in the template into a vector.

[0060] 4) Train a question classification model. Use the constructed templates as training data. Each line of the template is a training sample, and the label is the serial number, and each serial number has a corresponding example query statement. Use the Naive Bayes classification algorithm to train a classification model.

[0061] 5) Construct a dictionary of similar words. In the dictionary of similar words, each batch of similar words has a target word, and the choice of the target word is consistent with the entity name in the knowledge graph.

[0062] 6) Natural language question answering. The process of natural language question answering is as follows Figure 6 As shown in the figure. First, the trained classification model Naive Bayes classifier is used to classify natural language questions, obtain classification labels, and obtain query sentence examples based on the classification labels. Then, hanlp is used to perform named entity recognition on natural language questions. Finally, a dictionary of similar words is used to replace the entity with the target word, and the target word is filled into the query sentence to execute the query.

[0063] After the question-answering system is built, a question-answering standard set is constructed based on expert knowledge, which contains natural language questions and standard answers to the questions. Each natural language question in the question-answering standard set is read, and the question is input into the question-answering system. The knowledge credibility of the problem solution is evaluated by comparing the standard answer with the actual answer output by the question-answering system.

[0064] In a specific example of the present invention, Sentence-Bert is used to train a sentence vector representation model, which is used to vectorize the actual answer and the standard answer returned by the question-answering system, and then cosine similarity is used to represent the credibility between the answer returned by the question-answering system and the given standard answer. Let Q be the set of all question sentences, PA i is the predicted answer to the question with id i, GA i is the standard answer to the given i-th question, then the calculation formula for the credibility of the problem solution is as follows:

[0065]

[0066] Among them, similarity (PA i , G.A. i ) is PA i with GA i The cosine similarity of .

[0067] Assume that A and B are PA i with GA i The vector representation of cosine similarity is as follows:

[0068]

[0069] The fifth step is to measure the credibility of the knowledge graph. After obtaining the relationship credibility, attribute credibility, and problem-solving credibility, the credibility of the knowledge graph can be calculated.

[0070] In a specific embodiment of the present invention, an average weight method is used to perform weighted summation of relations, attributes, and problem-solving credibility, and the calculation formula is as follows:

[0071]

[0072] It should be noted that the above embodiments are merely preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Equivalent replacements or substitutions made on the basis of the above technical solutions all fall within the protection scope of the present invention.

Claims

1. A knowledge credibility measurement method based on knowledge graph, characterized in that: The following steps are involved: The first step is to measure the credibility of the relationship. First, the knowledge representation algorithm is used to train the vector representation of entities and relationships in the knowledge graph, and then the energy function and sigmoid function are combined to calculate the credibility of the relationship. The second step is to measure the credibility of attribute values. First, the credibility of the attribute values ​​of each attribute is calculated using the attribute standard set. Different calculation methods are selected according to the different attribute value types. Then, the credibility of the attribute values ​​of each entity is calculated by combining the credibility of the attribute values ​​of all its attributes. Finally, the credibility of the attribute values ​​of the entire knowledge graph is calculated based on the credibility of the attribute values ​​of each entity. The third step is to measure the credibility of problem solving. First, a question-answering system based on the knowledge graph is constructed, and then the credibility of problem solving is calculated using the question-answering standard set and the question-answering system. The fourth step is to perform weighted summation of the relationship credibility, attribute value credibility, and problem-solving credibility to calculate the knowledge graph credibility; The calculation method of the relationship credibility in the first step is as follows: After using the knowledge representation algorithm to train the vector representation of entities and relationships in the knowledge graph, the energy value of each triple, that is, the credibility of the relationship of each triple, is calculated using the following formula: E(h,r,t)=||h+rt|| Where h, r, and t are the vector representation of the head entity, the vector representation of the relationship, and the vector representation of the tail entity in a triple respectively; Then, the sigmoid function is used to convert the credibility of each triple relationship into a value between 0 and 1 to represent the credibility of each triple relationship: Among them, α is the threshold related to the relationship; Finally, the relationship credibility is calculated using the following formula: Among them, R is the set of all triples; The calculation method of the attribute value credibility in the second step is as follows: The credibility of attribute values ​​is calculated in different ways according to the type of attribute values. Assuming that x and y are the values ​​of attribute p in the attribute standard set and the knowledge graph respectively, the credibility of string type is calculated by combining synonyms and edit distance. The calculation formula is as follows: Where LD(x,y) is the edit distance between two words x and y, Length(x) and Length(y) represent the character lengths of x and y respectively, and dis(C1,C2) is the sum of the distance weights of words C1 and C2 in the tree structure; The trustworthiness of numerical values ​​is expressed by the distance between the value in the knowledge graph and the standard value in the attribute standard set. The calculation formula is as follows: After calculating the attribute value credibility of each attribute of entity e, assuming that P is the attribute set of entity e, the attribute value credibility calculation formula of entity e is as follows: After calculating the attribute credibility of each entity, assuming that E is the entity set, the attribute value credibility calculation formula of the knowledge graph is as follows: Among them, in the third step, the method of measuring knowledge credibility from the perspective of problem solving is as follows: After building a question-answering system based on the knowledge graph, cosine similarity is used to calculate the similarity between the actual answer returned by the question-answering system and the standard answer in the question-answering standard set. This similarity is used to represent the credibility of the problem solution. The calculation formula for credibility is as follows: Among them, Q is the set of all question sentences, PA i is the predicted answer to the question with id i, GA i is the standard answer to the given i-th question, similarity(PA i , G.A. i ) is PA i with GA i The cosine similarity of In the fourth step, the calculation method of the knowledge graph credibility is as follows: tw(KG)=γ1tw(relation)+γ2tw(property)+γ3tw(QA) Among them, γ1, γ2, and γ3 represent the weight of relationship credibility, the weight of attribute credibility, and the weight of problem solution credibility respectively. The values ​​of parameters γ1, γ2, and γ3 are limited to γ1+γ2+γ3=1.

Images