A Method for Evaluating the Significance of Knowledge Graph Triples Based on Natural Language Question Answering

Abstract

The present invention discloses a method for evaluating the significance of knowledge graph triples based on natural language question answering. The steps are as follows: First, for a given knowledge graph triple, extract the relationship it contains; then, according to the question generation templates corresponding to different pre-set relationship types, convert the triple into the form of a natural language question; based on the generated question sequence, the original task of evaluating the significance of knowledge graph triples can be transformed into a natural language question answering task, and then the existing large-scale pre-trained language model is further fine-tuned through the method proposed by the present invention, and finally the evaluation result of the significance of knowledge graph triples is output. This method significantly improves the accuracy of evaluating the significance of knowledge graph triples without relying on any external knowledge base and graph representation learning.

Description

Technical Field

[0001] The present invention relates to a method for evaluating the significance of knowledge graph triples in the field of natural language processing. Background Art

[0002] Nowadays, the construction of large-scale knowledge bases (knowledge graphs) provides important support for the research of many artificial intelligence tasks. However, with the rapid development of Internet information technology, the knowledge graph triples extracted from massive raw data often have the problem of incompleteness. The integrity problem of the knowledge graph mainly includes two aspects: the significance evaluation of knowledge graph triples and link prediction. Among them, the main purpose of the significance evaluation of knowledge graph triples is to judge whether the extracted triples conform to common sense, which can effectively reduce the cost of manual screening and greatly reduce the noise level of the existing large-scale knowledge base, and has important practical significance.

[0003] Currently, the existing research methods highly rely on retraining language models with external knowledge bases or constructing knowledge graphs based on existing data sets to complete this task. However, on the one hand, the construction of external knowledge bases requires a large amount of manpower and time, and due to the existence of subjective factors, it is difficult to judge the noise level of the existing knowledge base. On the other hand, when constructing knowledge graphs using existing data sets, it is often difficult to handle the knowledge difference problems existing between different fields. In addition, the equipment cost and time required to train models from scratch based on external knowledge bases are also very expensive. And models such as TransE, TransH, and RotateE, although not relying on external knowledge bases, only rely on the structural similarity of knowledge graph triples to evaluate the significance level of knowledge graph triples, and there are certain limitations. Therefore, how to fully exploit the rich general knowledge contained in existing large-scale pre-trained language models to make up for the differences between different fields is an urgent problem to be solved. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to provide a method for evaluating the significance level of knowledge graph triples based on natural language question answering for evaluating the significance level of knowledge graph triples.

[0005] To solve the above technical problem, the technical solution adopted by the present invention is: a method for evaluating the significance of knowledge graph triples based on natural language question answering, which uses the idea of prompt learning to convert this task into a natural language question answering task through specific question generation, so as to fully exploit the implicit knowledge contained in large-scale pre-trained language models, including the following steps:

[0006] 1) Extract the relationships contained in the knowledge graph triples according to the unique structure of the knowledge graph triples;

[0007] 2) Compose a relationship set without duplicate elements based on the relationships extracted in step 1), and set specific question generation templates according to different relationships, thereby transforming the original knowledge graph triple salience evaluation task into a natural language question-answering task;

[0008] 3) According to the question generation templates in step 2), reconstruct the original knowledge graph triple set, that is, transform the original knowledge graph triple format into the form of natural language questions, thereby obtaining a new set of natural language question sequences;

[0009] 4) Based on the set of natural language question sequences obtained in step 3), fine-tune the large-scale pre-trained language model, and finally output the evaluation result of its salience for the original knowledge graph triples.

[0010] The specific method for detecting and extracting the relationships of the knowledge graph triples in step 1) is: for the given original knowledge graph triple data, detect and extract the relationship between the head entity and the tail entity according to its unique triple structure.

[0011] The specific method for obtaining the non-repetitive relationship set and generating question sequences in step 2) is: on the basis of step 1), determine the relationship types of the knowledge graph triples, and set specific question generation templates according to the relationship types. The main forms of the templates are two types, namely "[T] head entity [T] relationship [T] tail entity [T]" and "[T] head entity [T] tail entity [T] relationship [T]", where "[T]" is a trigger word or a prompt word, and the specific quantity and form are determined according to the actual type of the relationship.

[0012] The method for constructing the set of natural language question sequences in step 3) is: according to the relationship-specific question templates obtained in step 2), reconstruct the original knowledge graph triple set, that is, transform the original knowledge graph triple structure <head entity, relationship, tail entity> into the form of natural language questions through the question template, and obtain a new set of natural language question sequences, thereby enhancing the semantic information of the knowledge graph triples and making the input closer to the corpus during model pre-training, which is more beneficial to the training and prediction of the model.

[0013] The specific method for fine-tuning the large-scale pre-trained language model through the set of natural language question sequences in step 4) is: add special tokens "[CLS]" and "[SEP]" to the beginning and end of the questions obtained in step 3) respectively and input them into the model for training, and finally obtain the evaluation result of the triple salience.

[0014] The beneficial effects of the present invention are as follows: The present invention proposes a new framework, which converts the task of evaluating the significance of knowledge graph triples into a natural language question-answering task by means of the idea of prompt learning, and designs a specific question generation template. Through the above method, the input can be made closer to the text corpus during the pre-training of the language model, so as to better mine the common sense knowledge contained in the model, and effectively improve the accuracy of evaluating the significance of knowledge graph triples without relying on any external knowledge base or graph representation learning. Specifically, the present invention has the following advantages:

[0015] 1. Converting the original knowledge graph triples into the form of natural text sequences can effectively enhance the context information of the triples;

[0016] 2. Further converting the task of evaluating the significance of knowledge graph triples into a natural language question-answering task by means of the idea of prompt learning can effectively mine the implicit knowledge contained in large pre-trained language models;

[0017] 3. Fine-tuning the pre-trained language model on the generated sequences to further improve the performance of the model;

[0018] 4. Different from the existing methods that highly rely on existing knowledge bases and graph representation learning or training models from scratch, the method proposed in this paper does not need to rely on any external knowledge and can directly fine-tune the existing models, thus greatly reducing the required time and equipment costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the flowchart of question generation for the present invention.

[0020] Figure 2 It is the flowchart of model fine-tuning for the present invention.

[0021] Figure 3 It is the overall system framework diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0022] The following further clarifies the present invention in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and not to limit the scope of the present invention. After reading the present invention, those skilled in the art will fall within the scope defined by the appended claims of this application for various equivalent transformations of the present invention.

[0023] Example 1: Refer to Figure 1 、 Figure 2 and Figure 3 As shown, a method for evaluating the significance of knowledge graph triples based on natural language question-answering according to the present invention includes the following steps:

[0024] Step 1: According to the given set of knowledge graph triples DS, detect and extract the relationship R contained in each triple according to its unique <head entity, relation, tail entity> structure.

[0025] Step 2: First, based on the relationship set RS extracted in Step 1, as shown in formula (1):

[0026] RS = set.add(R) (1)

[0027] Then, based on the relationship set RS, generate templates according to questions of different relationships, and transform the original knowledge graph triple significance evaluation task into a context information enhanced natural language question answering task. Since the significance evaluation output of knowledge graph triples is generally 0: representing not significant, 1: representing significant, the present invention accordingly limits the significance evaluation result of the model to 0 or 1.

[0028] Step 3: According to the relationship-specific question templates obtained in Step 2, reconstruct the original knowledge graph triple set, that is, transform the original knowledge graph triple structure <head entity, relation, tail entity> into the form of natural language questions through the corresponding question generation templates according to its relationship type, so as to obtain a new set of natural language question sequence.

[0029] Step 4: Based on the set of natural language question sequences QS obtained in Step 3, divide the data into two parts according to the ratio of 9:1: S and T, where S is used as the training set and T is used as the test set; and when inputting into the model, add additional "[CLS]" and "[SEP]" tags to each question q = {w1, w2,..., w l}(l is the sentence length), so that the sentence will be modified into the following form q = {[CLS], w1, w2,..., w l , [SEP]}, and then input it into the embedding layer to obtain its encoding representation, as shown in formula (2):

[0030] H e = Encoder([CLS], q, [SEP]) (2)

[0031] After obtaining the encoding representation H e of the text, input it into the autoencoder model to obtain its hidden layer vector representation H l , l ∈ [1, L], which can be expressed as shown in formula (3):

[0032] H l = Transformer l (H l-1 ), H 0= H(3)

[0033] e

[0034] Then, take out the feature vector H corresponding to the [CLS] bit of the last hidden layer vector representation [CLS] as the output of its pooling layer and input it into the linear layer to map to the corresponding label vector space, as shown in formula (4):

[0035] x = W o H [CLS] + b o (4)

[0036] where b o is the learnable parameter of the linear layer.

[0037] Finally, input it into the sigmoid activation function to get the final output of the model As shown in formula (5), the definition of the sigmoid activation function is shown in formula (6):

[0038] y = sigmoid(x) (5)

[0039]

[0040] The model is trained using binary cross-entropy loss, and its calculation formula is as follows:[[]]END]]

[0041]

[0042] where n represents the batch size, y represents the prediction result of the model, represents the true label.

[0043] It should be noted that the above embodiments are not intended to limit the protection scope of the present invention. Any equivalent transformation or substitution made on the basis of the above technical solutions falls within the protection scope of the claims of the present invention.

Claims

1. A method for evaluating the significance of knowledge graph triples based on natural language question answering, characterized in that Set specific question generation templates according to different relationship types to convert it into a natural language question-answering task, and then evaluate the significance of knowledge graph triples by fine-tuning a large-scale pre-trained language model, including the following steps: 1) For a given set of knowledge graph triples with true labels, extract the relationship categories contained in the set; 2) According to the relationship set extracted in step 1), and set specific question generation templates according to different relationships. The main forms of the templates are two types, namely "[T] head entity [T] relationship [T] tail entity [T]" and "[T] head entity [T] tail entity [T] relationship [T]", where "[T]" is a trigger word or prompt word, and the specific quantity and form are determined according to the actual type of the relationship, so as to convert the original knowledge graph triple significance evaluation task into a natural language question-answering task; 3) Based on the relationship-specific question templates obtained in step 2), reconstruct the original set of knowledge graph triples, that is, convert the original knowledge graph triple structure <head entity, relationship, tail entity> into the form of natural language questions through the question generation template, so as to obtain a new set of natural language question sequences; 4) Based on the set of natural language question sequences generated in step 3), fine-tune the large-scale pre-trained language model, and finally output the evaluation result of the significance of the knowledge graph triples. Specifically, as follows, based on the set of natural language question sequences QS obtained in step 3), divide the data into two parts according to a ratio of 9:1: S and T, where S is used as the training set and T is used as the test set; and add additional "[CLS]" and "[SEP]" markers to each question when inputting the model, and then input it into the embedding layer to obtain its encoded representation, and then input it into the autoencoder model to obtain its hidden layer vector representation, Extract the feature vector H corresponding to the [CLS] bit of the last hidden layer vector representation [CLS] as the output of its pooling layer and input it into the linear layer to map it to the corresponding label vector space; Finally, it is input into the sigmoid activation function to obtain the final output of the model, which is the true label.

2. The method for evaluating the significance of knowledge graph triples based on natural language question answering according to claim 1, wherein The specific method for relationship extraction in step 1) is: according to the triple format of the knowledge graph, extract the relationship categories contained in the original knowledge graph triples.

3. A method for evaluating the significance of knowledge graph triples based on natural language question answering according to claim 1, characterized in that, The specific method for task conversion in step 2) is: generate a triple relationship set without duplicate elements according to the extracted triple relationship categories, and set specific question generation templates according to different relationships.

4. A method for evaluating the significance of knowledge graph triples based on natural language question answering according to claim 1, characterized in that, The specific method for constructing the set of natural language question sequences in step 3) is to convert the original knowledge graph triple structure into the form of natural language questions through the question generation template.

Images