A question-answering reasoning method for knowledge graphs

Abstract

The present invention discloses a question-answering reasoning method for knowledge graphs, including: designing graph data concepts, creating entity and entity attribute triples, and constructing entity relationship triples for each graph; performing data normalization on all triple data; storing entities, attribute values ​​and attribute numbers in entity and entity attribute triples in an entity table or a relationship table as one of the data sets to be trained, and labeling the head entity and the tail entity in the entity relationship triples constructed by each graph in an entity table as another data set to be trained; training a "relation-attribute binary classification model"; training the two created data sets in sequence and generating two models; setting three formats of triples to be solved to be input, respectively calculating the similarity between entities and relationships and existing texts in the knowledge base, linking them to the knowledge representation model, and entering the corresponding core knowledge representation model to obtain the result; receiving user feedback on the answer, and storing the content that is correctly inferred but does not exist in the model or storing the content that the user enters as the correct answer after the reasoning is wrong.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing and computer information processing, and in particular to a question-answering reasoning method oriented to a knowledge graph. Background Art

[0002] Knowledge reasoning technology, as the name implies, is to infer unknown knowledge based on existing knowledge, so as to acquire new knowledge. Knowledge reasoning for knowledge graphs is to complete the triples of entities or relationships by using existing complete triples. It mainly revolves around the reasoning of relationships, inferring unknown relationships based on existing relationships in the graph, and generally focuses on the characteristic information of entities, relationships and graph structures. This technology is widely used, but its application in intelligent question and answering is in its infancy. Since knowledge reasoning can enrich the intricate relationships between entities in the graph, it can play an important role in optimizing question and answer understanding.

[0003] Intelligent question-and-answer services already exist in all aspects of life, such as automatic customer service systems, voice dialogue robots, etc. With the development of computer technology and natural language processing technology, people have higher requirements for the technologies used in intelligent question-and-answer products. Traditional intelligent question-and-answer services can achieve machine understanding, intelligent learning, automatic question-and-answer, and human-computer interaction. Specifically, they can be divided into online services represented by machine customer service and offline services represented by question-and-answer robots, which greatly facilitate people's daily lives. The key technologies of question-and-answer mainly involve three aspects: retrieval technology, knowledge network, and deep learning.

[0004] Question-answering systems can be divided into three categories: task-oriented, knowledge-oriented, and chat-oriented. Knowledge-oriented question-answering systems include question-answering knowledge bases and knowledge graph knowledge bases. With the development of intelligent question-answering technology, knowledge reasoning has gradually become one of its internal driving forces. The present invention focuses on the integration and innovation between the two, adopts a knowledge reasoning method oriented to knowledge graphs, and constructs a new question-answering reasoning method based on the triple knowledge base in the graph, which is applied to intelligent question-answering systems. And when the question-answering knowledge graph library is first built, the manual entry of triple data is prone to problems such as irregular construction, or some irregular triple knowledge bases shared by others are directly obtained from the Internet, resulting in the missing of some entity attributes or relationships, which makes it difficult to meet the question-answering requirements. And there may be erroneous triple data in the graph, forming a kind of noise information, which reduces the accuracy of the question-answering function. Summary of the invention

[0005] To solve the above technical problems, the purpose of the present invention is to provide a question-answering reasoning method for knowledge graphs. This method obtains higher quality answers, which are stored in the database after user feedback or review, and gradually self-iterates and improves.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A question-answering reasoning method for knowledge graphs, including:

[0008] A. Design graph data concepts, create entity and entity attribute triples, and construct entity relationship triples for each graph;

[0009] B. Normalize all triple data;

[0010] C. The entities, attribute values ​​and attribute numbers in the entity and entity attribute triples are placed in the entity table or relationship table as one of the training data sets, and the head entity and tail entity in the entity relationship triples constructed by each graph are numbered in an entity table as another training data set;

[0011] D. Train the “relationship-attribute binary classification model”;

[0012] E. Train and create entity-relationship datasets and entity-attribute datasets in sequence, and generate two knowledge representation models;

[0013] F. Set the system to allow the input of three formats of triples to be solved, calculate the similarity between entities and relations and the existing text in the knowledge base, link them to the knowledge representation model, and enter the corresponding core knowledge representation model to obtain the results;

[0014] G. Receive user feedback on the answer and store the content that is correctly inferred but does not exist in the model or the content that the user enters the correct answer after the inference is wrong.

[0015] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0016] This method can obtain higher quality answers, which can be stored in the database after user feedback or review, thus achieving gradual self-iteration and improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of the question-answering reasoning method for knowledge graph;

[0018] Figure 2 It is a complete question-answering reasoning model diagram;

[0019] Figure 3 This is a diagram showing the iterative update process of the question-answering reasoning method. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below in conjunction with embodiments and drawings.

[0021] like Figure 1 As shown in the figure, the question-answering reasoning method process for knowledge graph includes the following steps:

[0022] Step 10: Design the concept of graph data. First, create multiple entities and entity attribute triples in the form of [entity-attribute-attribute value], that is, [e, a, v]. Then, each graph constructs entity relationship triples in the format of [head entity, relation, tail entity], that is, [h, r, t]. The attributes of an entity are the characteristics of the entity itself and have no association with other entities, such as the name, age, and place of origin of a person. The relationship between entities mainly defines the association between entities, such as the association between people and people, the subordinate relationship between people and institutions, etc.

[0023] Step 20: Data normalization. This includes the completion, correction, and entity disambiguation of triple data. Triple completion and correction requires manual review and correction, which can be done by consulting materials, etc. Entity disambiguation mainly solves the phenomenon of polysemy and monosemy in natural language. Entity disambiguation is divided into two steps. The first step is to perform deep learning disambiguation before entity recognition and relationship recognition; the second step is to mainly use relationship dictionaries and subject dictionaries for matching disambiguation. Disambiguate the entities, relationships, and attributes in the triples.

[0024] Step 30: treat e and v in [e,a,v] as entities and number them in an entity table, treat a as a relationship and number them in a relationship table. Given that [e,a,v] and [h,r,t] are similar in model training logic, [e,a,v] will be collectively referred to as [h,r,t] as the first dataset to be trained. Number h and t in [h,r,t] in an entity table, and number r in a relationship table as the second dataset to be trained.

[0025] Step 40 uses r to train the “relationship-attribute binary classification model”.

[0026] Step 50 uses the dgl-ke command line tool to create embeddings. Multiple models can be selected, such as TransE, TransR, RESCAL, DistMult, ComplEx, and RotatE. Two data sets are trained in turn to generate two models, and finally the algorithm that minimizes the model loss is selected.

[0027] Step 60 allows the input of three formats of triples to be solved: [h, r, ? ], [? , r, h], [h, ? , t], where "?" represents the value to be solved. For the first format, r is judged as an attribute or a relationship through the "relation-attribute binary classification model", and the similarity between h and r and the existing text in the knowledge base is calculated respectively, linked to the knowledge representation model, and finally entered into the corresponding core knowledge representation model to obtain the result. The second and third formats are also the same.

[0028] Step 70 receives user feedback on the answer, stores the content that is correctly inferred but does not exist in the model, or stores the content of the correct answer entered by the user after the inference is wrong, and updates the model from time to time.

[0029] The above graph data concept design defines entity attribute categories and entity relationship categories. The attribute table of entities in the defined general data set is shown in Table 1:

[0030] Table 1

[0031]

[0032] Content that can be defined as an attribute, whose attribute value no longer meets the conditions to become an independent entity, for example: [Li Bai, gender, male]. According to the current business conditions, "male" is an attribute rather than an entity.

[0033] The entity relationship table in the general data set defined in this embodiment is shown in Table 2:

[0034] Table 2

[0035]

[0036] The entity table of the general data set defined in this embodiment is shown in Table 3:

[0037] Table 3

[0038]

[0039]

[0040] The triple table of the general data set defined in this embodiment is shown in Table 4:

[0041] Table 4

[0042]

[0043] The last data normalization process: remove wrong or unnecessary characters from all attributes, attribute values, entities, and relationships. In the stage of preparing the test data set, the missing content that can be supplemented is supplemented manually, and the data that cannot be supplemented is put into the data set to be predicted. After the knowledge representation model is trained, the prediction is supplemented and manually reviewed again. In the data processing stage, preliminary entity disambiguation is required to solve the phenomenon of multiple meanings and multiple words with one meaning, split different entities, and merge the same entities, attributes, and relationships.

[0044] The above data storage and training data set generation: According to Table 1, the entities (including attribute values), the relationships between entities, and the attributes within the entities are numbered, and the entity attribute triple id file and the entity relationship triple id file, as well as the training data set, are generated. A total of 4 files are shown in Table 5:

[0045] Table 5

[0046]

[0047] Among them, entities.dict is a mixed dictionary of entities and attribute values, relation.dict is a mixed dictionary of relations and attributes, train_att.tsv and train_rel.tsv are both stored as utf-8 encoded files, and the format is shown in the table. The specific format of entities.dict and relation.dict is: vocabulary + space + encoding, and the specific format of the following two triple training sets is: head encoding + space + relation encoding + tail encoding.

[0048] Question-answering reasoning model training: The question-answering reasoning model is a complete model that includes a classification model, a knowledge representation model, and result integration. The knowledge representation model in this embodiment is trained using the improved DGL-KE tool to speed up the training and expand the training mode. The specific improvements are as follows. There are 6 optional models: TransE, TransR, RESCAL, DistMult, ComplEx, and RotatE.

[0049] The basic idea of ​​TransE is that the closer the sum of the vector representation h of the head entity and the vector representation r of the relationship is to the vector representation t of the tail entity, the better, that is, h+r≈t. The "closeness" here can be measured using the L1 or L2 norm. Its loss function is L(h,r,t)=max(0,d pos -d neg +margin), where d = ||h+rt||. The TransR model establishes separate relationship spaces for different relationships based on the TransE model, and first maps the entities to the relationship space for calculation. r =hMr , t r =tM r , and then train in the relational space to make h r +r≈t r The RESCAL model is based on the TransE model and represents the relationship as a square matrix M r , the possibility of the relationship being established is determined by bilinear matching of the head and tail entities and the relationship. The relationship r between each head and tail entity is represented by a matrix, where the score function is represented by d = hM r t. DistMult is a simplification of RESCAL, using a vector to represent the relationship between two entities.

[0050] Different data sets have different performances in the face of various representation learning algorithms. The optimal model can be selected based on the number of iterations and loss. Since the data set of this embodiment is large, the model used after comprehensive consideration of multiple indicators such as comprehensive training, prediction time and result accuracy is TransE.

[0051] Compared with the widely used Tsinghua open source library OpenKE framework, DGL-KE has a faster training speed and a variety of different training modes. According to the data set and training requirements, CPU, GPU, CPU-GPU hybrid mode and distributed training can be selected. In the use case of this embodiment, the hybrid mode is used to generate embedding. If all data can be contained in the GPU memory, the GPU has the best training performance. Since this embodiment is based on large KG for training, a hybrid mode is selected to obtain a larger CPU and GPU memory pool, and at the same time, it can benefit from the powerful training acceleration capability of the GPU.

[0052] However, since the encapsulated interface provided by DGL-KE cannot be directly connected to the program, this embodiment rewrites the native dglke_train, dglke_predict, and dglke_emb_sim command line interfaces into web interfaces for calling, and rewrites the native input format (only supporting documents) to support web parameter input.

[0053] After the knowledge representation model is trained and tested, the optimal model is selected and put into the question-answering reasoning model.

[0054] The attribute-relationship binary classification model is trained using a pure BERT pre-trained model. In the classification experiment of this embodiment, the classification effect of the BERT model is better, but the model is too heavy. Later, a lightweight classification model is trained through the model distillation method. Finally, the classification model is put into the question-answering reasoning model.

[0055] In the model integration phase, one or more prediction results are obtained and output with the highest score. Figure 2This is a complete question-answering reasoning model.

[0056] There are three types of triples to be completed:

[0057] Case 1: Input [h, r, ? ], the tail entity is missing. First, determine whether r exists. Based on the existence of relationship r, use the "attribute-relationship binary classification model" to determine whether r is an entity attribute or an entity relationship. If r is an entity attribute, the input triple [h, r, ? ] to be completed enters the attribute knowledge representation model. In this model, h and r are first entity linked through the dglke_emb_sim interface, and then the model is predicted through the dglke_predict interface, and the result is finally input.

[0058] Case 2: Input [?, r, t], missing head entity. First, determine whether r exists. Based on the existence of relationship r, use the "attribute-relationship binary classification model" to determine whether r is an entity attribute or an entity relationship. If r is an entity attribute, the input triple [?, r, t] to be completed enters the attribute knowledge representation model. In this model, t and r are first linked through the dglke_emb_sim interface, and then the model is predicted through the dglke_predict interface, and the result is finally input.

[0059] Case 3: Input [h, ? , t], lacking relation. First, determine whether there is r. If there is no relation r, it is impossible to confirm whether to predict entity attributes or entity relations. Then enter the attribute knowledge representation model and the relation knowledge representation model respectively. In these two models, h and t are first linked through the dglke_emb_sim interface, and then the model is predicted through the dglke_predict interface. The prediction result with the highest accuracy is selected and finally input.

[0060] like Figure 3 The figure shows the iterative update process of the question-answering reasoning method. After the knowledge completion of the triples to be completed is performed through the question-answering reasoning model, the user's correction results or manual review results of the reasoning results are collected, the results are updated again to the database storing triples, and the existing model is updated regularly, thus forming an updated iterative model of knowledge reasoning.

[0061] Although the embodiments disclosed in the present invention are as above, the above contents are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the patent protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A question-answering reasoning method for knowledge graph, characterized in that: The method comprises: A. Design graph data concepts, create entity and entity attribute triples, and construct entity relationship triples for each graph; B. Normalize all triple data; C. The entities, attribute values ​​and attribute numbers in the entity and entity attribute triples are placed in the entity table or relationship table as one of the training data sets, and the head entity and tail entity in the entity relationship triples constructed by each graph are numbered in an entity table as another training data set; D. Train the "relationship-attribute binary classification model"; E. Train and create entity-relationship datasets and entity-attribute datasets in sequence, and generate two knowledge representation models; F. Set the system to allow the input of three formats of triples to be solved, calculate the similarity between entities and relations and the existing text in the knowledge base, link them to the knowledge representation model, and enter the corresponding core knowledge representation model to obtain the results; G. Receive user feedback on the answer, and store the content that is correctly inferred but does not exist in the model or the content that the user enters the correct answer after the inference is wrong; The step A specifically includes: creating multiple entities and entity attribute triples in the form of [entity entity-attribute attribute-attribute value value], that is, [e, a, v], from the entities in the test data; constructing entity relationship triples in the format of [head entity head, relationship relation, tail entity tail] for each graph, that is, [h, r, t]; The step C specifically includes: treating e and v in [e, a, v] as entities and numbering them in an entity table, treating a as a relationship and numbering them in a relationship table, and considering that [e, a, v] and [h, r, t] are similar in model training logic, [e, a, v] are collectively referred to as [h, r, t] as the first data set to be trained; numbering h and t in [h, r, t] in an entity table, and numbering r in a relationship table as the second data set to be trained; In step D, r is used to train the "relationship-attribute binary classification model"; The step E specifically includes creating an embedding using the dgl-ke command line tool, selecting one or more models to train the first data set and the second data set in sequence to generate two models, the two models being an attribute knowledge representation model and a relationship knowledge representation model; The step F specifically includes inputting three formats of triples to be solved: [h, r, ?], [?, r, h], [h, ?, t]. For the first format, r is judged as an attribute or a relationship through the "relationship-attribute binary classification model", and the similarity of h and r with the existing text in the knowledge base is calculated respectively, and linked to the knowledge representation model, and finally the corresponding core knowledge representation model is entered to obtain the result.

2. The question-answering reasoning method for knowledge graph according to claim 1, characterized in that: The data normalization processing in step B specifically includes the completion, correction and entity disambiguation of triple data, wherein entity disambiguation includes deep learning disambiguation before entity recognition and relationship recognition and matching disambiguation using relationship dictionary and subject dictionary.

Images