A knowledge graph question answering method based on representation learning

Abstract

The application discloses a kind of knowledge graph question answering method based on representation learning.The model can be roughly divided into four modules of question embedding, relationship screening, vector fusion and answer prediction, and the modules are interconnected.Based on the existing knowledge representation learning model, the application proposes an answer prediction method based on vector fusion representation, taking the question embedding module as the basis, the relationship screening and vector fusion module as the core, outputting the result through the answer prediction module, and realizing an end-to-end knowledge graph question answering process.In recent years, knowledge graph has been widely used in more vertical fields as a basic data service to support the construction of upper intelligent applications.In this paper, based on the characteristics and advantages of representation learning, combined with the characteristics and application landing needs of medical background, a question answering system based on medical knowledge graph is designed and implemented, which has broad application prospects in intelligent inquiry and auxiliary clinical diagnosis.

Description

TECHNICAL FIELD

[0001] The application designs a knowledge graph question answering method based on representation learning, and relates to the technical fields of deep learning, representation learning and natural language processing. BACKGROUND

[0002] In May 2012, Google proposed the concept of knowledge graph, marking the birth of knowledge graph. With the development of large-scale knowledge graphs such as Freebase, DBpedia and YAGO, many downstream tasks based on knowledge graphs have been derived, such as semantic search, intelligent recommendation, automatic question answering, etc. The prior knowledge relied on by automatic question answering is a large number of triples containing entities and relationships in the knowledge graph. These triples are the core information of the knowledge graph, and form the knowledge graph through complex connection relationships. The core goal of knowledge graph question answering (KGQA) is to analyze the semantics of the question and mine the most matching answer information from these triples.

[0003] There are many ways to implement knowledge graph question answering. In recent years, with the evolution and popularization of deep learning, more and more knowledge graph question answering methods map the answer prediction process to a vector space for solution. Among them, the method based on representation learning is one of the important means to realize knowledge graph question answering. In many tasks, the difficulty of solving a problem depends on the way information is represented, which is also the theoretical basis of representation learning. A good representation can often reduce the difficulty of subsequent tasks, and the implementation of subsequent tasks can in turn indicate which representation to choose. In the knowledge graph, representation learning embeds the triple information of the knowledge graph into a unified low-dimensional vector space through deep neural networks and certain specific mapping methods, and solves the problem in the vector space. Compared with traditional methods that need to introduce a large number of artificial rules and auxiliary annotation information, the method based on representation learning can autonomously learn the corresponding implicit information from the graph data, and can ideally predict more complex problems.

[0004] In recent years, the call for AI landing has been growing, and knowledge graph has been widely used in more vertical fields as a basic data service to support the construction of industry upper-layer intelligent applications. For example, in the medical field, with the continuous development of big data technology, a large amount of medical information appears on various forms of information carriers. In this digital background, how to effectively extract, store and apply these medical information and improve their utilization efficiency becomes crucial. SUMMARY

[0005] This paper leverages the characteristics and advantages of representation learning to achieve some progress in open-domain knowledge graph question answering. Drawing on transferable methods and considering the characteristics of the medical field and the needs of practical application, this paper constructs the necessary sub-modules and datasets for medical knowledge graph question answering, and designs and implements a question answering system based on medical knowledge graphs. This system ensures high accuracy in question answering while maintaining high answer retrieval efficiency, assisting doctors in information retrieval and clinical diagnosis, and serving as an important foundation for intelligent consultation.

[0006] Based on representation learning and query graph concepts, this invention proposes an answer prediction method based on vector fusion representation. Combined with knowledge representation learning, it constructs an end-to-end knowledge graph question answering model that can be trained and answered with the corresponding number of hops on both single-hop and multi-hop question answering datasets.

[0007] The MetaQA dataset is a publicly available large-scale multi-hop knowledge graph question-answering dataset. It contains over 400,000 questions in the movie domain of a general corpus, categorized into single-hop, two-hop, and three-hop question-answer pairs. It also provides a knowledge graph consisting of approximately 135,000 triples and 43,000 entity triples. Our model achieves good test results on the MetaQA multi-hop question-answering benchmark dataset, and the rationality of our proposed method is validated through ablation experiments analyzing the vector fusion strategy.

[0008] The present invention achieves the above objectives through the following technical solutions:

[0009] 1. In step (i), the input natural language question q is mapped to a question vector v representing the learning space. q For use by subsequent modules;

[0010] 2. In steps (ii) and (iii), all potentially relevant relations are filtered out from the input question to obtain a relation set R, which serves as one of the inputs to the triple encoder, providing the foundation for the vector fusion module:

[0011] (1) First, send the subject entity h in question q to the query graph generator. The query graph generator will perform a preliminary screening of all relations in the knowledge graph and generate a candidate set of relations R0.

[0012] (2) After passing q through RoBERTa, obtain q0 from formula (1), calculate the dot product between q0 and r in R0, and then use the Sigmoid function as shown in formula (2) to obtain their similarity score s. Relationships with similarity scores s greater than 0.5 are considered as a set of similarity scores.

[0013] Set, denoted as R1;

[0014] q0=RoBERTa(q) (1)

[0015]

[0016] (3) For each candidate answer entity a in the candidate answer set, filter the shortest relation path between the subject entity h and a, and form another set of all the relations contained in the shortest path, denoted as R2. Take the intersection operation of R1 and R2, see formula (3), and get the required relation set R;

[0017] R = R1 ∩ R2 (3)

[0018] 3. In step (iv), this invention proposes a vector fusion method v q The relation set R is organically integrated with the relation set R. First, each relation in R is encoded, and then a set of relation vectors V is obtained through a Pos-Att-complex encoder. R , for V R The mean of all relation vectors in the vectors was calculated, and then it was compared with v. q Adding them together completes the fusion representation of the relation and the question vector, and the output fusion representation vector is v. r' ;

[0019]

[0020] 4. In step (v), all entities in the candidate answer set A and the subject entity h are input into the Pos-Att-complex encoder to obtain the set V of candidate answer vectors. A and head entity vector v h V A Each vector in the vector is related to v h v r' The triplet is combined to form a triplet encoding, which is then fed into the Pos-Att-complex decoder to score the triplet, as shown in formula (5).

[0021] Attached Figure Description

[0022] Figure 1 It is a knowledge graph question-answering model structure.

[0023] Figure 2 It is an embedded module for questions.

[0024] Figure 3 It is the relationship filtering module. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings:

[0026] Figure 1The knowledge graph question answering model structure. The model can be roughly divided into four modules: question embedding, relationship screening, vector fusion, and answer prediction. The modules are organically connected, combined with the Pos-Att-complex encoder and decoder, and jointly complete the task of knowledge graph question answering.

[0027] The goal of the question embedding module is to map the input natural language question q into a question vector v q in the representation learning space for subsequent modules. As shown in Figure 2 , first, the question q is word-embedded by RoBERTa to obtain a word vector with a dimension of 768, and then input into an N-layer linear layer to further learn the representation of the question. The dimension of the linear layer is 512, and each layer of the linear layer is connected by a ReLU activation function, giving the network the ability of nonlinear transformation. At the end of the linear layer, a factory layer is used to convert the dimension of the feature vector to the dimension of the triple embedding, obtaining the question embedding vector v q .

[0028] The relationship screening module screens all relationships that may be related to the input question to obtain a relationship set R, which is one of the inputs of the triple encoder and provides the basis for the vector fusion module. The specific process is shown in Figure 3 , first, the subject entity h in the question q is sent to the query graph generator, which performs preliminary screening on all relationships in the knowledge graph to generate a relationship candidate set R0. The generation step is: taking h as the center node, traversing all relationship chains related to h without limiting the number of hops until no node can expand a new relationship chain, obtaining the set R0 of these relationships. After obtaining R0, further screening is performed to make the information output to the subsequent module as accurate as possible. On the one hand, for each relationship r in R0, the semantic similarity between the question q and r is calculated, and a set of relationships with a similarity score s greater than 0.5 is constructed, denoted as R1. On the other hand, for each candidate answer entity a in the answer candidate set, the shortest relationship path between the subject entity h and a is screened, and all relationships contained in the shortest path are constructed into another set, denoted as R2. At this point, R1 contains all relationships that may be related to the question q, and R2 contains the relationships in the shortest path from the subject entity h to all possible answers, and the relationship that the current round of question answering is most likely to involve should be in the intersection of the two sets, so the intersection operation is performed on R1 and R2 to obtain the required relationship set R.

[0029] In the vector fusion module, first, each relationship in R is encoded to obtain a set of relationship vectors V R . In order to represent the vectors in the relationship set and better fuse them with the question vector, the mean of all relationship vectors in V R is calculated, and then it is fused with vq Addition, complete the fusion of the relationship and question vector representation.

[0030] The answer prediction module outputs the answer with the highest score by calculating the score of the answer prediction. The calculation method is shown in formula (5). During training, the model will output an answer prediction score for all candidate answers, resulting in a score vector, while the score vector in the label is only 0 and 1, representing incorrect and correct answers, respectively. When calculating the loss, the binary cross-entropy (BCE) loss is calculated for the two vectors, and the average is calculated within a batch.

[0031] To reflect the ability of the question and answer model to deal with missing links, the triples of the knowledge graph are randomly deleted with a probability of 50% in a similar way to dropout, and the 1-3-hop dataset under the missing graph is obtained. The original dataset is expanded to obtain 6 datasets. The model will conduct experiments on the 6 datasets, and n-full and n-half represent the n-hop dataset under the complete graph and the missing graph, respectively.

[0032] After training on the 6 datasets, 6 models will be saved, and the accuracy of the question and answer will be calculated on the 6 test sets in turn. The test results are compared with the models published in recent years, and the test results are shown in Table 1:

[0033] Table 1

[0034]

[0035] The results show that the answer prediction method based on vector fusion representation proposed in this paper has a question and answer test index close to the best result of the open complete knowledge graph, and the test result on the question and answer dataset of the missing graph exceeds the model in recent years.

[0036] In order to verify the effectiveness of the vector fusion strategy, an ablation experiment was conducted, and the vector fusion was cancelled under the condition that other conditions remained unchanged, and the problem was directly embedded in the vector v q When v r' Input the Pos-Att-complex decoder, train the model under the same experimental environment, and the test results are shown in Table 2:

[0037] Table 2

[0038]

[0039] It can be seen that when the vector fusion is canceled, the results of the model on the 1-half and 2-half data sets are the same as the original model, and the accuracy on other data sets has decreased to varying degrees, verifying the effectiveness of the vector fusion strategy. Among them, the decrease is greater on the full graph question and answer data set, indicating that the vector fusion strategy is more important on the n-full data set, and the influence on the n-half data set in the missing link scenario is slightly smaller.

Claims

1. A knowledge graph question answering method based on representation learning, characterized in that... Includes the following steps: Step 1: Input the word vector obtained by word embedding of question q through the RoBERTa model into a linear layer. Each linear layer is connected by a ReLU activation function. Finally, a factory layer is passed to transform the dimension of the feature vector into the dimension of triplet embedding, resulting in the question embedding vector v. q ; Step 2: Send the subject entity h in question q to the query graph generator. The query graph generator will perform preliminary screening of all relations in the knowledge graph and generate a candidate set of relations R0. For each relation r in R0, calculate the semantic similarity between question q and r. Relations with similarity scores s greater than 0.5 are formed into a set, denoted as R1. Step 3: For each candidate answer entity a in the candidate answer set, filter the shortest relation path from the subject entity h to a, and form another set R2 containing all the relations in the shortest path. Take the intersection of R1 and R2 to obtain the relation set R. Step 4: Encode each relation in R using a Pos-Att-complex encoder to obtain a set of relation vectors V. R , for V R Calculate the mean of all relation vectors in the vector, and then combine it with v. q Adding them together yields a fused representation v of the relation and the question vector. r' ; Step 5: Input all entities in the candidate answer set A and the subject entity h into the Pos-Att-complex encoder to obtain the set V of candidate answer vectors. A and head entity vector v h V A Each vector in the vector is related to v h v r' The triplet is combined to form a triplet encoding, which is then fed into the Pos-Att-complex decoder. The triplet is scored, and the tail entity corresponding to the triplet with the highest score is taken as the predicted answer. Step Six: During training, the model will output a predicted answer score vector p for all candidate answers. i The label score vector contains only 0 and 1, representing incorrect and correct answers respectively, denoted as y. i , for p i and y i Calculate the binary cross-entropy to obtain the binary cross-entropy loss (p). i ,y i ), and calculate the average within a batch.

2. The knowledge graph question answering method based on representation learning according to claim 1, characterized in that... In step two, a method for filtering relationships related to question q in the knowledge graph is proposed. Taking the subject entity h in question q as the central node, all relational chains related to h are traversed without limiting the number of hops until no node can be extended to generate new relational chains, resulting in a set R0 of these relations. For each relation r in R0, the dot product is calculated between it and q through RoBERTa to obtain q0. Then, the similarity score s is obtained using the Sigmoid function. Relationships with similarity scores s greater than 0.5 are formed into a set R1.

Images