A knowledge base question generation method based on hybrid expert model and joint learning
Through a hybrid expert model and joint learning method, the GraphTransformer and Transformer networks are used to deal with the generation of knowledge graph problems, which solves the problem of difficult to generate complex relationships in the existing technology, and achieves more efficient generation of complex problems and improves diversity.
Patent Information
- Application Number
- CN202110490812.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-05-06
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2041-05-06
AI Technical Summary
The existing knowledge graph problem generation methods are difficult to effectively generate problems containing complex relationship information, especially in the multi-hop complex problem scenarios, the input triple subgraph information and constraint problem generation process cannot be effectively encoded.
Using a knowledge base problem generation method based on hybrid expert model and joint learning, the triple subgraph is encoded through the GraphTransformer network, and the complex relationship is predicted during the decoding process using the hybrid expert model and the Transformer network, combining hidden variable modeling structure information , realize the generation and question-and-answer of complex problems.
It improves the ability to obtain complex relationship information generated by knowledge graph problems, improves the diversity and accuracy of problem generation, ensures that the generation of problems includes the expected multi-hop complex relationships, and enhances the ability and robustness of complex problems.
Smart Images

Figure CN113127623B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of natural language processing and relates to a knowledge base question generation method based on a hybrid expert model and joint learning. Background Art
[0002] In recent years, with the rapid development of artificial intelligence, natural language processing technology has received more and more attention. Question generation (QG), as a subtask of natural language understanding, is a deeper study of natural language processing and plays an important role in promoting the development of artificial intelligence technology. Question generation, as a dual task of question answering (QA) tasks, can generate high-quality training data sets for QA tasks, realize the automatic generation of FAQ (Frequently Asked Questions) documents, and realize automatic tutoring systems.
[0003] The question generation task generates questions expressed in natural language based on the given fact source input (such as text, pictures, knowledge base) and answers. The main difference between Question Generation over Knowledge Base (KBQG) and general QG tasks is that the facts input from the knowledge base are generally expressed in the form of triples <subject, relational predicate, object>, and questions expressed in natural language are generated based on the given answers. For example, for a given triple <China, capital, Beijing>, the question "Which city is the capital of China?" can be generated. The structured triple information processed by knowledge graph question generation is widely used in search systems and has greater application value than text-based question generation tasks.
[0004] Traditional question generation methods are implemented by using rules, templates, and manual annotation. Such methods require a lot of manpower and material resources, and require a certain level of language proficiency. They cannot be used for large-scale data generation. In addition, the diversity of questions generated by templates is insufficient and cannot be close to real application scenarios. With the advancement of deep learning technology, especially the excellent performance of sequence network structures such as RNN in sequence to sequence (seq2seq) tasks such as machine translation and text summarization, many ideas have been inspired for question generation methods. Using deep learning methods to achieve automatic question generation can effectively improve the diversity of generated questions, because it does not require too much manual participation and can meet large-scale use. At present, the end-to-end deep learning model based on the encoder-decoder framework has achieved good results in simple question generation, but it cannot effectively handle the complex multi-hop relationship scenarios involved in knowledge graph question generation. However, in the knowledge graph question generation task, multi-hop questions are very common. For example, the question "What is the abbreviation of the capital of China?" involves the triples <China, capital, Beijing> and <Beijing, abbreviation, Beijing>. To generate this question, the relationship "capital" and "abbreviation" needs to be considered logically. For the generation of such multi-hop complex questions involving multiple triple information, existing methods cannot effectively encode the input triple subgraph information, and more importantly, cannot constrain the question generation process to ensure that the generated questions contain the expected multi-hop complex relationships.
[0005] Based on the problems existing in the existing knowledge graph complex question generation methods, the present invention discloses a knowledge graph complex question generation and question-answering joint learning method based on a hybrid expert model, which is mainly used to generate diversified complex natural language expression questions related to the given triple subgraph, answerable by the answer, and corresponding to the given relationship information given a knowledge graph triple subgraph, an answer and complex relationship information. Summary of the invention
[0006] Technical problem: The technical problem to be solved by the present invention is the research on the generation of complex questions in knowledge graphs. It is difficult to effectively generate questions containing complex relationship information. A method for generating knowledge base questions based on a hybrid expert model and joint learning is provided.
[0007] Technical solution: The technical solution adopted by the present invention to solve its technical problem is: a knowledge base question generation method based on hybrid expert model and joint learning. The method uses knowledge graph question answering to constrain the knowledge graph question generation process, improves the ability of knowledge graph question generation to obtain complex relationship information, and completes the process of complex question generation. Knowledge graph question generation and question answering are both implemented through an encoding and decoding framework, in which the encoder and decoder of question answering use Transformer networks, and the encoder of question generation uses GraphTransformer networks. In the decoding process, combined with the hybrid expert model, each step of the model will first predict the expert distribution, and then consider the relationship between the relationships of multi-hop complex problems, comprehensively predict the generated questions or perform relationship predictions. In the joint learning process, the method believes that for the same data, in the process of question answering or question generation, referring to the process of human logical thinking, the relationship information involved in the two tasks should be the same. Back to the method, the decoding process of the two tasks should be completed by similar experts. With the use of the hybrid expert model, the model can generate different types of questions and improve the diversity of question generation.
[0008] The knowledge base question generation method based on hybrid expert model and joint learning of the present invention is divided into three modules, including the following steps:
[0009] a. Knowledge graph complex question generation module
[0010] 1) By reconstructing the input triple subgraph, the complex relational information required for complex problem generation can be encoded from the input;
[0011] 2) Use the graph neural network GraphTransformer method to encode the reconstructed input triple subgraph;
[0012] 3) Use a hybrid expert model to predict experts’ selection of different generation results and improve the diversity of generation problems;
[0013] 4) Use latent variables to model the structural information of relationships, achieve the encoding of complex relationships, and guide the generation of complex problems;
[0014] 5) Using the Transformer network, decode the encoded results and generate questions under the guidance of the relational structure encoding information and the selection expert;
[0015] b. Knowledge Graph Complex Question Answering (KBQA) module
[0016] 6) Use Transformer to encode complex input problems;
[0017] 7) Use a hybrid expert model to select different experts to guide the question-answering process;
[0018] 8) Predict hierarchical dependencies of relationships and complete complex relationship coding and parsing of complex problems;
[0019] 9) Use the Transformer network to predict the relationship sequence corresponding to complex questions and complete the question-answering process;
[0020] c. Joint Learning Module
[0021] 10) Through joint learning, knowledge graph question answering implements constraints on the generation of knowledge graph questions, improves the accuracy of complex relationships contained in the generated questions, and generates high-quality complex questions.
[0022] Furthermore, the specific method of step 1) is:
[0023] For the input triples, a subgraph is formed Where V is the set of entity nodes, E is the set of relationship edges, and the subgraph is a weighted graph with edges. The subgraph of the weighted graph with edges is reconstructed by expanding the relationship edges with attributes into nodes, and then linking the entity nodes and the newly expanded relationship nodes through unweighted edges, and reconstructing the weighted graph subgraph with edges into an unweighted edge subgraph; more specifically, for each relationship edge, it is expanded into two nodes, representing the forward relationship and reverse relationship of the relationship respectively, and then the head entity node, forward relationship node and tail entity node of the triple are connected through the forward directed edge, and the tail entity node, reverse relationship node and head entity node of the triple are connected through the reverse directed edge, and an additional global node is added and connected to all relationship nodes.
[0024] Furthermore, the specific method of step 3) is:
[0025] There are K experts in the custom problem generation, and the goal is to maximize the following formula:
[0026]
[0027] where z q ∈{1,…,K} represents the expert in model selection, G is the input subgraph, θ q is the model parameter of the knowledge graph complex question generation module, Q is the target generated question, and p(.) represents the probability of the part in brackets.
[0028] Further, in the step 4),
[0029] Using hidden variables in decoder generation questions To indicate whether the word generated at time step t belongs to a new short sentence, the hidden variable The calculation formula is as follows:
[0030]
[0031]
[0032] Among them, e0 and e1 are two trainable embedding representations, which respectively indicate that the current word and the previous word belong to the same text sentence, or that the current word starts a new text sentence. Yes Select Expert z in step 3) q The vector representation of is the hidden state at time step t, σ is the sigmoid activation function, fc is the fully connected layer, and v is the initial state vector of the decoder obtained by the encoder.
[0033] Furthermore, the specific method of step 7) is: K experts are customized, and the goal of complex question answering in the knowledge graph is to maximize the following formula:
[0034]
[0035] where z r ∈{1,…,K} represents the expert in model selection, Q is the input complex problem, θ r is the model parameter of the complex question answering module of the knowledge graph, and R is the target relationship sequence.
[0036] Further, in the step 8),
[0037] Preprocess and encode the tree dependencies of each question, and add an additional starting node <h>and answer node , number all nodes; then in the complex problem relationship prediction process, introduce hidden variables to represent the parent node of each relationship in the tree structure, and the calculation formula is as follows:
[0038]
[0039] in Select Expert z in step 7) r The vector representation of is the hidden state at time step t, σ is the sigmoid activation function, The parent node representing the prediction relationship at time step t is the node numbered k.
[0040] Further, in the step 10),
[0041] Expert z in complex problem generation in knowledge graph q and the corresponding expert z in the complex question answering of the knowledge graph r Represent the complex relationship information of the two tasks respectively, and the representation vectors are the same. Add a joint learning task to minimize the following formula:
[0042]
[0043] in and Experts r and z q The vector representation of JS is the JS divergence (Jensen–Shannon Divergence) used to measure the difference between two distributions. and differences;
[0044] Comprehensive step 5) Calculate the loss of the knowledge graph complex question answering task and the knowledge graph complex problem generation task loss in step 9) The final goal is to minimize the following loss function:
[0045]
[0046] By minimizing the above loss function, complex questions containing complex relationships are generated through knowledge graph question answering constraints on knowledge graph questions. The encoding of relationship information by latent variables ensures the interaction of complex relationship information in the two tasks, and the introduction of the hybrid expert model effectively improves the diversity of generated questions.
[0047] Beneficial effects: Compared with the prior art, the present invention has the following advantages:
[0048] Compared with other knowledge graph question generation methods, this method takes into account complex problem situations and designs a joint learning method to ensure the generation of multi-relation complex problems. In addition, the hierarchical relationship between different relations is modeled by latent vectors to improve the parsing and encoding of complex relations. Finally, the generation of questions with different expert guidance is ensured by using the hybrid expert model relationship selection, ensuring the diversity of generated questions.
[0049] Experimental analysis proves that the proposed joint learning method and the use of latent variable models to encode relationships effectively improve the problem generation ability to handle complex problems and effectively ensure the complexity of the generated problems. In addition, the use of a hybrid expert model effectively improves the diversity of generated problems, making the method more robust when generating complex problems. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is an implementation example of the present invention for a complex problem of sequential relations;
[0051] Figure 2 is an example of input triple subgraph reconstruction in the present invention;
[0052] Figure 3 This is an example of implementing the complex tree relationship problem in the present invention;
[0053] Figure 4 It is a model framework diagram of the present invention. DETAILED DESCRIPTION
[0054] The present invention will be further described below in conjunction with the embodiments and the accompanying drawings.
[0055] The knowledge base question generation method based on hybrid expert model and joint learning of the present invention is divided into three modules, including the following steps:
[0056] a. Knowledge Graph Complex Question Generation (KBQG) Module
[0057] 1) By reconstructing the input triple subgraph, it is more effective to encode the multi-hop relationship information required for complex problem generation from the input;
[0058] like Figure 1 As shown, the input subgraph is composed of multiple triplets, each of which can be expressed as<subject,predicate,object> , respectively representing the subject, predicate, and object, such as Figure 1 As shown in the example, it contains a triple<edith_craig,parents,edward_william_godwin> and<edward_william_godwin, profession,architect> Input triples of subject and object as entity nodes, and relations (predicates) as edges linking entity nodes to form a subgraph Where V is the set of entity nodes and E is the set of relationship edges.
[0059] The complex problems solved by this method involve multiple relationships and involve relatively complex subgraphs. The previous methods based on RNN networks and Transformer encoding concatenate all triples into a sequence for encoding processing, which cannot effectively preserve the topological structure relationship between different triples in the subgraph and cannot effectively deal with complex problem scenarios of knowledge graphs. This method uses a graph neural network to encode the triple subgraph G, but the graph neural network cannot process graphs with edge weights. The previous methods usually expand the relationship edges into nodes, so that the edges of the subgraph are unweighted edges.
[0060] Considering the reversible transmission of relationship information and accelerating the transmission of graph node information, this method expands the relationship node into a pair of nodes, representing the positive and negative relationships respectively. In addition, this method needs to effectively encode complex relationships, so a global node is introduced to link all relationship nodes. On the one hand, it further accelerates the information exchange between different nodes, and on the other hand, it aggregates relationship information as the input of the encoding layer, which can effectively allow the model to learn the most important relationship information for complex problems without being overly affected by the entity node information.
[0061] An example of subgraph reconstruction is shown below: Figure 2 As shown, for the triple <V1,R 12 ,V2> and <V2,R 23 ,V3>, V1, V2 and V3 are all entity nodes, R 12 and R 23 is the relationship edge linking the entity nodes, the original one is shown on the left side of 2. The reconstructed subgraph is as follows Figure 2 As shown on the right, g is the newly added global relationship node, R 21 is the relationship R 12 The inverse relationship, R 32 Similarly, the reconstructed subgraph connects all nodes, the information of V1 node can be transmitted to V3 through the forward relationship, and the information of V3 node can also be transmitted to V1 through the reverse relationship. At the same time, the global node g connects all nodes, which can accelerate the interaction of information.
[0062] 2) Use the graph neural network GraphTransformer method to encode the reconstructed input triple subgraph;
[0063] The reconstructed subgraph can be expressed as where V = {v1,v2,…,v N }, N is the number of nodes, and V1 is the global relationship node. It is an adjacency matrix that records the connection relationship between the N nodes of the subgraph. In addition, the input given answer a is a node in the node list. Through one-hot processing, the answer vector can be obtained. The node list V is passed through the embedding layer to obtain the vector representation and add it to the answer vector embedding representation to obtain the node representation Then, through a multi-layer GraphTransformer encoding, each node vector represents the iterative process as follows:
[0064]
[0065]
[0066] Where i represents the i-th node, l represents the vector representation obtained after passing through the l-th layer of graph neural network, is node v i The set of neighbor nodes can be obtained through the adjacency matrix E. W is a trainable variable, a is the attention mechanism calculation function, and this method is implemented through a linear layer.
[0067] Finally, the N nodes encoded by the graph neural network GraphTransformer are represented as H = {h1,h2,…,h N }, this method aggregates all the information of the entire subgraph, especially the global node representation h1 of the relationship information, as the initial state input v=h1 of the subsequent decoding layer.
[0068] 3) Use a hybrid expert model to predict experts’ selection of different generation results and improve the diversity of generation problems;
[0069] The goal of question generation is to generate a question sentence Q = (q1,…,q Tq ), where T q is the length of the question, and the decoder generates questions to maximize the following probability distribution:
[0070]
[0071] where θ q is a parameter of the model. The hybrid expert model assumes that there is a hidden expert guiding the prediction process. Different experts have their own preferences. For example, some prefer to generate multi-hop complex questions, while others prefer to generate simple questions. The method is customized with K experts. Then the goal of the method changes from formula (1) to maximizing the following formula:
[0072]
[0073] The model selects an expert z q ∈{1,…,K}, and in z q Generate question Q under the guidance of .
[0074] The calculation formula is as follows:
[0075] p(z q |G;θ q )=softmax(fc(v))(3)
[0076] Where fc is the fully connected layer, v is the initial state vector of the decoder obtained by the encoder, and softmax is the activation function.
[0077] 4) Use latent variables to model the structural information of relationships, achieve the encoding of complex relationships, and guide the generation of complex problems;
[0078] The question generation decoding process predicts word by word until a complete sentence is generated, such as Figure 1 In the example in , the decoding process predicts the generated word "what", and then predicts the next word "is" based on this result. In the decoding process, there is an obvious jump between the generated words corresponding to the complex relationship. "occupation of" and "father of" can be regarded as two text ranges. This jump between the two text ranges is achieved by using hidden variables. To calculate:
[0079]
[0080]
[0081] Among them, e0 and e1 are two trainable embedding representations, which respectively indicate that the current word and the previous word belong to the same text as a whole, or that the current word starts a new text range. Is to choose experts q The vector representation of is the hidden state at time step t, where the initial state is the global relationship node representation obtained by the encoder σ is the sigmoid activation function.
[0082] 5) Use the Transformer network to decode the encoded results and generate questions under the guidance of the relational structure encoding information and selection experts.
[0083] As mentioned in step 3), the goal of problem generation is to maximize equation (2), where p(z q |G;θ q ) is calculated as in equation (3), and the other part is calculated as follows:
[0084]
[0085] The goal of the final model is to minimize the following loss function:
[0086]
[0087] b. Knowledge Graph Complex Question Answering (KBQA) module
[0088] Knowledge graph complex question answering answers to given questions and obtains results, which generally includes processes such as named entity recognition, entity linking, relationship extraction, and ranking and scoring. Among them, relationship extraction plays a leading role in the entire task. In complex problem scenarios, the complexity is mainly reflected in multiple relations, so this method simplifies the knowledge graph question answering task into a relationship extraction task, and this processing has been used in multiple works in related fields.
[0089] 6) Use Transformer to encode complex input problems;
[0090] Input question text q=(q1,…,q Tq ), where T q is the length of the question, and after embedding, the question representation Q = {h1,h2,…,h Tq }, the Transformer network includes a multi-head layer, a self-attention layer, and a FFN layer. The formula for the problem encoding process is as follows:
[0091] First through the Multi-head layer:
[0092] MultiHead(Q,K,V)=Concat(head1,…,head h )W O
[0093] Where W O is a trainable variable, K=V=Q, The Attention function is the self-attention layer:
[0094]
[0095] where d k is the dimension of vector K.
[0096] After passing through the self-attention layer, we get the vector representation h of the problem, and then pass it through the FFN layer:
[0097] FFN(h)=max(0,hW1+b1)W2+b2
[0098] W1 and W2 are trainable matrix variables, and b1 and b2 are trainable biases.
[0099] 7) Use a hybrid expert model to select different experts to guide the question-answering process;
[0100] The complex question answering of knowledge graph is simplified into a relation extraction task, the task goal is to predict the relation sequence R = (r1, r2, ..., r Tr ), where 1 <T r ≤n r , n r is the number of relationships.
[0101] Similar to step 3), the hybrid expert model is used to assume that there is a hidden expert guiding the prediction process. Different experts have their own preferences. The number of experts is defined as K. The goal of the knowledge graph complex question answering task is to maximize the following formula:
[0102]
[0103] The model selects an expert z r ∈{1,…,K}, and in z r The predicted relationship R is guided by .
[0104] The calculation formula is as follows:
[0105] p(z r |Q;θ r )=softmax(fc(h))
[0106] Where fc is the fully connected layer, h is the vector representation of question Q obtained by the encoder, and softmax is the activation function.
[0107] 8) Predict hierarchical dependencies of relationships and complete complex relationship coding and parsing of complex problems;
[0108] There are certain dependencies between different relationships, such as Figure 1 and Figure 3 , the predicted relationship results are [parents, profession] and [form_of_government, administrative_divisions], such as Figure 1 As shown in P1, the two relationships are logically sequential chain relationships, and Figure 3 The two relationships in the example are shown in P2, which is a parallel relationship. This method preprocesses the tree dependency relationship of the encoding relationship and adds an additional starting node <h>and answer node , number all nodes. Then in the complex problem relationship prediction process, introduce hidden variables to represent the parent node of each relationship in the tree structure. The calculation formula is as follows:
[0109]
[0110] in Is to choose experts r The vector representation of is the hidden state at time step t, σ is the sigmoid activation function, The parent node representing the prediction relationship at time step t is the node numbered k.
[0111] 9) Use the Transformer network to predict the relationship sequence corresponding to complex questions and complete the question-answering process.
[0112] At each time step, in the expert z r Under the guidance of t , the goal is to minimize the following loss function:
[0113]
[0114] in The calculation of is as in step 8) (5), Relation r representing the prediction at time step t t Is the parent node of the node coded as i? r are model parameters.
[0115] c. Joint Learning Module
[0116] 10) Through joint learning, knowledge graph question answering implements constraints on the generation of knowledge graph questions, improves the accuracy of complex relationships contained in the generated questions, and generates high-quality complex questions.
[0117] The method considers that the expert z in the complex problem generation of knowledge graph q and the corresponding expert z in the complex question answering of the knowledge graph r The thinking logic for the same problem is consistent, and the vector representation is the same. Therefore, the joint learning process needs to minimize the following formula:
[0118]
[0119] in and Experts r and z q The vector representation of JS is the JS divergence (Jensen–Shannon Divergence) used to measure the difference between two distributions. and difference.
[0120] Finally, by combining equation (4) in step 5) and equation (6) in step 9), the method aims to minimize the following loss function:
[0121]
[0122] By minimizing the above loss function, complex questions containing complex relationships can be generated through knowledge graph question-answering constraints on knowledge graph questions. The encoding of relational information by latent variables ensures that complex relational information interacts between the two tasks, and the introduction of the hybrid expert model effectively improves the diversity of generated questions. Compared with other knowledge graph question generation methods, this method takes into account complex problem situations and designs a joint learning method to ensure the generation of multi-relational complex problems. In addition, the hierarchical relationship between different relations is modeled by latent vectors to improve the parsing and encoding of complex relations. Finally, the generation of questions guided by different experts in the hybrid expert model relationship selection is ensured to ensure the diversity of generated problems.
[0123] Experimental analysis proves that the proposed joint learning method and the use of latent variable models to encode relationships effectively improve the problem generation ability to handle complex problems and effectively ensure the complexity of the generated problems. In addition, the use of a hybrid expert model effectively improves the diversity of generated problems, making the method more robust when generating complex problems.
[0124] The above embodiments are only preferred implementation modes of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and equivalent substitutions without departing from the principles of the present invention. These technical solutions after improvements and equivalent substitutions to the claims of the present invention all fall within the protection scope of the present invention. < / h> < / h>
Claims
1. A method for complex question generation and question-answering joint learning based on a knowledge graph with a hybrid expert model, characterized in that: The learning method includes the following steps: 1) By reconstructing the input triple subgraph, the complex relational information required for complex problem generation can be encoded from the input; 2) Use the graph neural network GraphTransformer method to encode the reconstructed input triple subgraph; 3) Using a hybrid expert model, the prediction experts select different generated results; 4) Use latent variables to model the structural information of relationships, achieve the encoding of complex relationships, and guide the generation of complex problems; 5) Use the Transformer network to decode the encoded results under the guidance of the relational structure encoding information and the selection expert to generate questions, 6) Use Transformer to encode complex input problems; 7) Use a hybrid expert model to select different experts to guide the question-answering process; 8) Predict hierarchical dependencies of relationships and complete complex relationship coding and parsing of complex problems; 9) Use the Transformer network to predict the relationship sequence corresponding to complex questions and complete the question-answering process. 10) Through joint learning, knowledge graph question answering realizes constraints on knowledge graph question generation; Wherein, in the step 1), The input triples form a subgraph Where V is the set of entity nodes, E is the set of relationship edges, and the subgraph is a weighted graph with edges. The subgraph is reconstructed by expanding the relationship edges with attributes into nodes, and then linking the entity nodes and the newly expanded relationship nodes through unweighted edges, and reconstructing the subgraph into an unweighted edge subgraph. For each relationship edge, it is expanded into two nodes, representing the forward relationship and reverse relationship of the relationship respectively, and then the head entity node, forward relationship node and tail entity node of the triple are connected through the forward directed edge, and the tail entity node, reverse relationship node and head entity node of the triple are connected through the reverse directed edge; In the step 3), Using the hybrid expert model, it is assumed that there is a hidden expert guiding the prediction process. Different experts have their own preferences. Some prefer to generate multi-hop complex questions, while others prefer to generate simple questions. There are K experts customized, and the goal of question generation is to maximize the following formula: where z q ∈{1,…,K} represents the expert in model selection, G is the input subgraph, θ q is the model parameter of the knowledge graph complex question generation module, Q is the target generated question; p(.) represents the probability of the part in brackets, In the step 4), The question text generated by complex questions is divided into multiple short sentences. Each short sentence is related to one of the relationship information in the complex question. There is a certain jump in semantic information between different short sentences. Hidden variables are used when the decoder generates questions. To indicate whether the word generated at time step t belongs to a new short sentence, The calculation formula is as follows: Among them, e0 and e1 are two trainable embedding representations, which respectively indicate that the current word and the previous word belong to the same text sentence, or that the current word starts a new text sentence. Yes Select Expert z in step 3) q The vector representation of is the hidden state at time step t, σ is the sigmoid activation function, fc is the fully connected layer, v is the initial state vector of the decoder obtained by the encoder, In the step 7), The complexity of questions designed for complex question answering on knowledge graphs varies. A hybrid expert model is used. It is assumed that there is a hidden expert to guide the relationship extraction of knowledge graph question answering. There are K custom experts. The goal of complex question answering on knowledge graphs is to maximize the following formula: where z r ∈{1,…,K} represents the expert in model selection, Q is the input complex problem, θ r is the model parameter of the complex question answering module of the knowledge graph, and R is the target relationship sequence.
2. The method for generating complex questions in knowledge graphs and jointly learning questions and answers based on a hybrid expert model according to claim 1 is characterized in that: In the step 8), Complex problems involve multiple relationships. Preprocess and encode the tree-like dependencies of each problem and add additional starting nodes. <h>and answer node , number all nodes, and then in the complex problem relationship prediction process, introduce hidden variables to represent the parent node of each relationship in the tree structure. The calculation formula is as follows: < / h> in Select Expert z in step 7) r The vector representation of is the hidden state at time step t, The parent node representing the prediction relationship at time step t is the node numbered k, and h is the vector representation of question Q obtained through the encoder.
3. The method for complex question generation and question-answering joint learning based on knowledge graph and hybrid expert model according to claim 1 is characterized in that: In the step 10), Referring to the thinking logic of humans on complex question-answering and complex problem generation tasks in knowledge graphs, both have common complex relationship information. q and the corresponding expert z in the complex question answering of the knowledge graph r They represent the complex relationship information of the two tasks respectively, and the representation vectors are the same. To this end, a joint learning task is added to minimize the following formula: in and Experts r and z q The vector representation of JS is the JS divergence used to measure the difference between two distributions. Here it is used to measure and The difference The final comprehensive step 5) calculates the loss of the knowledge graph complex question answering task and the knowledge graph complex problem generation task loss in step 9) The ultimate goal is to minimize the following loss function: By minimizing the above loss function, complex questions containing complex relationships are generated through knowledge graph question answering constraints.
