A semantic parsing method for complex problems based on a knowledge base

Abstract

The present invention discloses a semantic parsing method for complex problems based on a knowledge base, belonging to the technical fields of natural language processing and intelligent question answering. The present invention directly retrieves a candidate entity and relationship set in the knowledge base dense space, and performs multi-task learning training on entity disambiguation and relationship classification as auxiliary tasks together with structured query generation. The three sub-tasks share the same set of encoder parameters, and at the same time, each sub-task designs an independent layer on the basis of the encoder for training the corresponding sub-task. Finally, the generated structured query statement is used as the model output result, and the answer to the question is obtained by executing the query statement in the knowledge base. The present invention considers the collaborative relationship between the three sub-tasks. Through joint training, it overcomes the defects of mutual independence and error propagation existing in the traditional scheme, improves the accuracy of the structured query generation task, and at the same time improves the overall effect of the semantic parsing model.

Description

Technical Field

[0001] The present invention relates to the technical fields of natural language processing and intelligent question answering, and particularly relates to a semantic parsing method for complex questions based on a knowledge base. Background Art

[0002] With the continuous development of Internet technology, the question answering system, as a tool for assisting people in obtaining and processing massive information, has been applied to various aspects of life. With the continuous improvement of users' requirements for the effectiveness of information retrieval results, the traditional retrieval-based question answering system returns relevant web pages to users through text keyword matching and other methods, which often contain a large amount of irrelevant information and cannot meet the needs of users. The question answering system based on knowledge base query can accept and understand the natural language questions input by users, infer and query in a large-scale knowledge base to obtain accurate answers to the questions, and finally output the answers to users. Currently, knowledge base question answering has become a hot research direction in the field of intelligent question answering.

[0003] A knowledge base is a database composed of multiple knowledge triples, where the form of a triple is <subject, predicate, object>. Different from web pages that use unstructured free text as the knowledge carrier, a knowledge base can be queried and retrieved through structured query statements such as SPARQL, enabling users to obtain the required knowledge more accurately. In the research of knowledge base question answering tasks, the semantic parsing-based approach has received extensive attention. Semantic parsing refers to semantic parsing of the input natural language question to obtain the corresponding structured query statement in the knowledge base, and then obtaining the answer by executing the query statement in the knowledge base. Experimental data shows that the performance of the knowledge base question answering model for simple questions has basically exceeded the human level. Simple questions refer to those that contain only one topic entity and only require finding one knowledge triple to obtain the answer. Different from simple questions, complex questions require reasoning through multiple knowledge triples and involve operations such as set operations, numerical comparisons, and sorting. The diversity of entities, relationships, and constraints therein increases the difficulty of semantic parsing of the question. Currently, the semantic parsing task for complex questions can be divided into three steps: entity linking, relation linking, and structured query generation. The main purpose of entity linking is to detect entity mentions in the question and link them to the corresponding entities in the knowledge base. The main purpose of the relation linking task is to find the relation predicates in the knowledge base that match the semantics of the question sentence. The main purpose of the structured query generation task is to generate the corresponding structured query statement for the question based on the semantic features of the question obtained in the previous two steps and obtain the answer to the question by execution. However, the introduction of errors in each subtask in the traditional three-stage semantic parsing process will lead to a reduction in the performance of the overall model. At the same time, the three subtasks independently solve different semantic parsing tasks in the same question, which does not conform to the human way of understanding question sentences and will result in a lack of collaborative adjustment ability between the subtasks. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: to propose a semantic parsing method for complex questions based on a knowledge base, to solve the deficiencies in entity and relation diversity in the prior art when dealing with complex questions, and to improve the semantic understanding ability of the model for complex questions and the accuracy rate when answering complex questions.

[0005] The technical solution adopted by the present invention to solve the above technical problem is:

[0006] A semantic parsing method for complex questions based on a knowledge base, characterized in that the method directly retrieves candidate entity and relation sets in the knowledge base dense space, and performs multi-task learning training on entity disambiguation and relation classification as auxiliary tasks together with structured query generation.

[0007] Specifically, the method includes the following steps:

[0008] A. Training a multi-task learning model:

[0009] A1. Set the background knowledge base KB in the current knowledge base question-answering task; then obtain training samples, where each training sample consists of a natural language question and its corresponding SPARQL query statement, and the result of executing the SPARQL query statement in the knowledge base is the answer corresponding to the question.

[0010] A2. Candidate entity retrieval: For the input natural language question, first identify the entity mentions included in the question sentence, and then obtain the candidate entity set through the mapping relationship.

[0011] A3. Candidate relationship retrieval: Retrieve candidate relationships from the knowledge base to finally obtain the candidate relationship set.

[0012] A4. Generate a structured query statement through a multi-task learning model: During the process of structured query generation, entity disambiguation and relation classification are used as auxiliary tasks to form a multi-task learning model; among them, the input of the entity disambiguation task is the natural language question and the candidate entity set obtained in A2, and the output is the entity set included in the question; the input of the relation classification task is the natural language question and the candidate relation set obtained in A3, and the output is the relation set included in the question; the input of the structured query generation task is the result of the entity disambiguation task, the result of the relation classification task, and the natural language question, and the output is a structured query statement.

[0013] Furthermore, in the multi-task learning model, the three sub-tasks share the same set of encoder parameters, and at the same time, each sub-task designs an independent training layer based on the encoder to train the corresponding sub-task, and finally takes the generated structured query statement as the output result of the multi-task learning model.

[0014] A5. Loop through steps A2 - A4 to iteratively train the multi-task learning model until the preset number of training times is reached or the model has converged.

[0015] B. Perform the semantic parsing task of complex questions:

[0016] Use the multi-task learning model trained in step A for the actual complex question semantic parsing task; specifically, take the natural language question as the input, obtain the structured query statement corresponding to the question based on the trained multi-task learning model, and get the answer to the question by executing the query statement in the knowledge base, and at the same time output the entity set and relation set included in the question.

[0017] Furthermore, in step A2, the candidate entity retrieval specifically includes:

[0018] A21. Perform word segmentation on the natural language question, and use the Bert encoder to obtain the embedding vector corresponding to each word in the segmented natural language question.

[0019] A22. Use the BiLSTM encoder to encode the natural language question from two directions to obtain the hidden state vector containing the context information of the natural language question; then, through the CRF layer, use the state transition matrix to consider the label constraint relationship between adjacent words, and finally obtain the optimal annotation result to obtain the entity mentions in the question.

[0020] A23. Construct the mapping relationship from entity mentions to knowledge base entities according to the character matching method, and finally obtain the candidate entity set E of size K1 through the mapping relationship i 。

[0021] Furthermore, in step A3, the candidate relationship retrieval specifically includes:

[0022] A31. Preprocess the natural language question q, and replace the entity mentions identified in step A22 with [MASK]

[0023] tokens to obtain the processed question T q 。

[0024] A32. Use the dual encoder to encode the question T q and the relationship r respectively, and use the dot product to calculate the correlation S(q,r) between the two. The calculation formula is as follows:

[0025] Y q =BERTCLS(T q ) (1)

[0026] Y r =BERTCLS(r) (2)

[0027] S(q,r)= Y q .Y r (3)

[0028] where BERTCLS represents the output vector of the [CLS] token in the BERT encoding process, Y q is the vector representation of the question T q , Y r is the vector representation of the relationship r; the correlation score between the question T q and the relationship r is calculated by clicking, denoted as S(q,r), and the K2 relationships with the highest correlation are selected and added to the candidate relationship set to obtain the candidate relationship set R j 。

[0029] Further, in step A4, generating a structured query statement through a multi-task learning model specifically includes:

[0030] A41. Complete the entity disambiguation task: Complete the entity disambiguation task by performing binary classification on the candidate entity set obtained in A2; specifically, first process the entity e i in the candidate entity set E i , concatenate the entity e i with the adjacent relationships r1, r2,..., r n in the knowledge base to obtain the following entity representation containing more semantic knowledge information

[0031]

[0032] Then concatenate the natural language question q and the entity representation , encode them using a multi-task parameter-sharing encoder, and obtain the similarity score S(q, e i between the question q and the entity e i ) through a fully connected layer and an activation function. The formula is as follows:

[0033]

[0034]

[0035] where, represents the vector representation obtained after concatenating the question q and the entity e i ; during the training process, use cross-entropy as the loss function L ENT . The formula is as follows:

[0036]

[0037] where, v i represents the label of the candidate entity e i , where 1 indicates that e i is in the SPARQL query statement corresponding to the question q, and 0 indicates that e i is not in the corresponding SPARQL query. K1 is the number of candidate entities; finally, add the entities identified as positive samples in the entity disambiguation task to the entity set corresponding to the question to obtain the entity set E q of the question q.

[0038] A42. Complete the relationship classification task: Complete the relationship classification task by performing binary classification on the relationship r j in the candidate relationship set R j obtained in A3; specifically, concatenate the question q and the relationship r jAfter splicing, it is input into the multi-task parameter-sharing encoder. After being processed by the average pooling layer and the activation function, the final score s(q, r j ) is obtained, and the formula is as follows:

[0039]

[0040] Among them, represents the vector representation obtained after splicing the question q and the relation r j ; during the training process, binary cross-entropy is used as the loss function L REL , and the formula is as follows:

[0041]

[0042] Among them, u j represents the label of the candidate relation r j , where 1 means r j is in the SPARQL query statement corresponding to the question q, and 0 means r j is not in the corresponding SPARQL query. K2 is the number of candidate relations; the relations judged as positive samples in the relation classification task are added to the relation set, and finally the relation set R q of the question q is obtained.

[0043] A43. Complete the structured query generation task: The input is the entity set E q of the question q q , the relation set R

[0044] and the question q, and the target output is the structured query. q First, splice the question q with E q and R all to obtain the question representation T

[0045] T all = q[REL]r1[REL]r2,…,[ENT]e1,[ENT]e2,…

[0046] Input T all into the multi-task parameter-sharing encoder to obtain the encoded representation [h1, h2,…, h n corresponding to the question q, and the formula is as follows:

[0047] [h1, h2,…, h n = ENCODER(T all ) (11)

[0048] Then use the decoder to process the obtained encoded vectors h1, h2,…, h nDecode token by token into a structured query statement; during the training process, the structured query statement consists of m tokens {a1, a2, …, a m}, and the teacher forcing method is used for training, while cross-entropy is used as the loss function L Gen , and the formula is as follows:

[0049]

[0050]

[0051] where p k represents the probability distribution of the generated word at the k-th position generated by decoding.

[0052] A44. Jointly train three subtasks in A41, A42, and A43, and set the loss function according to formula (14):

[0053] L = L ENT + L REL + L Gen (14)

[0054] This setting of the loss function can not only supervise the structured query generation task, but also simultaneously supervise the entity disambiguation task and the relation classification task, enabling the structured query generation task to learn from the two auxiliary tasks of the entity disambiguation task and the relation classification task.

[0055] The beneficial effects of the present invention are as follows:

[0056] In this solution, the collaborative relationship between the three subtasks of entity linking, relation linking, and structured query generation in the problem semantic parsing process is considered. By using a multi-task learning framework to jointly train the model, the defects of mutual independence and error propagation existing in the traditional semantic parsing solution are overcome, the accuracy of the structured query generation task is improved, and the overall effect of the semantic parsing model is simultaneously enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart in an embodiment of the present invention.

[0058] Figure 2 It is a framework diagram of the multi-task learning model used in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The present invention aims to propose a complex problem semantic parsing method based on a knowledge base, which overcomes the defects of mutual independence and error propagation existing in the traditional solution, improves the accuracy of the structured query generation task, and simultaneously enhances the overall effect of the semantic parsing model. Its overall implementation process is as Figure 1As shown, the present invention first sets the background knowledge base KB in the current knowledge base question-answering task, and then obtains training data. Each training sample consists of a natural language question and its corresponding SPARQL query statement. The result of executing the SPARQL query statement in the knowledge base is the answer corresponding to the question. Then, for the input natural language question, the Bert-BiLSTM-CRF model is used to identify the entity mentions contained in the question sentence. Then, the mapping relationship in the FACC1 project is used to obtain the candidate entity set. Then, the dual-encoder relationship retrieval model is used to directly retrieve the candidate relationships from the knowledge base to obtain the candidate relationship set. Finally, a structured query statement is generated through multi-task learning, taking entity disambiguation and relationship classification as auxiliary tasks to form a multi-task learning model. The three sub-tasks share the same set of encoder parameters, and at the same time, each sub-task designs an independent layer on the basis of the encoder for training the corresponding sub-task. Finally, the generated structured query statement is used as the model output result, and the answer to the question is obtained by executing the query statement in the knowledge base.

[0060] Embodiment:

[0061] In this embodiment, the complex question semantic parsing method based on the knowledge base mainly includes training a complex question semantic parsing model and using the trained model to perform a question-answering task. The entire question-answering process is shown in Figure 1 , and specifically includes the following implementation steps:

[0062] S1. Set the knowledge base KB as the knowledge source used in the question-answering task, and then input the natural language question to be answered.

[0063] S2. For each natural language question, obtain the corresponding candidate entity set in the knowledge base.

[0064] In this step, for the input natural language question, the Bert-BiLSTM-CRF model is used to identify the entity mentions contained in the question sentence, and then the mapping relationship in the FACC1 project is used to obtain the candidate entity set.

[0065] Specifically, it includes the following steps:

[0066] S21. Perform word segmentation on the natural language question sentence, and obtain the embedding vector corresponding to each word in the segmented question sentence by using the pre-trained language model Bert encoder.

[0067] S22. Select to use the BiLSTM encoder to encode the question sentence in two directions, from left to right and from right to left, to obtain the hidden state vector containing the context information of the question sentence. At the same time, for the hidden state, a state transition matrix is used to consider the label constraint relationship between adjacent words, and finally the optimal annotation result is obtained to obtain the entity mentions in the question sentence.

[0068] S23. Construct a mapping dictionary from entity mentions to knowledge base entities according to the character matching method, and finally obtain a candidate entity set E of size K1 through the mapping i 。

[0069] S3. For each natural language question, obtain its corresponding candidate relationship set in the knowledge base.

[0070] In this step, design a combined dual-encoder relationship retrieval model to directly retrieve candidate relationships from the knowledge base, and obtain the candidate relationship set corresponding to the question. The specific steps are as follows:

[0071] S31. Preprocess the question q, and replace the entity mentions identified in the previous step with [MASK] tokens to obtain the processed question T q 。

[0072] S32. Use the pre-trained language model BERT to encode the question T q and the relationship r respectively to obtain their corresponding vector representations, and then use the dot product to calculate the correlation between the two. The calculation formula is as follows

[0073] Y q =BERTCLS(T q )

[0074] Y r =BERTCLS(r)

[0075] S(q,r)=Y q .Y r

[0076] According to the calculation results, select the top K2 relationships with the highest correlation and add them to the candidate relationship set to obtain the candidate relationship set R j 。

[0077] S4. Generate a structured query statement. During the generation of the structured query, entity disambiguation and relationship classification are used as auxiliary tasks to form a multi-task learning model. The model framework is as Figure 2 shown.

[0078] Specifically, the input of the entity disambiguation task is the natural language question and the candidate entity set, and the output is the entity set included in the question; the input of the relationship classification is the natural language question and the candidate relationship set, and the output is the relationship set included in the question; while the input of the structured query statement generation is the result of entity disambiguation, the result of relationship classification, and the natural language question, and the output is a structured query statement.

[0079] S41. Obtain the entity set corresponding to the question. The entity disambiguation task is completed by performing binary classification on the candidate entity set obtained in S2. First, process the entity e in the candidate entity set E i in it i , and splice it with the adjacent relationships in the knowledge base to obtain the following entity representation T containing more semantic knowledge information e :

[0080]

[0081] Then splice the question q and the entity representation , encode it using a multi-task parameter-sharing encoder, and obtain the similarity score between the question and the entity through a fully connected layer and an activation function. The formula is as follows:

[0082]

[0083] During the model training process, choose to use cross-entropy as the loss function. The formula is as follows:

[0084]

[0085] Among them, v i represents the label of the candidate entity e i , where 1 means e i is in the SPARQL query statement corresponding to the question q, and 0 means e i is not in the corresponding SPARQL query. K1 is the number of candidate entities; finally, add the entities recognized as positive samples in the entity disambiguation task to the entity set corresponding to the question to obtain the entity set E of the question q q .

[0086] S42. Obtain the relationship set corresponding to the question. The relationship classification task is completed by performing binary classification on the candidate relationship set obtained in S3. Splice the question q and the relationship r j and input it into the multi-task parameter-sharing encoder. After processing through the average pooling layer and the activation function, obtain the final score s(q, r j ), and the formula is as follows:

[0087]

[0088] Among them, u j represents the label of the candidate relationship r j , where 1 means r j is in the SPARQL query statement corresponding to the question Q, and 0 means r jNot in the corresponding SPARQL query, K2 is the number of candidate relationships; the relationships determined as positive samples in the relationship classification task are added to the relationship set, and finally the relationship set R of the question q is obtained q 。

[0089] S43. The input of the structured query generation task is the problem entity set E q , the relationship set R q and the question q, and the target output is a structured query. First, the question q is concatenated with E q and R q to obtain the question form T with auxiliary information all :

[0090] T all = q[REL]r1[REL]r2,…,[ENT]e1,[ENt]e2,…

[0091] Input T all into the multi-task parameter sharing encoder to obtain the encoded representation [h1, h2, …, h n corresponding to the question q. The formula is as follows:

[0092] [h1, h2, …, h n = ENCODER(T all )

[0093] Then use the decoder to decode the obtained encoded vectors h1, h2, …, h n token by token into a structured query statement; during training, the structured query statement is composed of m tokens {a1, a2, …, a m}, and teacher forcing is used for training, and cross-entropy is used as the loss function L Gen . The formula is as follows:

[0094]

[0095] where p k represents the probability distribution of the word generated at the k-th position decoded and generated.

[0096] S45. Jointly train the three subtasks in A41, A42, and A43, and set the loss function according to formula (14):

[0097] L = L ENT + L REL + L Gen (14)

[0098] The setting of this loss function can not only supervise the structured query generation task, but also simultaneously supervise the entity disambiguation task and the relation classification task, enabling the structured query generation task to learn from the two auxiliary tasks of the entity disambiguation task and the relation classification task.

[0099] S5. After obtaining the structured query statement through the decoder, execute the query statement in the knowledge base to obtain the answer to the question, and simultaneously output the entity set and relation set included in the question, which together serve as the semantic parsing result of the input question.

[0100] Although the present invention has been described herein with reference to embodiments of the present invention, the above embodiments are only preferred embodiments of the present invention, and the embodiments of the present invention are not limited by the above embodiments. It should be understood that those skilled in the art can design many other modifications and embodiments, and these modifications and embodiments will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A semantic parsing method for complex problems based on a knowledge base, characterized in that, This method directly retrieves candidate entity and relationship sets in the knowledge base dense space, and performs multi-task learning training on entity disambiguation and relationship classification as auxiliary tasks together with structured query generation; Specifically, this method includes the following steps: A. Train a multi-task learning model: A1. Set the background knowledge base KB in the current knowledge base question-answering task; then obtain training samples, each training sample consists of a natural language question and its corresponding SPARQL query statement, where the result of executing the SPARQL query statement in the knowledge base is the answer corresponding to the question; A2. Candidate entity retrieval: For the input natural language question, first identify the entity mentions included in the question sentence, and then obtain the candidate entity set through the mapping relationship; A3. Candidate relationship retrieval: Retrieve candidate relationships from the knowledge base to finally obtain the candidate relationship set; A4. Generate a structured query statement through the multi-task learning model: During the process of structured query generation, entity disambiguation and relationship classification are used as auxiliary tasks to form a multi-task learning model; among them, the input of the entity disambiguation task is the natural language question and the candidate entity set obtained in A2, and the output is the entity set included in the question; the input of the relationship classification task is the natural language question and the candidate relationship set obtained in A3, and the output is the relationship set included in the question; the input of the structured query generation task is the result of the entity disambiguation task, the result of the relationship classification task, and the natural language question, and the output is a structured query statement; A5. Loop through steps A2 - A4 to iteratively train the multi-task learning model until the preset number of training times is reached or the model has converged; B. Perform the semantic parsing task of complex questions: Use the multi-task learning model trained in step A for the actual semantic parsing task of complex questions; specifically, use the natural language question as the input, obtain the structured query statement corresponding to the question based on the trained multi-task learning model, and obtain the answer to the question by executing the query statement in the knowledge base, and at the same time output the entity set and relationship set included in the question.

2. The semantic parsing method for complex problems based on a knowledge base according to claim 1, characterized in that, In the multi-task learning model, the three sub-tasks share the same set of encoder parameters, and at the same time, each sub-task designs an independent training layer based on the encoder for training the corresponding sub-task, and finally takes the generated structured query statement as the output result of the multi-task learning model.

3. The semantic parsing method for complex problems based on a knowledge base according to claim 2, characterized in that, In step A2, the candidate entity retrieval specifically includes: A21. Perform word segmentation on the natural language question sentence, and use the Bert encoder to obtain the embedding vector corresponding to each word in the word-segmented natural language question sentence; A22. Use the BiLSTM encoder to encode the natural language question sentence from two directions to obtain the hidden state vector containing the context information of the natural language question sentence; then through the CRF layer, use the state transition matrix to consider the label constraint relationship of adjacent words, and finally obtain the optimal annotation result to obtain the entity mentions in the question; A23. Construct the mapping relationship between entity mentions and knowledge base entities according to the character matching method, and finally obtain a candidate entity set E of size K1 through the mapping relationship i .

4. The semantic parsing method for complex problems based on a knowledge base according to claim 3, characterized in that, Step A 3. The candidate relationship retrieval specifically includes: A31. Preprocess the natural language question q, and replace the entity mentions identified in step A22 with [MASK] tokens to obtain the processed question T q ; A32. Encode the question sentence T using two encoders respectively q and the relation r, and calculate the correlation S(q, r) between the two using the dot product. The calculation formula is as follows: Y q = BERTCLS(T q ) (1) Y r = BERTCLS(r) (2) S(q,r) = Y q .Y r (3) Among them, BERTCLS represents the output vector of the [CLS] token in the BERT encoding process, and Y q is the vector representation of the question T q , and Y r is the vector representation of the relation r; the relevance score between the question T q and the relation r is calculated by dot product, denoted as S(q, r), and the K2 relations with the highest relevance scores are selected and added to the candidate relation set to obtain the candidate relation set R j .

5. The semantic parsing method for complex problems based on a knowledge base according to claim 4, characterized in that In step A4, generating a structured query statement through the multi-task learning model specifically includes: A41. Complete the entity disambiguation task: complete the entity disambiguation task by performing binary classification on the candidate entity set obtained in A2; specifically, first process the entity e in the candidate entity set E i in i , and splice the entity e i with the adjacent relationships r1, r2,..., r n in the knowledge base to obtain the following entity representation containing more semantic knowledge information Then, the natural language question q and the entity representation are concatenated, encoded using a multi-task parameter-sharing encoder, and after passing through a fully connected layer and an activation function, the similarity score S(q, e i ) between the question q and the entity e i ) is obtained. The formula is as follows: Among them, represents the vector representation obtained by concatenating the question q and the entity e i During the training process, cross-entropy is used as the loss function L ENT , and the formula is as follows: Among them, v i represents the label of the candidate entity e i where 1 means e i is in the SPARQL query statement corresponding to the question q, and 0 means e i is not in the corresponding SPARQL query. K1 is the number of candidate entities; finally, the entities recognized as positive samples in the entity disambiguation task are added to the entity set corresponding to the question to obtain the entity set E q ; A42. Complete the relationship classification task: By performing binary classification on the relationship r j in the candidate relationship set R obtained in A3 j to complete the relationship classification task; specifically, concatenate the question q and the relationship r j and input them into a multi-task parameter-sharing encoder, and obtain the final score s(q, r j ) after processing by the average pooling layer and the activation function. The formula is as follows: Among them, represents the vector representation obtained by splicing the question q and the relationship r j During the training process, binary cross-entropy is used as the loss function L REL , and the formula is as follows: where u j represents the label of the candidate relationship r j where 1 represents r j in the SPARQL query statement corresponding to the question q, 0 represents r j not in the corresponding SPARQL query, and K2 is the number of candidate relationships; add the relationships judged as positive samples in the relationship classification task to the relationship set, and finally obtain the relationship set R of the question q q ; A43. Complete the structured query generation task: the entity set E of the input question q q , the relationship set R q and the question q, and the target output is a structured query; First, splice the question q with E q and R q to obtain the question representation T with auxiliary information all : T all = q[REL]r1[REL]r2,…,[ENT]e1,[ENT]e2,… Input T all into the multi-task parameter sharing encoder to obtain the encoded representation [h1, h2, …, h n corresponding to the question q. The formula is as follows: [h1,h2,…,h n = ENCODER(T all ) (11) Next, the decoder is used to decode the obtained encoded vectors h1, h2, …, h n token by token into a structured query statement; during the training process, the structured query statement is composed of m tokens {a1, a2, …, a m}, and the teacher forcing method is used for training, while the cross-entropy is adopted as the loss function L Gen , and the formula is as follows: where p k represents the probability distribution of the generated word at the k-th position generated by decoding; Jointly train three subtasks in A41, A42, and A43, and set the loss function according to formula (14): L = L ENT + L REL + L Gen (14) The setting of this loss function can not only supervise the structured query generation task, but also simultaneously supervise the entity disambiguation task and the relation classification task, enabling the structured query generation task to learn from the two auxiliary tasks of the entity disambiguation task and the relation classification task.

Images