Knowledge enrichment type question generation method and device for question and answer system robustness
By identifying the entity to be queried, obtaining factual descriptions from the knowledge base, and generating synonyms, combined with editing mechanisms and dynamic weight optimization, the problems of insufficient question diversity and robustness in question-answering systems are solved, generating high-quality, understandable questions and improving the performance of question-answering models.
Patent Information
- Application Number
- CN202211239666.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-10-11
- Publication Date
- 2026-01-02
- Estimated Expiration
- 2042-10-11
AI Technical Summary
Existing question enhancement methods in question-answering systems limit the diversity of questions, fail to effectively improve robustness and generalization ability, and data-driven and knowledge-based methods have failed to significantly improve question representation.
By identifying the entity to be queried in the input question, obtaining factual descriptions, definitions, and synonyms from the knowledge base, generating knowledge-rich questions using an editing mechanism, and dynamically adjusting the attention weights of the injected knowledge through a detector and booster, the performance of the question answering model is optimized.
It generates more diverse and meaningful questions that can be understood by humans, and improves the robustness and performance of the question-answering model. Dynamically adjusting weights makes the model pay more attention to relevant information and reduce the influence of irrelevant information.
Smart Images
Figure CN117909442B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of question generation of question and answer systems, and particularly relates to a knowledge-rich question generation method and device for robustness of question and answer systems. BACKGROUND
[0002] Question and answer systems have great application value in many scenarios, such as search engines, chat robots, online customer service, etc., but their robustness is still greatly restricted. One method to improve the robustness of question and answer systems is question enhancement, that is, generating diverse questions to improve the generalization ability and robustness of question and answer systems.
[0003] The commonly used question enhancement method is data-driven, such as inserting or replacing words in the question to generate semantically equivalent questions as enhancement samples, or using back-translation to generate enhancement samples, or embedding discrete text into continuous space to rewrite the question as an enhancement sample, or using a generative model to globally generate enhancement samples from scratch. However, question and answer systems can learn the surface patterns in these questions to correctly answer the questions, rather than developing "true" language understanding ability. This also limits the diversity of questions, and the effect of question enhancement is not obvious.
[0004] With the background knowledge found to have great value in question enhancement, more and more research tends to use background knowledge as supporting facts to construct knowledge-based question and answer pairs. They mostly use generative models to generate questions and answers from scratch with the support of knowledge bases to increase the diversity of questions. However, these question and answer pairs are mostly not the same as the given question and answer pairs, and there is no further improvement in question expression. The knowledge base is only used as a knowledge source for question and answer pairs, which also limits the effect of question enhancement.
[0005] As described above, the data-driven question enhancement method can prompt the question and answer system to learn the surface patterns in the questions to correctly answer the questions, rather than developing "true" language understanding ability. The question and answer pairs generated with the support of knowledge bases are mostly not the same as the given question and answer pairs, and there is no further improvement in question expression. Both methods limit the diversity of questions, and the effect of question enhancement is not obvious. SUMMARY
[0006] The application is made to solve the above problems, and aims to provide a question generation method capable of improving the diversity of question expression and the performance and robustness of question and answer models. The application adopts the following technical solutions:
[0007] The application provides a knowledge-rich question generation method for robustness of question and answer systems, characterized by the following steps:
[0008] Step S1, identifying the to-be-queried entity in the input question, and searching in the knowledge base to obtain the factual description of the to-be-queried entity;
[0009] Step S2, distilling the factual description, and taking the distilled factual description as injected knowledge;
[0010] Step S3, using a dictionary to interpret the general word in the input question to obtain its paraphrase, and finding the synonym of the general word, both of which are taken as injected knowledge;
[0011] Step S4, using the injected knowledge and the editing mechanism, and adopting any one or a combination of multiple ways to generate a knowledge-rich question:
[0012] injecting the distilled factual description into the input question;
[0013] replacing the general word in the input question with its paraphrase;
[0014] replacing the general word in the input question with its synonym;
[0015] Step S5, detecting the knowledge level attention weight of each injected knowledge in the knowledge-rich question through a detector, the detector comprising an encoder-decoder model same as the structure of the question and answer model;
[0016] Step S6, sequentially removing the injected knowledge with the highest knowledge level attention weight, and generating a new question using the remaining injected knowledge, inputting the new question into the question and answer model, and adjusting the knowledge level attention weight of the removed injected knowledge according to the performance change of the question and answer model.
[0017] The knowledge-rich question generation method for the robustness of the question and answer system provided by the application can also have the following technical features, wherein in step S1, the to-be-queried entity is identified through a TAGME tool, and the entity output by the TAGME with a confidence greater than a predetermined value is taken as the to-be-queried entity.
[0018] The question and answer system robustness knowledge enrichment type question generation method provided by the application can also have the following technical features: in step S2, first, a triple (h, r, t) is extracted from the fact description, wherein h, r and t represent a head entity, a relationship and a tail entity respectively, then co-reference resolution is performed on the fact description using a NeuralCoref tool to extract more triples, in step S4, the triples are organically injected into the input question by using a grammar rule to generate a fact enrichment type question, if all the triples in the fact enrichment type question are related to the corresponding to-be-queried entity in the input question, the fact enrichment type question is a positive example, and the answer of the fact enrichment type question is the same as that of the corresponding input question, otherwise, the fact enrichment type question is a negative example and is marked as unanswerable.
[0019] The question and answer system robustness knowledge enrichment type question generation method provided by the application can also have the following technical features: in step S5, the encoder-decoder model comprises a word-level encoder and a knowledge-level encoder, both of which adopt a hierarchical attention network and are respectively used to obtain a word-level attention weight of each word in the knowledge enrichment type question and a knowledge-level attention weight of each injected knowledge.
[0020] The question and answer system robustness knowledge enrichment type question generation method provided by the application can also have the following technical features: the detector further comprises a word embedding unit, BERT is used for word embedding, the word-level encoder and the knowledge-level encoder both adopt a multi-layer multi-bidirectional GRU, first, word embedding is performed by using the word embedding unit to obtain a semantic representation of each word in the knowledge enrichment type question, then, a hidden vector is obtained by using the word-level encoder to obtain the word-level attention weight of each word:
[0021]
[0022] In the formula, alpha ij is a word-level attention weight of a jth word (denoted as k ij ) in an ith injected knowledge, x ij is word embedding of k ij , h ij is an output of a word-level bidirectional GRU network hidden state, f1() is an activation function, e ij is a hidden vector of k ij , s i is an output of a word-level attention layer, a hidden vector is obtained by using the knowledge-level encoder to obtain the knowledge-level attention weight of each injected knowledge:
[0023]
[0024] wherein A i is the knowledge level attention weight of the i-th injected knowledge, e i is the hidden vector of s i , h i is the output of the knowledge level bidirectional GRU network hidden state, f2() is an activation function, and c is the output of the knowledge level attention layer.
[0025] The knowledge-rich question generation method for the robustness of the question and answer system provided by the application can also have the following technical features: in step S6, the new question is input into the question and answer model that has been fine-tuned on a given question and answer pair to detect the performance of the question and answer model on the new question, if the performance becomes better, the removed injected knowledge is considered as negative knowledge, and the knowledge level attention weight between the negative knowledge and the correct relationship corresponding to the entity to be queried is reduced; if the performance becomes worse, the removed injected knowledge is considered as positive knowledge, and the knowledge level attention weight between the positive knowledge and the correct relationship corresponding to the entity to be queried is increased.
[0026] The knowledge-rich question generation method for the robustness of the question and answer system provided by the application can also have the following technical features: in step S6, the knowledge level attention weight is optimized by using a factor δ, which represents the change amount of the probability value of the detector when decoding the correct relationship corresponding to the entity, when one of the injected knowledge exists in the knowledge-rich question, the matching probability of the correct relationship corresponding to the entity to be queried obtained by decoding is After removing the injected knowledge and generating the new question by using the remaining injected knowledge, the matching probability drops to The change amount of the probability value is Then the knowledge level attention weight of the injected knowledge is updated to A' i The context vector input into the decoder is updated to c':
[0027]
[0028] wherein λ is a hyperparameter adjusted through experiments.
[0029] The knowledge-rich question generation method for the robustness of the question and answer system provided by the application can also have the following technical features: in step S6, the encoder-decoder model is retrained by using a multi-task learning method according to the adjusted knowledge level attention weight, and two loss functions are optimized at the same time, one of the loss functions is used for the knowledge level attention weight optimization task, and the other loss function is used for the knowledge base question and answer task based on the knowledge-rich question:
[0030]
[0031] In the formula, φ1 and φ2 respectively represent weight factors of loss functions of two tasks, β is 0 when the current position is 'PAD', and β is 1 otherwise, is the cross-entropy loss of the knowledge base question answering task based on the knowledge-rich question under different question answering models, p(r1,...,r T |q i ) is the probability distribution of the candidate relationship.
[0032] The application provides a knowledge-rich question generation device for robustness of a question answering system, characterized by comprising: an entity description module, configured to identify a to-be-queried entity in an input question, and search a fact description of the to-be-queried entity in a knowledge base; a fact distillation module, configured to distill the fact description searched by the entity description module to obtain a distilled fact description as injected knowledge; a word explanation module, configured to use a dictionary to explain a general word in the input question to obtain a paraphrase of the general word, and find a synonym of the general word, both of which are used as injected knowledge; a knowledge-rich question generation module, configured to use the injected knowledge to generate a knowledge-rich question in any one or a combination of the following ways: injecting the distilled fact description into the input question; replacing the general word in the input question with the paraphrase thereof; replacing the general word in the input question with the synonym thereof; a detector module, configured to detect a knowledge level attention weight of each of the injected knowledge in the knowledge-rich question, and having an encoder-decoder model same as a question answering model structure; and a booster module, configured to sequentially remove the injected knowledge with the highest knowledge level attention weight, and generate a new question using the remaining injected knowledge, input the new question into the question answering model, and adjust the knowledge level attention weight of the removed injected knowledge according to a performance change of the question answering model.
[0033] Inventive action and effect
[0034] According to the knowledge-rich question generation method and device for robustness of a question answering system, the fact description, the paraphrase and the synonym of the to-be-queried entity after distillation are obtained as injected knowledge, the existing question is rewritten using an editing mechanism to generate a knowledge-rich question, different types of knowledge can be used to expand the original question without changing the meaning of the original question, and more diversified and more meaningful knowledge-rich questions can be generated. The editing mechanism can easily generate high-quality questions, the generated questions are meaningful, can be understood by humans, and can meet the requirement of verifying the correctness of the facts. In addition, the position and content of the editing are known, which is conducive to the question answering system to specifically strengthen its weaknesses.
[0035] Further, the present application also inspiringly provides "diagnosis" information for the question and answer model, provides a dynamic weight for each injected knowledge, which makes the question and answer model pay more attention to the question part containing clue information to predict the correct answer, and pay less attention to the question part containing irrelevant information, so that the performance of the question and answer model on knowledge-rich questions and original questions can be effectively improved by dynamically adjusting the weight. In addition, unlike the gradient-based training method, the present application does not implicitly adjust the gradient in the training step, but modifies the attention weight of the removed injected knowledge according to the performance change of the question and answer model after removing the injected knowledge, so that the optimization of the weight is more controllable and explicit. BRIEF DESCRIPTION OF DRAWINGS
[0036] Figure 1 is a flowchart of the knowledge-rich question generation method for the robustness of the question and answer system in the embodiment of the present application;
[0037] Figure 2 is a framework diagram of the knowledge-rich question generation in the embodiment of the present application;
[0038] Figure 3 is a framework diagram of the model improvement module in the embodiment of the present application;
[0039] Figure 4 is a structural block diagram of the knowledge-rich question generation device for the robustness of the question and answer system in the embodiment of the present application. DETAILED DESCRIPTION
[0040] In order to make the technical means, creative features, purposes and effects realized by the present application easy to understand, the knowledge-rich question generation method and device for the robustness of the question and answer system are specifically described below in combination with embodiments and drawings.
[0041] <EMBODIMENT>
[0042] Figure 1 is a flowchart of the knowledge-rich question generation method for the robustness of the question and answer system in the embodiment of the present application.
[0043] As shown in Figure 1 , the knowledge-rich question generation method for the robustness of the question and answer system includes the following steps:
[0044] Step S1, identifying the query entity in the input question, searching the factual description of the query entity in the knowledge base.
[0045] Step S2, distilling the corresponding factual description searched, and taking the distilled factual description as injected knowledge.
[0046] Step S3, the general words in the input question are interpreted using a dictionary to obtain paraphrases and find their synonyms, and the paraphrases or synonyms are used as injected knowledge.
[0047] Step S4, using the injected knowledge and the editing mechanism, the knowledge-rich question is generated in one or a combination of the following ways.
[0048] injecting the distilled fact description into the input question;
[0049] replacing the general words in the input question with their paraphrases;
[0050] replacing the general words in the input question with their synonyms.
[0051] Step S5, the knowledge level attention weight of each injected knowledge in the knowledge-rich question is detected by a detector, which contains an encoder-decoder model identical to the structure of the question and answer model.
[0052] Step S6, the injected knowledge with the highest knowledge level attention weight is removed in sequence, and a new question is generated using the remaining injected knowledge, which is input into the question and answer model, and the knowledge level attention weight of the removed injected knowledge is adjusted according to the performance change of the question and answer model.
[0053] The above steps will be described in detail below.
[0054] Figure 2 is the framework diagram of the generation of the knowledge-rich question in this embodiment.
[0055] As shown in Figure 2 , in this embodiment, the input question as an example is "Which fictional universe does david north appear in?", and the answer to this question is "American comic books".
[0056] Step S1, the entity description module identifies the query entity in the input question, and searches for the fact description of the query entity in the knowledge base as the output.
[0057] As shown in Figure 2 (a), in this embodiment, the entity description module first identifies the query entities "fictional universe" and "david north" in the input question through the TAGME tool, and to improve the accuracy of entity recognition, the entities with a confidence greater than 0.5 output by TAGME are used as query entities.
[0058] Then, for each entity to be queried, the corresponding Wikipedia text is crawled and treated as a factual description of the entity to be queried. For example, the factual description corresponding to "fictional universe" is "A fictional universe, or fictional world, is a self-consistent setting with events, and often other elements, that different from the real world…".
[0059] In step S2, the fact distillation module distills the corresponding fact descriptions retrieved from the query and uses the distilled fact descriptions as injected knowledge.
[0060] like Figure 2 As shown in (b), the distillation process extracts triples (h, r, t) from each fact description obtained from the search, where h, r, and t represent the head entity, relation (usually a verb or verb phrase), and tail entity, respectively. These triples are concise forms of the fact descriptions, removing noise and preserving the essential information from the fact description of the entity to be queried. Next, the NeuralCoref tool is used to perform coreference resolution on the fact descriptions to extract more triples. Then, these triples are organically injected into the input question using grammatical rules to generate multiple fact-rich questions. Finally, the answers to these questions are given to generate fact-rich question-answer pairs. If all triples in a fact-rich question are relevant to the entity to be queried in the input question, the fact-rich question is marked as a positive example, and its answer is the same as the corresponding input question; otherwise, the fact-rich question is marked as a negative example, i.e., unanswerable.
[0061] For example, after organically injecting the distilled factual description corresponding to "fictional universe" into the input question, the generated fact-rich question is "Which fictional universe, that is self-consistent setting with events differing from the real world, does David North appear in?".
[0062] In step S3, the word explanation module uses a dictionary to explain the common words in the input question and find their synonyms, all of which are injected as knowledge.
[0063] like Figure 2(c) as shown, the general word is defined as all the words in the input question except the question mark, the function word, the punctuation mark and the entity to be queried. In this embodiment, first, the paraphrase and the synonym of the general word are selected in the Cambridge dictionary, and the general word is defined as all the words in the input question except the question mark, the function word, the punctuation mark and the entity to be queried. Then, the synonym and the paraphrase of each general word are used to replace the general word respectively to generate the synonym-rich question and the paraphrase-rich question. When the general word is a non-ambiguous word, the answer of the generated question is the same as the corresponding input question. When the general word is an ambiguous word, word sense disambiguation is performed using a pre-trained language model to select the most context-related paraphrase of each general word in the dictionary.
[0064] Specifically, in this embodiment, the cosine similarity of the word embedding is used to calculate the similarity between the input question containing the ambiguous general word and the example sentence of each paraphrase in the dictionary. The higher the similarity, the more similar the paraphrase of the ambiguous general word in the input question to the paraphrase corresponding to the example sentence. Then, the paraphrase of each ambiguous general word is injected into the input question to generate different paraphrase-rich questions, and the perplexity of each question is calculated. The lower the perplexity, the more suitable the paraphrase of the ambiguous general word in the context of the paraphrase-rich question. We weight and fuse the similarity and the perplexity of each paraphrase to obtain the final score. Then, we inject the paraphrase with the highest final score into the input question as the positive example of the paraphrase-rich question, and the answer of the question is the answer of the corresponding input question. We also inject other paraphrases of the ambiguous word into the input question as negative examples of the paraphrase-rich question, and these questions are marked as unanswerable.
[0065] By injecting the paraphrase or the synonym into the input question as the injected knowledge, the paraphrase-rich question and the synonym-rich question can be generated. For example, the general word “appear” is identified, and its synonym “show” and its paraphrase “perform publicly” are found, and the two words are used to replace “appear” in the original question respectively.
[0066] In step S4, the injected knowledge and the editing mechanism are used to generate the knowledge-rich question in any one or a combination of the following ways:
[0067] The distilled fact description is organically injected into the original input question;
[0068] The general word in the input question is replaced by its paraphrase.
[0069] The general word in the input question is replaced by its synonym.
[0070] In this embodiment, the above-mentioned fact-rich questions, synonym-rich questions and explanation-rich questions are generated separately by the above-mentioned methods, and the knowledge-rich questions generated by the combination of the above-mentioned three methods are generated, for example: "which fictional universe, that is self-consistent setting with events differing from real world, does the fictional mutant character david north perform publicly in?".
[0071] In step S5, the knowledge level attention weight of each injected knowledge is detected by a detector, which includes an encoder-decoder model with the same structure as the question and answer model.
[0072] As shown in Figure 3 (a), the detector is used to detect the weaknesses of the question and answer model in processing different types of knowledge in the generated question. The reason for the poor performance of the question and answer system is that the relationship prediction performance of the entity corresponding is poor, and the reason for the poor relationship prediction performance is that the question and answer model assigns unreasonable attention weight to the injected knowledge. Therefore, the present application designs a corresponding detector to detect the matching degree between the injected knowledge and the correct relationship corresponding to the entity. The detector includes a word embedding unit, an encoder-decoder model, wherein the word embedding unit adopts a fine-tuned BERT, and the encoder includes a word level encoder and a knowledge level encoder, both of which adopt a hierarchical attention network (multi-layer bidirectional GRU), that is, including word level attention and knowledge level attention. First, the fine-tuned BERT is used for word embedding to obtain the semantic representation of each word in the knowledge-rich question. Then, the multi-layer bidirectional GRU is used to obtain the hidden vector to obtain the word level attention weight of each word:
[0073]
[0074] In the formula, α ij is the word level attention weight of the jth word (denoted as k ij ) in the ith injected knowledge, x ij is the word embedding of k ij , h ij is the output of the word level bidirectional GRU network hidden state, f1() is an activation function, e ij is the hidden vector of k ij , and s i is the output of the word level attention layer.
[0075] In a similar manner, the knowledge level attention weight of each injected knowledge can be obtained:
[0076]
[0077] In the formula, A i For the knowledge level attention weight of the i-th injected knowledge, e i 'for s i The latent vector, h i ' represents the output of the hidden state of the knowledge-level bidirectional GRU network, f2() is the activation function, and c is the output of the knowledge-level attention layer.
[0078] Next, word embeddings based on candidate relations (denoted as E) t-1 The latent vector of the decoder is obtained. t And calculate the probability distribution p(r1, ..., r) of the generated candidate relations. T |q i ):
[0079] o t =f3(c,o) t-1 E t-1 p(r1, ..., r) T |q i ) = softmax(o t E t-1 )
[0080] Step S6: Remove the injected knowledge with the highest knowledge level attention weight in sequence, generate a new question using the remaining injected knowledge, input the new question into the question answering model, and dynamically adjust the knowledge level attention weight of the removed injected knowledge according to the performance changes of the question answering model.
[0081] The injected knowledge with the highest knowledge-level attention weight is removed sequentially, and new knowledge-enriched questions are generated using the remaining injected knowledge to select the knowledge most closely related to the correct entity. The newly generated knowledge-enriched questions are then input into a question-answering model that has been fine-tuned on a given question-answer pair to test the model's performance on the new questions. If performance improves, the removed knowledge is considered negative knowledge, meaning it should be given less knowledge-level attention weight. Otherwise (i.e., performance deteriorates), the removed knowledge is considered positive knowledge, meaning it should be given more knowledge-level attention weight.
[0082] like Figure 3 As shown in (b), the booster improves the performance of the question-answering model by dynamically optimizing the attention weights of the knowledge level. Optimizing the attention weights of the knowledge level is achieved by reducing the attention weight between negative knowledge and the correct relation corresponding to the query entity, and increasing the attention weight between positive knowledge and the correct relation corresponding to the query entity.
[0083] In this embodiment, a factor δ is designed to optimize the knowledge level attention weight, which represents the change in probability value when the detector decodes the correct relationship corresponding to the entity. If an injected knowledge exists in the question, the matching probability of the correct relationship corresponding to the decoded entity is initially After removing this injected knowledge and generating a new question using the remaining injected knowledge, the matching probability drops to Then the change in probability value is In this case, the knowledge level attention weight of this injected knowledge is updated to A' i The context vector input into the decoder is updated to c' as follows:
[0084]
[0085] where λ is a hyperparameter adjusted through experiments.
[0086] Then, the encoder-decoder model in the detector is retrained according to the knowledge level attention weight optimized by the booster, and the masked cross-entropy is used as the loss function, i.e., the loss of the "PAD" position is filtered out.
[0087] In this embodiment, a multi-task learning method is adopted, i.e., two loss functions are optimized simultaneously, one of which is used for the knowledge level attention weight optimization task, and the other of which is used for the knowledge base question answering task based on the knowledge-rich question:
[0088]
[0089] where φ1 and φ2 represent the weight factors of the loss functions of the two tasks, respectively; β is 0 when the current position is "PAD", otherwise, β is 1; is the cross-entropy loss of the knowledge base question answering task based on the knowledge-rich question under different question answering models.
[0090] Figure 4 is the structural block diagram of the knowledge-rich question generation device for the robustness of the question answering system in this embodiment.
[0091] As shown in Figure 4 The embodiment also provides a knowledge-rich question generation device for the robustness of the question answering system 10 corresponding to the above method, which comprises an entity description module 11, a fact distillation module 12, a word explanation module 13, a knowledge-rich question generation module 14, a detector module 15, a booster module 16, and a control module 17.
[0092] The entity description module 12 is used to identify the query entity in the input question, and search for the fact description of the query entity in the knowledge base as the output.
[0093] The fact distillation module 12 is configured to distill the fact description searched by the entity description module 11 to obtain a distilled fact description as injected knowledge.
[0094] The word explanation module 13 is configured to explain a general word in the input question by using a dictionary to obtain a paraphrase of the general word and find a synonym of the general word, both of which are injected knowledge.
[0095] The knowledge-enriched question generation module 14 is configured to generate a knowledge-enriched question by using the injected knowledge in any one or a combination of the following ways:
[0096] injecting the distilled fact description into the input question;
[0097] replacing the general word in the input question with the paraphrase of the general word;
[0098] replacing the general word in the input question with the synonym of the general word.
[0099] The detector module 15 is configured to detect the knowledge level attention weight of each injected knowledge in the generated knowledge-enriched question, and has an encoder-decoder model same as the question and answer model structure.
[0100] The booster module 16 is configured to sequentially remove the injected knowledge with the highest knowledge level attention weight, and generate a new knowledge-enriched question by using the remaining injected knowledge, input the new question into the question and answer model, and adjust the knowledge level attention weight of the removed injected knowledge according to the performance change of the question and answer model.
[0101] The control module 17 is configured to control the work of each module.
[0102] In the embodiment, the parts not described in detail are known in the art.
[0103] Effects of the embodiment
[0104] According to the knowledge-enriched question generation method and device for a robust question and answer system provided in the embodiment, the distilled fact description, paraphrase and synonym of the entity to be queried are obtained as injected knowledge, and the existing question is rewritten by using the editing mechanism to generate a knowledge-enriched question, so that the original question can be expanded by using different types of knowledge without changing the meaning of the original question, and more diversified and meaningful knowledge-enriched questions can be generated. The editing mechanism can easily generate high-quality questions, and the generated questions are meaningful, can be understood by humans, and can meet the requirement of verifying the correctness of the fact. In addition, the position and content of the editing are known, which is beneficial to the question and answer system to specifically strengthen its weaknesses.
[0105] Further, the booster module of the embodiment also provides "diagnostic" information for the question and answer model, and provides a dynamic weight for each injected knowledge, which makes the question and answer model pay more attention to the question part containing clue information to predict the correct answer, and pay less attention to the question part containing irrelevant information, so that the performance of the question and answer model on knowledge-rich questions and original questions can be effectively improved by dynamically adjusting the weight. In addition, unlike gradient-based training, the booster module does not implicitly adjust the gradient in the training step, but modifies the attention weight of the removed injected knowledge according to the performance change of the question and answer model after the injected knowledge is removed, so that the optimization of the weight is more controllable and explicit.
[0106] The above embodiments are only used to illustrate the specific implementation of the present application, and the present application is not limited to the description range of the above embodiments.
Claims
1. A knowledge-rich question generation method for robustness of a question-answering system, characterized in that, The method comprises the following steps: Step S1, identifying the to-be-queried entity in the input question, and searching the knowledge base to obtain a factual description of the to-be-queried entity; Step S2, distilling the factual description, and taking the distilled factual description as injected knowledge; Step S3, using a dictionary to interpret a general word in the input question to obtain its paraphrase, and finding a synonym of the general word, both of which are taken as injected knowledge; Step S4, using the injected knowledge and an editing mechanism, and generating a knowledge-rich question in any one or a combination of the following ways: injecting the distilled factual description into the input question; replacing the general word in the input question with its paraphrase; replacing the general word in the input question with its synonym; Step S5, detecting the knowledge level attention weight of each injected knowledge in the knowledge-rich question by a detector, the detector comprising an encoder-decoder model identical to the structure of a question and answer model; Step S6, sequentially removing the injected knowledge with the highest knowledge level attention weight, and generating a new question using the remaining injected knowledge, inputting the new question into the question and answer model, and adjusting the knowledge level attention weight of the removed injected knowledge according to the performance change of the question and answer model, wherein in step S6, the new question is input into the question and answer model that has been fine-tuned on a given question and answer pair to detect the performance of the question and answer model on the new question, if the performance becomes better, regarding the removed injected knowledge as negative knowledge, and reducing the knowledge level attention weight between the negative knowledge and the correct relationship corresponding to the to-be-queried entity; if the performance becomes worse, regarding the removed injected knowledge as positive knowledge, and increasing the knowledge level attention weight between the positive knowledge and the correct relationship corresponding to the to-be-queried entity, in step S6, using a factor δ to optimize the knowledge level attention weight, the factor δ representing the change amount of the probability value of the detector when decoding the correct relationship corresponding to the entity, When one of the injected knowledge exists in the knowledge-rich question, the matching probability of decoding the correct relationship corresponding to the entity to be queried is , the matching probability is reduced to after removing the injected knowledge and generating the new question using the remaining injected knowledge, and the change amount of the probability value is , the knowledge level attention weight of the injected knowledge is updated to , and the context vector as the decoder input is updated to : ， In the formula, are hyperparameters adjusted through experiments, in step S6, further retraining the encoder-decoder model using a multi-task learning method according to the adjusted knowledge level attention weight, and optimizing two loss functions, one of which is used for a knowledge level attention weight optimization task, and the other of which is used for a knowledge base question and answer task based on a knowledge-rich question: ， wherein, and respectively represent the weight factors of the loss functions of the two tasks, the current position is "PAD" is 0, otherwise is 1, is the cross-entropy loss of the knowledge base question answering task based on the knowledge-rich question under different question answering models, is the probability distribution of the candidate relationship.
2. The knowledge-rich question generation method for the robustness of a question and answer system according to claim 1, characterized in that: wherein in step S1, the to-be-queried entity is identified by a TAGME tool, and entities with a confidence value greater than a predetermined value output by the TAGME are taken as the to-be-queried entity.
3. The knowledge-rich question generation method for the robustness of a question and answer system according to claim 1, characterized in that: wherein in step S2, first extracting a triple (h, r, t) from the factual description, wherein h, r, and t represent a head entity, a relationship, and a tail entity, respectively, Then, the NeuralCoref tool is used to perform coreference resolution on the fact description to extract more triples, In step S4, the triples are organically injected into the input question by using syntax rules to generate a fact-rich question, If all the triples in the fact-rich question are related to the corresponding query entity in the input question, the fact-rich question is a positive example, and its answer is the same as that of the corresponding input question, Otherwise, the fact-rich question is a negative example and is marked as unanswerable.
4. The knowledge-rich question generation method for robustness of a question-answering system according to claim 1, characterized in that: wherein In step S5, the encoder-decoder model comprises a word-level encoder and a knowledge-level encoder, both of which use hierarchical attention networks to obtain the word-level attention weight of each word in the knowledge-rich question and the knowledge-level attention weight of each injected knowledge, respectively.
5. The knowledge-rich question generation method for robustness of a question-answering system according to claim 4, characterized in that: wherein The detector further comprises a word embedding unit that uses BERT for word embedding, Both the word-level encoder and the knowledge-level encoder use multi-layer multi-bi GRU, First, the word embedding unit is used for word embedding to obtain the semantic representation of each word in the knowledge-rich question, and then the word-level encoder is used to obtain a hidden vector to obtain the word-level attention weight of each word: ， where α ij is the word-level attention weight for the i-th injected knowledge and the j-th word k ij , x ij is the word embedding of k ij , h ij is the output of the word-level bi-directional GRU network hidden state, f1() is an activation function, e ij is the hidden vector of k ij , s i is the output of the word-level attention layer, The knowledge-level encoder is used to obtain a hidden vector to obtain the knowledge-level attention weight of each injected knowledge: ， In the formula, A i is the knowledge level attention weight of the i-th injected knowledge, e i is the hidden vector of s i , h i is the output of the knowledge level bidirectional GRU network hidden state, f2() is an activation function, and c is the output of the knowledge level attention layer.
6. A knowledge-rich question generation apparatus for robustness of a question-answering system, characterized by, It comprises: An entity description module for identifying a query entity in an input question and searching for a fact description of the query entity in a knowledge base; A fact distillation module for distilling the fact description searched by the entity description module to obtain the distilled fact description as injected knowledge; A word explanation module for using a dictionary to explain a general word in the input question to obtain its paraphrase and find a synonym of the general word, both of which are used as injected knowledge; A knowledge-rich question generation module for using the injected knowledge to generate a knowledge-rich question in any one or a combination of multiple ways: Injecting the distilled fact description into the input question; Replacing the general word in the input question with its paraphrase; Replacing the general word in the input question with its synonym; A detector module for detecting the knowledge-level attention weight of each injected knowledge in the knowledge-rich question, which has the same encoder-decoder model structure as a question-answering model; And An enhancer module for sequentially removing the injected knowledge with the highest knowledge-level attention weight and generating a new question using the remaining injected knowledge, inputting the new question into the question-answering model, and adjusting the knowledge-level attention weight of the removed injected knowledge according to the performance change of the question-answering model, The booster module inputs the new question into the question and answer model that has been fine-tuned on a given question and answer pair to detect the performance of the question and answer model on the new question, If the performance becomes better, the removed injected knowledge is considered as negative knowledge, and the knowledge level attention weight between the negative knowledge and the correct relationship corresponding to the entity to be queried is reduced; If the performance becomes worse, the removed injected knowledge is considered as positive knowledge, and the knowledge level attention weight between the positive knowledge and the correct relationship corresponding to the entity to be queried is increased, The booster module optimizes the knowledge level attention weight by using a factor δ, which represents the change amount of the probability value of the detector when decoding the correct relationship corresponding to the entity, When one of the injected knowledge exists in the knowledge-rich question, the matching probability of decoding the correct relationship corresponding to the entity to be queried is , the matching probability is reduced to after removing the injected knowledge and generating the new question using the remaining injected knowledge, and the change amount of the probability value is , the knowledge level attention weight of the injected knowledge is updated to , and the context vector as the decoder input is updated to : ， In the formula, are hyperparameters adjusted through experiments, The booster module further re-trains the encoder-decoder model by using a multi-task learning method according to the adjusted knowledge level attention weight, while optimizing two loss functions, one of which is used for the knowledge level attention weight optimization task, and the other of which is used for the knowledge base question and answer task based on the knowledge-rich question: ， wherein, and respectively represent the weight factor of the loss function of two tasks, the current position is "PAD" 0, otherwise 1, is the cross-entropy loss of the knowledge base question answering task based on the knowledge-rich question under different question answering models, is the probability distribution of the candidate relationship.