A question answering method and device, electronic equipment and storage medium

Abstract

The application discloses a question answering method and device, an electronic device and a storage medium. The method comprises the following steps: obtaining a target question and retrieved knowledge about the target question retrieved from an external data source, performing answer prediction based on the target question and the retrieved knowledge, obtaining an answer prediction result, wherein the answer prediction result comprises at least one predicted answer; the answer prediction result is obtained by performing answer prediction after supplementing part of the content of the retrieved knowledge; and determining a target answer of the target question based on the answer prediction result. In this way, the application can improve the accuracy of question answering.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a question-answering method and apparatus, electronic device and storage medium. Background Technology

[0002] Common sense reasoning is one of the most important advancements in natural language processing. Its aim is to help computers learn common sense knowledge and use that knowledge for deeper understanding and reasoning. Current common sense reasoning models primarily focus on factual questions, which often lack direct references, and the model can only answer based on the question itself.

[0003] Most existing technologies implicitly answer questions by directly using the knowledge acquired by the model during the pre-training stage. Only when the model is large enough can it acquire more knowledge to provide answers to questions. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a question-answering method, apparatus, electronic device, and storage medium that can improve the accuracy of question answers.

[0005] To address the aforementioned technical problems, the first aspect of this application provides a question-answering method, comprising: acquiring a target question; acquiring retrieval knowledge about the target question retrieved from an external data source; using a question-answering model to predict an answer based on the target question and the retrieval knowledge, obtaining an answer prediction result, the answer prediction result including at least one predicted answer; the answer prediction result being obtained by supplementing part of the retrieval knowledge before performing answer prediction; and determining a target answer to the target question based on the answer prediction result.

[0006] To address the aforementioned technical problems, a second aspect of this application provides a question-answering apparatus, comprising: an acquisition module for acquiring a target question and acquiring retrieval knowledge about the target question retrieved from an external data source; an answer prediction module for using a question-answering model to predict an answer based on the target question and the retrieval knowledge, obtaining an answer prediction result, wherein the answer prediction result includes at least one predicted answer; the answer prediction result is obtained by supplementing a portion of the retrieval knowledge before performing answer prediction; and a question-answering module for determining a target answer to the target question based on the answer prediction result.

[0007] To address the aforementioned technical problems, a third aspect of this application provides a question-answering device, which includes a memory and a processor coupled to each other. The memory stores program instructions, and the processor executes the program instructions to implement the method described in the first aspect.

[0008] To address the aforementioned problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the method described in the first aspect.

[0009] The beneficial effects of this application are as follows: After obtaining the target question and retrieval knowledge about the target question from external data sources, this application predicts the answer based on the target question and the retrieval knowledge, obtaining an answer prediction result, which includes at least one predicted answer; based on the answer prediction result, the target answer to the target question is determined. Unlike existing technologies that answer target questions based on knowledge stored in a model, this application can determine the target answer to the target question based on retrieval knowledge. The retrieval knowledge is obtained from external data sources, and its quantity far exceeds the amount of knowledge stored in the model. Furthermore, the answer prediction result is obtained by supplementing some content of the retrieval knowledge before predicting the answer. Supplementing some content of the retrieval knowledge can improve the accuracy of the retrieval knowledge. Therefore, answering the target question based on a larger and more accurate amount of retrieval knowledge can improve the accuracy of the question answer. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the first embodiment of the question-answering method provided in this application;

[0011] Figure 2 This is a flowchart illustrating the second embodiment of the question-answering method provided in this application;

[0012] Figure 3 This is a general technical block diagram of the second embodiment of the question-answering method provided in this application;

[0013] Figure 4 This is a flowchart illustrating the third embodiment of the question-answering method provided in this application;

[0014] Figure 5 This is a simplified schematic diagram of an implementation of the question-answering model provided in this application;

[0015] Figure 6 This is a flowchart illustrating the fourth embodiment of the question-answering method provided in this application;

[0016] Figure 7 This is a schematic diagram of the framework of one embodiment of the question-answering device provided in this application;

[0017] Figure 8 This is a schematic diagram of the framework of one embodiment of the question-answering device provided in this application;

[0018] Figure 9 This is a schematic diagram of a framework of one embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0020] It should be noted that the embodiments of this application contain descriptions involving "first," "second," etc., which are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] Please see Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the question-answering method provided in this application, which includes:

[0023] S11: Obtain the target question and retrieve the search knowledge about the target question from external data sources.

[0024] In one implementation, the target question can be a multiple-choice question or a judgment question. The retrieved knowledge can include word definitions from the target question, i.e., explanations of the words contained in the target question; triple information about the target question, i.e., triples contained in the target question, for example, if Xiaoming is the captain of the basketball team, then the triple information could be Xiaoming, basketball team, captain; similar questions similar to the target question and their answers, and at least one of the following: search results related to the target question found in a search engine.

[0025] External data sources can include common knowledge bases, Wikipedia, and various search engines. Different external data sources can employ different retrieval methods. For example, when the external data source is a common knowledge base or Wikipedia, lexical and syntactic analysis tools, such as Spacy and LTP, can be used to extract keywords for retrieval. When the external data source is various search engines, a similarity model can be used to obtain a preset number of relevant pages, similar questions, and their corresponding answers from the search engine. The preset number can be set according to the user's needs.

[0026] In one implementation, a model or algorithm can be used to obtain retrieval knowledge about the target problem.

[0027] S12: Based on the target question and retrieved knowledge, predict the answer and obtain the answer prediction result, which includes at least one predicted answer.

[0028] In one implementation, the answer can be predicted directly using the target question and the retrieved knowledge, thus obtaining the answer prediction result.

[0029] In another implementation, supplementary knowledge can be obtained by supplementing a portion of the retrieved knowledge. The supplementary knowledge and the target question are then used to predict the answer, yielding a prediction result. Specifically, the retrieved knowledge can be acquired by a knowledge supplementation model or other models, and a portion of its content can be deleted. The knowledge supplementation model then predicts and supplements the deleted portion of the retrieved knowledge, resulting in several pieces of supplementary knowledge, each different from the deleted portion. A question-answering model then uses these supplementary pieces of knowledge and the target question to predict the answer, yielding a prediction result. In other words, the prediction result is obtained by supplementing a portion of the retrieved knowledge before predicting the answer. In other implementations, an algorithm can also be used to predict the answer based on the target question and the retrieved knowledge, yielding a prediction result.

[0030] In one embodiment, the output layers of the question-answering model and the knowledge supplementation model are different, but the other network layers are the same. The output layer of the question-answering model can be used to output the target answer, or to output several predicted answers and their corresponding first probabilities; the output layer of the knowledge supplementation model is used to output several supplementary knowledge items. In another embodiment, the question-answering model and the knowledge supplementation model can be different modules of a single model. For example, the question-answering model and the knowledge supplementation model can be different modules of a T5 (Text-to-Text Transfer Transformer) model.

[0031] In one specific implementation, the retrieved knowledge is processed by a knowledge supplementation model to output several supplementary knowledge items. These supplementary knowledge items and the target question are then input into a question-answering model. The question-answering model can use the supplementary knowledge to predict the answer, obtaining a prediction result corresponding to each supplementary knowledge item. Each prediction result includes at least one predicted answer corresponding to the supplementary knowledge and a first probability corresponding to each predicted answer. Based on the prediction results corresponding to each supplementary knowledge item, the target answer to the target question is determined. Notably, each prediction result corresponding to each supplementary knowledge item may contain at least one predicted answer; that is, multiple predicted answers can be obtained for the target question. In this implementation, multiple parts of a single retrieved knowledge item can be deleted to predict several supplementary knowledge items; alternatively, multiple parts of multiple retrieved knowledge items can be deleted to predict several supplementary knowledge items.

[0032] S13: Based on the answer prediction results, determine the target answer to the target question.

[0033] In one embodiment, the answer prediction result includes at least one predicted answer, and one predicted answer can be selected as the target answer from the at least one predicted answer. In a specific embodiment, the answer prediction result includes at least one predicted answer and a first probability that each predicted answer is a correct answer, and a predicted answer is determined as the target answer based on the first probability that each predicted answer is a correct answer.

[0034] In another implementation, the answer prediction result is obtained by supplementing the retrieved knowledge with additional information from the question-answering model before predicting the answer. This yields prediction results for each supplemented knowledge item, including at least one predicted answer and a first probability for each predicted answer. The first probability of each predicted answer within the prediction results for each supplemented knowledge item is calculated to obtain a second probability that each predicted answer is the correct answer. Based on the second probability that each predicted answer is the correct answer, the target answer to the target question is determined.

[0035] The above method involves acquiring the target question and retrieved knowledge about the target question from external data sources, then predicting the answer based on the target question and the retrieved knowledge. The predicted answer includes at least one predicted answer. Based on the predicted answer, the target answer to the target question is determined. Unlike answering the target question based on knowledge stored in a model, this method determines the target answer based on retrieved knowledge. The retrieved knowledge is obtained from external data sources, and its quantity far exceeds the amount of knowledge stored in the model. Furthermore, the predicted answer is obtained by supplementing some content of the retrieved knowledge before prediction. Supplementing some content of the retrieved knowledge improves its accuracy. Therefore, answering the target question based on a larger and more accurate amount of retrieved knowledge improves the accuracy of the answer.

[0036] Please see Figure 2-3 , Figure 2 This is a flowchart illustrating the second embodiment of the question-answering method provided in this application. Figure 3 This is a general technical block diagram of the second embodiment of the question-answering method provided in this application, the method including:

[0037] S21: Obtain the target question; and obtain retrieval knowledge about the target question from external data sources.

[0038] The retrieved knowledge may include at least one of the following: word definitions in the target question, triple information about the target question, similar questions and answers to similar questions, and search results for the target question found in a search engine.

[0039] S22: Delete part of the search knowledge to obtain the deleted search knowledge.

[0040] In one implementation, the retrieved knowledge can be randomly masked to obtain deleted retrieved knowledge. In a specific implementation, the retrieved knowledge consists of several token representations, and some token representations can be masked by randomly spanning them.

[0041] Step S22 can be performed by the knowledge supplementation model or by other models. If performed by other models, the deleted retrieved knowledge can be input into the knowledge supplementation model, allowing the knowledge supplementation model to predict and supplement the deleted parts.

[0042] S23: Use a knowledge supplementation model to predict and supplement the deleted parts of the retrieved knowledge, and obtain several supplementary knowledge.

[0043] In one specific implementation, the supplementary knowledge differs from the supplementary content of the deleted portion. For example, if the retrieved knowledge is "teenagers should ensure eight hours of sleep per day," and the deleted retrieved knowledge is "teenagers should ensure [number] hours of sleep per day," then the supplementary knowledge could be "teenagers should ensure nine hours of sleep per day," or it could be "teenagers should ensure seven hours of sleep per day." Understandably, in other implementations, the supplementary knowledge may also be the same as the supplementary content of the deleted portion.

[0044] S24: Use the question-answering model to predict the answer based on each supplementary knowledge and the target question, and obtain the prediction result corresponding to each supplementary knowledge.

[0045] The prediction results corresponding to each supplementary knowledge include at least one predicted answer corresponding to the supplementary knowledge and the first probability corresponding to each predicted answer.

[0046] In one implementation, supplementary knowledge output by the knowledge supplementation model and the target question can be input into the question-answering model. Hints can also be added, allowing the question-answering model to answer the target question based on the hints. The hint content can be "The answer is" or "This answer is," without specific limitations. The question-answering model can output at least one predicted answer and a first probability corresponding to each predicted answer. The number of predicted answers can be set according to the type of the target question. For example, if the target question is a multiple-choice question, four predicted answers can be set; if the target question is a judgment question, two predicted answers can be set. The device can provide the first probability corresponding to each predicted answer based on the supplementary knowledge. For each piece of supplementary knowledge, the device can provide each predicted answer and its corresponding first probability based on the supplementary knowledge. Figure 3 As shown, after predicting and supplementing the deleted portion of the retrieved knowledge, two supplementary knowledge items are obtained. Assume that each supplementary knowledge item can yield two predicted answers. For supplementary knowledge item 1, the first probability corresponding to predicted answer A and the first probability corresponding to predicted answer B can be obtained; similarly, for supplementary knowledge item 2, the first probability corresponding to predicted answer A and the first probability corresponding to predicted answer B can also be obtained. Understandably, the first probabilities corresponding to each predicted answer in the prediction results for each supplementary knowledge item can be the same or different.

[0047] S25: Based on the prediction results corresponding to each supplementary knowledge, determine the target answer to the target question.

[0048] In one implementation, a first probability of each predicted answer in the prediction results corresponding to each supplementary knowledge can be calculated to obtain a second probability that each predicted answer is the correct answer; based on the second probability that each predicted answer is the correct answer, the target answer of the target question can be determined.

[0049] In one specific implementation, the first probabilities of the predicted answer in the prediction results corresponding to each supplementary knowledge can be added together to obtain the second probability of the predicted answer. The predicted answer whose second probability satisfies a preset requirement is selected as the target answer. The preset requirement can be the maximum value. For example, the prediction result corresponding to supplementary knowledge 1 includes predicted answer A and its corresponding first probability P(A1), and predicted answer B and its corresponding first probability P(B1); the prediction result corresponding to supplementary knowledge 2 includes predicted answer A and its corresponding first probability P(A2), and predicted answer B and its corresponding first probability P(B2). Then, the first probabilities P(A1) and P(A2) of predicted answer A in the prediction result corresponding to supplementary knowledge 1 and supplementary knowledge 2 can be summed to obtain the second probability that predicted answer A is the correct answer. Similarly, the first probabilities P(B1) and P(B2) of predicted answer B in the prediction result corresponding to supplementary knowledge 1 and supplementary knowledge 2 can be summed to obtain the second probability that predicted answer B is the correct answer. Determine the maximum value of the second probability that each predicted answer is the correct answer, and take the predicted answer corresponding to the maximum second probability value as the correct answer.

[0050] In this embodiment, step S25 can be executed by other models, that is, the question-answering model outputs the prediction results corresponding to each supplementary knowledge, and other models determine the target answer to the target question based on the prediction results corresponding to each supplementary knowledge. In other embodiments, step S25 can also be executed by the question-answering model, that is, the question-answering model outputs the target answer to the target question.

[0051] Please see Figure 4 and Figure 5 , Figure 4 This is a flowchart illustrating the third embodiment of the question-answering method provided in this application. Figure 5 This is a simplified schematic diagram of an implementation method of the question-answering model provided in this application; the method includes:

[0052] S41: Obtain the target question; and obtain retrieval knowledge about the target question from external data sources.

[0053] In one implementation, the target question can be a multiple-choice question or a judgment question. The retrieved knowledge can include at least one of the following: word definitions of the target question, triple information about the target question, similar questions and answers to similar questions, and search results for the target question found in a search engine. External data sources can be commonsense databases, Wikipedia, various search engines, etc.

[0054] S42: Train at least one of the question-answering model and the knowledge-enrichment model separately.

[0055] In one embodiment, the step of training the question-answering model separately includes: acquiring a first sample question and first sample retrieval knowledge about the first sample question; using the question-answering model to predict the answer based on the first sample retrieval knowledge and the first sample question, obtaining a first sample prediction result; and adjusting the parameters of the question-answering model based on the first sample prediction result and the labeled answer to the first sample question. The first sample question may be the same as or different from the target question. Similarly, the first sample retrieval knowledge may be the same as or different from the retrieval knowledge. In one embodiment, the first sample retrieval knowledge may include at least one of the following: word definitions in the first sample question, triple information about the first sample question, similar questions and answers to similar questions, and search results about the first sample question found in a search engine. The first sample retrieval knowledge and the first sample question are input into the question-answering model, which encodes them together into the same vector space for interaction, obtaining an aggregated vector. The first sample prediction result is then obtained based on the aggregated vector. In one specific implementation, the question-answering model can be a T5 (Text-to-Text Transfer Transformer) model. After encoding the first sample retrieval knowledge and the first sample question to obtain an aggregated vector, the question-answering model can provide relevant soft hints, such as... Figure 5 The answer is "yes". When the training model encounters this prompt, it generates the first sample prediction result.

[0056] The first sample prediction result includes at least one first sample predicted answer and the probability that each first sample predicted answer is the correct answer. Specifically, the question-answering model can use first sample retrieval knowledge to answer the first sample question, obtaining at least one first sample predicted answer and the probability that each first sample predicted answer is the correct answer. In one embodiment, the maximum value among the probabilities of each first sample predicted answer being the correct answer can be obtained. The similarity between the first sample predicted answer corresponding to the maximum probability value and the labeled answer can be calculated, and the parameters of the question-answering model can be adjusted based on the similarity result. Specifically, the first sample predicted answer with the highest probability in the first sample prediction result can be taken as the correct answer, and the parameters of the question-answering model can be adjusted based on the correct answer and the labeled answer. For example, the first sample prediction result includes first sample predicted answer A, first sample predicted answer B, and the probability P1 that first sample predicted answer A is the correct answer, and the probability P2 that first sample predicted answer B is the correct answer. If P1 is greater than P2, then first sample predicted answer A is taken as the correct answer. The parameters of the question-answering model are adjusted based on first sample predicted answer A and the labeled answer. In one specific implementation, the similarity between the predicted answer A of the first sample and the labeled answer can be calculated, and the parameters of the question-answering model can be adjusted based on the similarity.

[0057] The steps for training the knowledge supplementation model separately include: deleting part of the second sample retrieval knowledge to obtain the deleted second sample retrieval knowledge; using the knowledge supplementation model to predict and supplement the deleted part of the deleted second sample retrieval knowledge to obtain the first sample supplementary knowledge; and adjusting the parameters of the knowledge supplementation model based on the first sample supplementary knowledge and the second sample retrieval knowledge.

[0058] The second sample retrieval knowledge and the first sample retrieval knowledge can be the same or different. The second sample retrieval knowledge is randomly masked to obtain deleted second sample retrieval knowledge. A knowledge supplementation model is then used to predict and supplement the deleted portion of the deleted second sample retrieval knowledge, resulting in first sample supplemented knowledge. The similarity between the first sample supplemented knowledge and the second sample retrieval knowledge is calculated, and the parameters of the knowledge supplementation model are adjusted based on the similarity.

[0059] S43: Based on the retrieved knowledge and the target question, predict the answer to obtain the prediction result.

[0060] In one implementation, a portion of the retrieved knowledge is deleted to obtain deleted retrieved knowledge. If the knowledge supplementation model and the question-answering model have been trained, the trained knowledge supplementation model is used to predict and supplement the deleted portion of the deleted retrieved knowledge, resulting in several supplementary knowledge entries, each of which differs from the content of the deleted portion. The trained question-answering model is then used to predict answers based on each supplementary knowledge entry and the target question, yielding prediction results for each supplementary knowledge entry. The prediction results for each supplementary knowledge entry include at least one predicted answer corresponding to the supplementary knowledge entry and a first probability corresponding to each predicted answer.

[0061] S44: Based on the answer prediction results, determine the target answer to the target question.

[0062] In one embodiment, the answer prediction result includes at least one predicted answer and a first probability that the predicted answer is correct. The predicted answer with the highest first probability can be used as the target answer. For example, if the first probability of predicted answer A is 0.3 and the first probability of predicted answer B is 0.5, then predicted answer B can be used as the target answer.

[0063] For detailed implementation of steps S43-S44, please refer to steps S12-S13 of the first implementation of the question-and-answer method; they will not be repeated here.

[0064] In this embodiment, one of the question-answering model and the knowledge supplementation model can be trained separately. Training the knowledge supplementation model enables it to recover the masked part of the retrieved knowledge, while training the question-answering model enables it to generate answers based on the retrieved knowledge and the target question.

[0065] Please see Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the question-answering method provided in this application, which includes:

[0066] S61: Obtain the target question; and obtain retrieval knowledge about the target question from external data sources.

[0067] S62: Train at least one of the question-answering model and the knowledge supplementation model separately.

[0068] For detailed implementation of steps S61-S62, please refer to steps S41-S42 of the third implementation of the question-and-answer method; they will not be repeated here.

[0069] S63: Obtain the third sample problem and knowledge about third sample retrieval.

[0070] The third-sample retrieval knowledge may include at least one of the following: word definitions in the third-sample question, triple information about the third-sample question, similar questions and answers to similar questions, and search results about the third-sample question found in a search engine.

[0071] The third sample question can be the same as the target question. If the third sample question is the same as the target question, step S61 can be executed first. If the third sample question is different from the target question, steps S62-S67 can be executed first to train the question answering model and the knowledge supplementation model. After the training is completed, steps S61 and S68-S611 can be executed.

[0072] S64: Delete part of the third sample retrieval knowledge to obtain the deleted third sample retrieval knowledge.

[0073] In one implementation, the third-sample retrieval knowledge can be randomly masked to obtain deleted third-sample retrieval knowledge. In a specific implementation, the third-sample retrieval knowledge consists of several token representations, and some token representations among these token representations can be masked by randomly spanning them.

[0074] S65: Use the knowledge supplementation model to predict and supplement the deleted part of the knowledge retrieved from the deleted third sample, and obtain several supplementary knowledge for the second sample.

[0075] In one embodiment, a knowledge supplementation model is used to predict and supplement the masked portion of the knowledge retrieved from the third sample, resulting in several supplementary second sample knowledge. Each supplementary second sample knowledge may be different from or the same as the supplementary content of the deleted portion.

[0076] S66: Using a question-answering model, we predict answers based on supplementary knowledge for each sample and the question of the third sample, and obtain the prediction results of the third sample corresponding to the supplementary knowledge for each sample.

[0077] In one embodiment, a third sample question and several second sample supplementary knowledge points output by the knowledge supplementation model are input into a question-answering model. The question-answering model then uses the supplementary knowledge points and the third sample question to predict the answer, obtaining a third sample prediction result corresponding to each supplementary knowledge point. The third sample prediction result corresponding to each supplementary knowledge point includes at least one third predicted answer corresponding to the supplementary knowledge point and the probability that each third predicted answer is a correct answer. In a specific embodiment, the third sample prediction result includes a third predicted answer and the probability that that answer is a correct answer.

[0078] S67: Based on the prediction results of the third sample, the labeled answers to the third sample questions, and the supplementary knowledge and retrieval knowledge of the third sample, adjust the parameters of the question answering model and the knowledge supplementation model.

[0079] In one implementation, a first loss can be calculated based on the prediction results of the third sample and the labeled answers to the third sample questions. A second loss can be calculated based on the supplementary knowledge from the samples and the retrieval knowledge from the third samples. The parameters of the question-answering model and the knowledge supplementation model can then be adjusted based on the first and second losses. Understandably, when adjusting the parameters of the question-answering model and the knowledge supplementation model based on the first and second losses, each loss can be multiplied by its corresponding weight to obtain the total loss, and the parameters of the question-answering model and the knowledge supplementation model can be adjusted based on the total loss.

[0080] S68: Delete part of the search knowledge to obtain the deleted search knowledge;

[0081] After the question-answering model and the knowledge supplementation model are trained, the target answer to the target question can be obtained using these models. Specifically, after obtaining the target question and the retrieved knowledge, the retrieved knowledge can be input into the knowledge supplementation model, which will then delete some content from the retrieved knowledge and supplement the deleted parts. Alternatively, other models can delete some content from the retrieved knowledge to obtain deleted retrieved knowledge, which is then input into the knowledge supplementation model. The knowledge supplementation model will then predict and supplement the deleted retrieved knowledge to obtain supplementary knowledge.

[0082] S69: Use a knowledge supplementation model to predict and supplement the deleted retrieved knowledge to obtain several supplementary knowledge.

[0083] When a knowledge supplementation model predicts and supplements deleted retrieved knowledge, it can combine the contextual information of the retrieved knowledge with the knowledge learned during the training phase. In essence, the knowledge supplementation model can supplement a deleted portion of the retrieved knowledge, resulting in multiple supplementary knowledge entries. Each supplementary knowledge entry differs from the content of the deleted portion.

[0084] S610: Use a question-answering model to predict answers based on each supplementary knowledge and the target question, and obtain the prediction results corresponding to each supplementary knowledge.

[0085] The supplementary knowledge obtained from the knowledge supplementation model is input into the question answering model. The question answering model answers the target question based on each supplementary knowledge, and obtains at least one predicted answer and the first probability corresponding to the predicted answer.

[0086] S611: Based on the prediction results corresponding to each supplementary knowledge, determine the target answer to the target question.

[0087] For each piece of supplementary knowledge, calculate the first probability of each predicted answer in the prediction results corresponding to each piece of supplementary knowledge, and obtain the second probability that each predicted answer is the correct answer; select the predicted answer whose second probability meets the preset requirements as the target answer.

[0088] In this embodiment, a combined model consisting of a question-answering model and a knowledge supplementation model can be trained, so that the combined model considers more comprehensively when answering the target question, thereby improving the robustness and accuracy of the combined model.

[0089] Please see Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the question-answering device provided in this application. The question-answering device 70 includes an acquisition module 71, an answer prediction module 72, and a question-answering module 73. The acquisition module 71 is used to acquire a target question and acquire retrieval knowledge about the target question retrieved from an external data source; the answer prediction module 72 is used to predict the answer based on the target question and the retrieval knowledge, and obtain an answer prediction result, which includes at least one predicted answer; the answer prediction result is obtained by supplementing part of the retrieval knowledge before performing answer prediction; the question-answering module 73 is used to determine the target answer to the target question based on the answer prediction result.

[0090] The answer prediction results also include the first probability that each predicted answer is the correct answer; based on the answer prediction results, the target answer of the target question is determined, including: based on at least one predicted answer and the first probability corresponding to each predicted answer, the target answer of the target question is determined.

[0091] The process of predicting answers based on the target question and retrieved knowledge includes: deleting a portion of the retrieved knowledge to obtain deleted retrieved knowledge; using a knowledge supplementation model to predict and supplement the deleted portion of the retrieved knowledge to obtain several supplementary knowledge items, each of which differs from the content of the deleted portion; using a question-answering model to predict answers based on each supplementary knowledge item and the target question to obtain prediction results for each supplementary knowledge item; and determining the target answer to the target question based on the answer prediction results, including: determining the target answer to the target question based on the prediction results corresponding to each supplementary knowledge item.

[0092] The process of deleting a portion of the retrieved knowledge to obtain the deleted retrieved knowledge includes: performing random masking on the retrieved knowledge to obtain the deleted retrieved knowledge; and / or, the prediction results corresponding to the supplementary knowledge include at least one predicted answer corresponding to the supplementary knowledge and a first probability corresponding to each predicted answer; determining the target answer to the target question based on the prediction results corresponding to each supplementary knowledge includes: calculating the first probability of each predicted answer in the prediction results corresponding to each supplementary knowledge to obtain a second probability that each predicted answer is a correct answer; and determining the target answer to the target question based on the second probability that each predicted answer is a correct answer.

[0093] Specifically, the first probability of each predicted answer in the prediction results corresponding to each supplementary knowledge is calculated to obtain the second probability that each predicted answer is the correct answer. This includes: for each predicted answer, the first probability of the predicted answer in the prediction results corresponding to each supplementary knowledge is added together to obtain the second probability corresponding to the predicted answer.

[0094] Based on the second probability that each predicted answer is the correct answer, the target answer for the target question is determined, including: selecting the predicted answer whose second probability meets the preset requirements as the target answer.

[0095] Before predicting the answer based on the target question and retrieved knowledge, the process includes training at least one of the question-answering model and the knowledge-supplementing model separately, wherein the network output layers of the question-answering model and the knowledge-supplementing model are different, while the other network layers are the same.

[0096] The separate training of the question-answering model includes: acquiring the first sample question and the first sample retrieval knowledge about the first sample question; using the question-answering model to predict answers based on the first sample retrieval knowledge and the first sample question, obtaining the first sample prediction result, which includes at least one first sample predicted answer and the probability that each first sample predicted answer is the correct answer; and adjusting the parameters of the question-answering model based on the first sample prediction result and the labeled answer of the first sample question.

[0097] The steps for training the knowledge supplementation model separately include: deleting part of the second sample retrieval knowledge to obtain the deleted second sample retrieval knowledge; using the knowledge supplementation model to predict and supplement the deleted second sample retrieval knowledge to obtain the first sample supplementary knowledge; and adjusting the parameters of the knowledge supplementation model based on the first sample supplementary knowledge and the second sample retrieval knowledge.

[0098] The process includes, after training at least one of the question-answering model and the knowledge supplementation model separately, the following steps: acquiring a third sample question and third sample retrieval knowledge about the third sample question; deleting a portion of the third sample retrieval knowledge to obtain deleted third sample retrieval knowledge; using the knowledge supplementation model to predict and supplement the deleted portion of the deleted third sample retrieval knowledge to obtain several sample supplementary knowledge, each sample supplementary knowledge having different supplementary content than the deleted portion; using the question-answering model to predict answers based on each sample supplementary knowledge and the third sample question to obtain the third sample prediction result corresponding to each sample supplementary knowledge, the third sample prediction result corresponding to each sample supplementary knowledge including at least one third predicted answer corresponding to the sample supplementary knowledge and the probability that each third predicted answer is the correct answer; and adjusting the parameters of the question-answering model and the knowledge supplementation model based on the third sample prediction results, the labeled answer of the third sample question, and the sample supplementary knowledge and the third sample retrieval knowledge.

[0099] The retrieved knowledge includes: word definitions in the target question, triple information about the target question, similar questions and answers to similar questions, and at least one of the search results for the target question found in a search engine.

[0100] Please see Figure 8 , Figure 8 This is a schematic diagram illustrating one embodiment of the question-answering device provided in this application. The question-answering device 80 includes a memory 81 and a processor 82 coupled to each other. The memory 81 stores program instructions, and the processor 82 executes the program instructions to implement the steps in any of the above method embodiments. Specifically, the question-answering device 80 may include, but is not limited to, desktop computers, laptops, servers, mobile phones, tablets, etc., and is not limited thereto.

[0101] Specifically, processor 82 controls itself and memory 81 to implement the steps in any of the above method embodiments. Processor 82 can also be referred to as a CPU (Central Processing Unit). Processor 82 may be an integrated circuit chip with signal processing capabilities. Processor 82 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 82 can be implemented using integrated circuit chips.

[0102] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to one embodiment of the present application. The computer-readable storage medium 90 stores program instructions 91, which, when executed by a processor, are used to implement the steps in any of the above method embodiments.

[0103] The computer-readable storage medium 90 can specifically be a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, or a medium that can store computer programs. Alternatively, it can be a server that stores the computer program, which can send the stored computer program to other devices for execution or can also run the stored computer program itself.

[0104] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0105] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0106] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0107] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0108] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A question-answering method, characterized in that, The method includes: To identify the target problem; and Obtain retrieval knowledge about the target problem from external data sources; Delete a portion of the retrieved knowledge to obtain the retrieved knowledge after deletion; The deleted portion of the retrieved knowledge is predicted and supplemented using a knowledge supplementation model to obtain several supplementary knowledge items, each of which is different from the supplementary content of the deleted portion. A question-answering model is used to predict answers based on each of the supplementary knowledge and the target question, respectively, to obtain prediction results corresponding to each of the supplementary knowledge. The prediction results corresponding to the supplementary knowledge include at least one predicted answer corresponding to the supplementary knowledge. Based on the prediction results corresponding to the aforementioned supplementary knowledge, the target answer to the target question is determined.

2. The method according to claim 1, characterized in that, The answer prediction results also include the first probability that each predicted answer is the correct answer; Determining the target answer to the target question based on the answer prediction result includes: The target answer to the target question is determined based on the at least one predicted answer and the first probability corresponding to each predicted answer.

3. The method according to claim 1, characterized in that, The step of deleting a portion of the retrieved knowledge to obtain the deleted retrieved knowledge includes: The retrieved knowledge is then subjected to a random masking process to obtain the retrieved knowledge after deletion. And / or, the prediction result corresponding to the supplementary knowledge includes at least one predicted answer corresponding to the supplementary knowledge and a first probability corresponding to each predicted answer; determining the target answer to the target question based on the prediction results corresponding to each of the supplementary knowledge includes: By calculating the first probability of each predicted answer in the prediction results corresponding to each supplementary knowledge, a second probability is obtained that each predicted answer is the correct answer. The target answer to the target question is determined based on the second probability that each of the predicted answers is the correct answer.

4. The method according to claim 3, characterized in that, The method of calculating the first probability of each predicted answer in the prediction results corresponding to each supplementary knowledge, and obtaining the second probability that each predicted answer is the correct answer, includes: For each of the predicted answers, the first probability of the predicted answer in the prediction results corresponding to each of the supplementary knowledge is added together to obtain the second probability corresponding to the predicted answer; Determining the target answer to the target question based on the second probability that each of the predicted answers is a correct answer includes: The predicted answer that satisfies the preset requirement with the second probability is selected as the target answer.

5. The method according to claim 1, characterized in that, The question-answering model and the knowledge supplementation model have different network output layers, but the other network layers are the same except for the network output layer. And / or, in the process of obtaining the target problem; After acquiring retrieval knowledge about the target problem from external data sources, the method further includes: At least one of the question-answering model and the knowledge supplementation model is trained separately.

6. The method according to claim 5, characterized in that, The steps for training the question-answering model separately include: Obtain the first sample question and the first sample retrieval knowledge about the first sample question; The question-answering model is used to predict answers based on the knowledge retrieved from the first sample and the first sample question, to obtain a first sample prediction result. The first sample prediction result includes at least one first sample predicted answer and the probability that each first sample predicted answer is the correct answer. Based on the prediction results of the first sample and the labeled answers to the questions in the first sample, the parameters of the question-answering model are adjusted; The steps for training the knowledge supplementation model separately include: By deleting a portion of the second sample retrieval knowledge, the deleted second sample retrieval knowledge is obtained. The knowledge supplementation model is used to predict and supplement the deleted portion of the retrieved knowledge of the deleted second sample to obtain supplemented knowledge of the first sample. Based on the knowledge supplemented by the first sample and the knowledge retrieved from the second sample, the parameters of the knowledge supplementation model are adjusted.

7. The method according to claim 5, characterized in that, After training at least one of the question-answering model and the knowledge-supplementing model separately, the method further includes: Acquire knowledge about the third-sample problem and third-sample retrieval related to the third-sample problem; The deleted portion of the third sample retrieval knowledge is obtained as the deleted third sample retrieval knowledge. The knowledge supplementation model is used to predict and supplement the deleted part of the retrieved knowledge of the deleted third sample, resulting in several supplementary second sample knowledge. Each supplementary second sample knowledge is different from the supplementary content of the deleted part. The question-answering model is used to predict answers based on the supplementary knowledge of each sample and the third sample question, respectively, to obtain the third sample prediction result corresponding to the supplementary knowledge of each sample. The third sample prediction result corresponding to the supplementary knowledge of each sample includes at least one third predicted answer corresponding to the supplementary knowledge of the sample and the probability that each third predicted answer is the correct answer. Based on the prediction results of the third sample, the labeled answers to the third sample questions, the supplementary knowledge of the samples, and the retrieval knowledge of the third sample, the parameters of the question answering model and the knowledge supplementation model are adjusted.

8. The method according to claim 1, characterized in that, The retrieved knowledge includes at least one of the following: word definitions in the target question, triple information about the target question, similar questions similar to the target question and their answers, and search results for the target question found in a search engine.

9. A question-answering device, characterized in that, The device includes: The acquisition module is used to acquire the target question and to acquire retrieval knowledge about the target question retrieved from external data sources; The answer prediction module is used to delete a portion of the retrieved knowledge to obtain the deleted retrieved knowledge; to use a knowledge supplementation model to predict and supplement the deleted portion of the retrieved knowledge to obtain several supplementary knowledge items, each of which is different from the supplementary content of the deleted portion; and to use a question answering model to predict the answer based on each of the supplementary knowledge items and the target question to obtain the prediction result corresponding to each of the supplementary knowledge items; the prediction result corresponding to each supplementary knowledge item includes at least one predicted answer corresponding to the supplementary knowledge item. The question-answering module is used to determine the target answer to the target question based on the prediction results corresponding to the supplementary knowledge.

10. A question-and-answer device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program instructions, and the processor executing the program instructions to implement the question-answering method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the question-answering method according to any one of claims 1 to 8.

Images