Question answering method, device and related equipment and storage medium

Abstract

The present application discloses a method, device and related equipment and storage medium for answering questions; the method comprises: retrieving target evidence text from a candidate evidence set based on a question text, and incorporating the target evidence text into the target evidence set of the question text; analyzing the question text and the target evidence set to obtain candidate answer results for the question text; wherein the candidate answer results at least include candidate answer texts; evaluating the target evidence set based on the candidate answer results to obtain evaluation results for the target evidence set; wherein the evaluation results characterize whether the information amount of the target evidence set is sufficient; in response to the evaluation results not meeting the preset conditions, re-executing the step of retrieving the target evidence text from the candidate evidence set based on the question text until the evaluation results meet the preset conditions, and obtaining the target answer text based on the latest candidate answer results. The above scheme improves the utilization rate of evidence and the efficiency of question answering.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a question answering method, apparatus, and related equipment and storage medium. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology in my country, the use of natural language processing technology in artificial intelligence technology, especially machine reading comprehension / question-answering technology, to build a knowledge question-answering system from massive text knowledge will greatly help improve the efficiency of question-answering services.

[0003] However, the existing supporting evidence analysis model uses batch evidence input when answering questions in question answering tasks, which leads to low evidence utilization. In addition, in the process of answering questions, it is necessary to extract relevant texts from the batch evidence that are helpful in answering questions. Therefore, when there are many batch evidences, the question answering efficiency is low. In view of this, how to improve the evidence utilization rate when answering questions and improve the efficiency of question answering has become an urgent problem to be solved. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a question answering method, apparatus and related equipment and storage medium, which can improve the utilization rate of evidence and the efficiency of question answering.

[0005] In order to solve the above-mentioned technical problems, the first aspect of the present application provides a question answering method, which includes: retrieving a target evidence text from a candidate evidence set based on a question text, and incorporating the target evidence text into a target evidence set of the question text; analyzing the question text and the target evidence set to obtain a candidate answer result for the question text; wherein the candidate answer result includes at least a candidate answer text; evaluating the target evidence set based on the candidate answer result to obtain an evaluation result of the target evidence set; wherein the evaluation result represents whether the amount of information in the target evidence set is sufficient; in response to an evaluation result not meeting a preset condition, re-executing the step of retrieving the target evidence text from the candidate evidence set based on the question text until the evaluation result meets the preset condition, and then obtaining the target answer text based on the latest candidate answer result.

[0006] In order to solve the above-mentioned technical problems, the second aspect of the present application provides a question answering device, which includes an evidence retrieval module, a question analysis module, an evidence evaluation module and a loop execution module; the evidence retrieval module is used to retrieve the target evidence text in the candidate evidence set based on the question text, and incorporate the target evidence text into the target evidence set of the question text; the question analysis module is used to analyze based on the question text and the target evidence set to obtain candidate answer results for the question text; wherein the candidate answer results at least include candidate answer text; the evidence evaluation module is used to evaluate the target evidence set based on the candidate answer results to obtain evaluation results of the target evidence set; wherein the evaluation results characterize whether the amount of information in the target evidence set is sufficient; the loop execution module is used to respond to the evaluation result not meeting the preset condition, re-execute the step of retrieving the target evidence text in the candidate evidence set based on the question text, until the evaluation result meets the preset condition, and obtains the target answer text based on the latest candidate answer result.

[0007] In order to solve the above technical problems, the third aspect of the present application provides an electronic device, which includes a memory and a processor, the memory stores program instructions, and the processor is used to execute the program instructions to implement the above question answering method.

[0008] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium, which is used to store program instructions, and the program instructions can be executed to implement the above question answering method.

[0009] In the above scheme, when the amount of information in the target evidence set of the question text does not meet the preset conditions, new target evidence texts are retrieved from the candidate evidence set to increase the amount of information in the target evidence set. Therefore, the number of target evidence texts in the target evidence set provided by the present application, that is, the amount of information in the target evidence set, is a gradually increasing process, which can be regarded as the selective and incremental input of new target evidence texts according to the amount of information in the target evidence set, thereby reducing the loss of evidence and improving the utilization rate of evidence.

[0010] In addition, when the amount of information in the target evidence set meets the preset conditions, the retrieval of the target evidence text is stopped and the answer corresponding to the question is determined. On the one hand, it ensures that the answer is determined under the premise that the amount of information in the target evidence set is sufficient, thereby improving the certainty of the correct answer to the question, that is, improving the accuracy of the target answer text and the efficiency of answering the question; on the other hand, it achieves an improvement in the accuracy of answering the question while reducing evidence consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flowchart of an embodiment of a question answering method provided by the present application;

[0012] Figure 2 It is an overall technical block diagram of an embodiment of a question answering system provided by the present application;

[0013] Figure 3 yes Figure 1 The flowchart of step S11 is shown as an embodiment;

[0014] Figure 4 yes Figure 3 The flowchart of step S111 is shown as an embodiment;

[0015] Figure 5 It is a schematic diagram of an embodiment of an updating process of evidence retrieval weight provided by the present application;

[0016] Figure 6 yes Figure 1 The flowchart of step S12 is shown as an embodiment;

[0017] Figure 7 yes Figure 6 The flowchart of step S121 of an embodiment is shown;

[0018] Figure 8 yes Figure 7 The flowchart of step S72 is shown as an embodiment;

[0019] Fig. 9 yes Figure 7 The flowchart of step S74 is shown as an embodiment;

[0020] Fig.10 yes Figure 6 The flowchart of step S122 of an embodiment is shown;

[0021] Fig.11 yes Figure 1 The flowchart of step S13 is shown as an embodiment;

[0022] Fig.12 It is a flowchart of an embodiment of random deactivation of a question answering model provided by the present application;

[0023] Fig.13 It is a flowchart of an embodiment of the evidence prediction model training method provided by the present application;

[0024] Fig.14 It is a structural schematic diagram of an embodiment of a question answering device provided by the present application;

[0025] Fig.15 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0026] Fig.16 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0027] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0028] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0029] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0030] See also Figure 1-Figure 2 , Figure 1 is a flowchart of an embodiment of a question answering method provided by the present application, Figure 2 is an overall technical block diagram of an embodiment of the question answering system provided by the present application. It should be noted that if there are substantially the same results, the embodiments of the present application are not based on Figure 1 The process sequence shown is limited. Figure 1 As shown, this embodiment includes:

[0031] Step S11: Retrieve target evidence text from the candidate evidence set based on the question text, and include the target evidence text in the target evidence set of the question text.

[0032] The method of this embodiment is used to obtain the answer text based on the target evidence text in the target evidence text set in the question answering task, thereby improving the utilization rate of evidence. The types corresponding to the question text described in this article include but are not limited to multiple-choice questions, true-or-false questions, answer questions, etc., which are not specifically limited here; the fields involved in the question text described in this article include but are not limited to the medical field, the biological field, the mathematical field, the physical field, the chemical field, the geographical field, etc., which are not specifically limited here. Among them, it should be noted that when the type corresponding to the question text is an answer question and a true-or-false question, the question text is the stem text of the answer question corresponding to it; when the type corresponding to the question text is a multiple-choice question, the stem text of the multiple-choice question corresponding to it and any option text constitute the question text, and at this time the number of question texts corresponding to the multiple-choice question is the number of options of the multiple-choice question, that is, the stem text will constitute different question texts with each option text. For example, a medical multiple-choice question I contains three candidate options, namely option A, option B and option C; therefore, for medical multiple-choice question I, there are three corresponding question texts, namely the question text consisting of the question stem text and option A text, the question text consisting of the question stem text and option B text, and the question text consisting of the question stem text and option C text.

[0033] In one embodiment, the question text may be any text to which an answer needs to be determined, and may be obtained from local storage or cloud storage. It is understandable that in other embodiments, the current question may be collected by a collection device and a corresponding question text may be generated.

[0034] In this embodiment, the target evidence text is retrieved from the candidate evidence set based on the question text, and the target evidence text is included in the target evidence set of the question text. That is, the question text is used as a search formula to search in the candidate evidence set to obtain the target evidence text corresponding to the question text, and the target evidence text corresponding to the question text is included in the target evidence set, so as to facilitate subsequent analysis based on the question text and the target evidence text in the target evidence set.

[0035] Specifically, when searching for the target evidence text in the candidate evidence set in the first round based on the question text, the scores between the question text and each candidate evidence text in the candidate evidence set will be directly calculated, wherein any scoring function can be used to calculate the scores between the question text and each candidate evidence text in the candidate evidence set; then, the scores are sorted in descending order, and the candidate evidence text corresponding to the highest score is taken as the target evidence text. Since the information amount of the target evidence set is not sufficient in the subsequent evaluation results, it is necessary to search again in the candidate evidence set based on the question text. When searching again in the candidate evidence set based on the question text, the feature representation of the question text will be first updated using the evidence retrieval weight, and then the updated feature representation of the question text will be used to search in the candidate evidence set; by updating the feature representation of the question text before each search, and searching in the candidate evidence set using the updated feature representation of the question text, and by iteratively adjusting the search weight of each word in the search formula, on the one hand, it can ensure that more valuable, i.e., effective target evidence text can be retrieved in each round of search, and the accuracy of the question answer can be improved under the premise of equal evidence text, and on the other hand, it can avoid the accuracy of the target answer text determined subsequently being affected by the similarity of the target evidence text retrieved each time.

[0036] Optionally, in one embodiment, the candidate evidence set is a set of evidence corresponding to the field to which the question text belongs, and the candidate evidence set includes, but is not limited to, textbooks, supplementary teaching materials, literature materials, etc., which are not specifically limited here. For example, the field to which the question text belongs is the medical field, and its corresponding candidate evidence set includes medical textbooks, medical supplementary teaching materials, medical literature, etc. It can be understood that in other embodiments, the candidate evidence set can also be a set of evidence including all fields, that is, the candidate evidence set includes relevant materials, literature, etc. in any field, and the candidate evidence set corresponding to the question text in each field is the same.

[0037] In one embodiment, when the question stem text and each option text constitute different question texts, the different question texts correspond to their own target evidence sets. That is to say, when the question stem text and each option text constitute different question texts, each question text will be used as a search formula to search in the candidate evidence set to obtain the target evidence set corresponding to each question text. For example, a medical multiple-choice question I contains two candidate options, namely candidate option A and candidate option B; therefore, for the medical multiple-choice question I, there are two corresponding question texts, namely question text α1 composed of the question stem text and the option text of candidate option A and question text α2 composed of the question stem text and the option text of candidate option B; therefore, the question text α1 and the question text α2 will be searched for the target evidence text in the candidate evidence set, and the retrieved target evidence text corresponding to the question text α1 will be included in the target evidence set β1 corresponding to the question text α1, and the retrieved target evidence text corresponding to the question text α2 will be included in the target evidence set β2 corresponding to the question text α2.

[0038] In a specific embodiment, Figure 2 As shown, the question answering system includes an evidence screening module, which retrieves target evidence text from a candidate evidence set based on the question text and incorporates the target evidence text into the target evidence set of the question text.

[0039] Step S12: Analyze the question text and the target evidence set to obtain candidate answer results for the question text.

[0040] In this embodiment, the question text and the target evidence set are analyzed to obtain candidate answer results for the question text, wherein the candidate answer results at least include the candidate answer text.

[0041] In one embodiment, if Figure 2 As shown, the question answering system includes a question answering model, which analyzes the question text and each target evidence text in the target evidence set respectively to predict candidate answer results for the question text.

[0042] In one embodiment, when the question text and each option text constitute different question texts respectively, the candidate answer result includes the first probability value of each option text as a candidate answer text. That is to say, when the question is a multiple-choice question, the candidate answer result of the question text is the probability of each option text of the multiple-choice question as a candidate answer text. In other embodiments, when the question text is composed of only the question text, that is, when the question is not a multiple-choice question, such as when the question is an answer question, the candidate answer result of the question text is not the probability value of each option text as a candidate answer text, but needs to be set according to its own needs. For example, based on the analysis of the question text and a certain target evidence text, the probability value of each word in the target evidence text as the starting character of the candidate answer text and the probability value as the ending character of the candidate answer text are predicted, and the joint probability value of each word and each word after it constituting the candidate answer text can be obtained.

[0043] In a specific implementation, when the question stem text and each option text constitute different question texts, for the question text and each target evidence text in its target evidence set, feature vectors of the question text and each target evidence text may be extracted respectively, and the similarity between the question text and each target evidence text may be determined; then, based on the similarity between the question text and each target evidence text, the probability value of the option text constituting the question text as the candidate answer text may be determined. In other words, the probability value of the option text constituting the question text as the candidate answer text may be determined at the sentence granularity.

[0044] In order to improve the accuracy of the probability values ​​of the selected texts of each question text as candidate answer texts, in other specific embodiments, when the question stem text and each option text constitute different question texts, for the question text and each target evidence text in its target evidence set, the semantic similarity features between the question text and each target evidence text can also be determined at the granularity of words, and then predictions are made based on the semantic similarity features to obtain the probability values ​​of the option texts constituting the question text as candidate answer texts.

[0045] Step S13: Evaluate the target evidence set based on the candidate answer results to obtain the evaluation results of the target evidence set.

[0046] In this embodiment, the target evidence set is evaluated based on the candidate answer results to obtain the evaluation results of the target evidence set. Among them, the evaluation results of the target evidence set represent whether the information amount of the target evidence set is sufficient. Whether the information amount of the target evidence set is sufficient can be understood as the degree of certainty that the target answer text obtained based on the candidate answer results is correct in the case of the number of target evidence texts in the current target evidence set. In other words, after obtaining the candidate answer results based on the target evidence text in the target evidence set, whether the information amount of the target evidence set is sufficient will be evaluated, so that the target answer text can be obtained based on the candidate answer results under the premise that the information amount of the target evidence set is sufficient, thereby improving the accuracy or credibility of the target answer text.

[0047] In one embodiment, if Figure 2 As shown, the question answering system also includes an evidence evaluation model. The question answering model inputs the candidate answer results of the question text obtained by analysis into the evidence evaluation model. The evidence evaluation model evaluates the target evidence set based on the candidate answer results to obtain the evaluation results of the target evidence set. Among them, the evidence evaluation model is essentially a binary classifier; it can be assumed that when the output of the evidence evaluation model is 0, it means that the evaluation result is that the information content of the target evidence set is insufficient, and when the output of the evidence evaluation model is 1, it means that the evaluation result is that the information content of the target evidence set is sufficient.

[0048] Step S14: In response to the evaluation result not satisfying the preset condition, the step of retrieving the target evidence text in the candidate evidence set based on the question text is re-executed until the evaluation result satisfies the preset condition, and the target answer text is obtained based on the latest candidate answer result.

[0049] In this embodiment, in response to the evaluation result not meeting the preset condition, the step of retrieving the target evidence text in the candidate evidence set based on the question text is re-executed until the evaluation result meets the preset condition, and the target answer text is obtained based on the latest candidate answer result. It should be noted that when the evaluation result meets the preset condition, it indicates that the target evidence set corresponding to the question text has sufficient information, and the certainty of obtaining the correct target answer text by using the target evidence set with sufficient information is relatively large. In other words, when the information amount of the target evidence set is insufficient, the target answer text is temporarily not determined based on the candidate answer result, but a new target evidence text is retrieved from the candidate target evidence set to increase the certainty of the correct answer to the question, until the evaluation determines that the information amount of the target evidence set is sufficient, the target answer text is determined based on the latest candidate answer result, so as to ensure that the target answer text of the question is determined under the premise of sufficient information amount of the target evidence set, thereby improving the certainty of the correct answer to the question, that is, improving the accuracy of the target answer text. Among them, the preset condition is not limited, and can be specifically set according to actual use needs. For example, the preset condition is that the information amount of the target evidence set is sufficient.

[0050] Since when the amount of information in the target evidence set of the question text does not meet the preset conditions, new target evidence texts are retrieved from the candidate evidence set to increase the amount of information in the target evidence set; therefore, the number of target evidence texts in the target evidence set provided by the present application, i.e., the amount of information in the target evidence set, is a gradually increasing process, which can be regarded as selectively and incrementally inputting new target evidence texts according to the amount of information in the target evidence set. Compared with batch input of evidence texts, different amounts of evidence texts can be used to answer questions for different questions, which reduces the loss of evidence and improves the utilization rate of evidence. In addition, when the amount of information in the target evidence set meets the preset conditions, the retrieval of the target evidence text is stopped and the answer to the question is determined. On the one hand, it ensures that the answer is determined under the premise that the amount of information in the target evidence set is sufficient, which improves the certainty of the correct answer to the question, i.e., the accuracy of the target answer text is improved; on the other hand, the accuracy of the question answer is improved while consuming less evidence.

[0051] In one embodiment, if Figure 2 As shown, the question answering system includes an evidence evaluation model and a question answering model. The candidate answer results of the question text are analyzed by the question answering model, and whether the information content of the target evidence set corresponding to the question text is sufficient is predicted by the evidence evaluation model. Since the question answering model uses a streaming or incremental evidence input method, the setting of the evidence evaluation model can enable the question answering model to estimate the sufficiency of evidence information under the streaming evidence input situation, so as to ensure that the answer to the question is determined on the premise of sufficient evidence information, thereby improving the certainty of the correct answer to the question.

[0052] In the above implementation, when the amount of information in the target evidence set of the question text does not meet the preset conditions, new target evidence texts are retrieved from the candidate evidence set to increase the amount of information in the target evidence set; therefore, the number of target evidence texts in the target evidence set provided by the present application, that is, the amount of information in the target evidence set, is a gradually increasing process, which can be regarded as selectively and incrementally inputting new target evidence texts according to the amount of information in the target evidence set, thereby reducing the loss of evidence and improving the utilization rate of evidence.

[0053] In addition, when the amount of information in the target evidence set meets the preset conditions, the retrieval of the target evidence text is stopped and the answer corresponding to the question is determined. On the one hand, it ensures that the answer is determined under the premise that the amount of information in the target evidence set is sufficient, thereby improving the certainty of the correct answer to the question, that is, improving the accuracy of the target answer text and the efficiency of answering the question; on the other hand, it achieves an improvement in the accuracy of answering the question while reducing evidence consumption.

[0054] See also Figure 3 , Figure 3 yes Figure 1 It should be noted that if there is substantially the same result, this embodiment does not use Figure 3 The process sequence shown is limited. Figure 3 As shown, in this embodiment, when searching again in the candidate evidence set based on the question text, the feature representation of the question text is first updated using the evidence retrieval weight, and then the updated feature representation of the question text is used to search in the candidate evidence set, specifically including:

[0055] Step S111: Obtain the evidence retrieval weight updated after the last retrieval of the target evidence text.

[0056] Because the target evidence text was searched and matched in the candidate evidence set based on the question text last time, some of the first words in the question text may be fully matched by the target evidence text, that is, these first words are fully referenced in the retrieval process. If these first words are still fully referenced in the subsequent retrieval to determine the target evidence text, the determined target evidence text will be similar to the target evidence text determined in the previous retrieval, so that although the number of target evidence texts in the target evidence set increases, the amount of information is still insufficient. The added target evidence texts are of low value, which is not conducive to the determination of the subsequent target answer text.

[0057] Therefore, in order to make the target evidence text determined by this search valuable, that is, conducive to the determination of the subsequent target answer text, in this embodiment, the evidence retrieval weight updated after the last search for the target evidence text is obtained, wherein the evidence retrieval weight includes the first sub-weight of each first word in the question text, so that the first question representation of the updated question text can be obtained based on the evidence retrieval weight, and then the target evidence text can be determined based on the updated first question representation, without focusing on certain first words that were fully referenced in the previous search process. In other words, through the updated evidence retrieval weight, the retrieval weight of each first word in the question text composed of the question stem text and the option text can be adjusted iteratively in the subsequent search, which is helpful for the subsequent retrieval of the target evidence text that is conducive to the correct answer to the question.

[0058] In one embodiment, if Figure 4 As shown, Figure 4 yes Figure 3 The flowchart of step S111 of an embodiment is shown. After the target evidence text is retrieved and determined in the candidate evidence set based on the question text, the evidence retrieval weight is updated. Specifically, the following sub-steps are included:

[0059] Step S41: Obtain the second word representation of each second word in the latest target evidence text.

[0060] In this embodiment, the second word representation of each second word in the latest target evidence text is obtained. That is, after the target evidence text is retrieved and determined in the candidate evidence set based on the question text, the second word representation of each second word in this target evidence text needs to be obtained, so as to facilitate the subsequent determination of the first word in the question text that is more fully matched by the latest target evidence text.

[0061] In one embodiment, the second word representation may be a global word vector (Global Vectors for Word Representation, GloVo). Of course, in other embodiments, the second word representation may also be other word vectors, which are not specifically limited here.

[0062] Step S42: For each first word, based on the feature similarity between the weighted word representation and each second word representation, a matching score of the first word is obtained.

[0063] In this embodiment, the first question expression includes weighted word expressions of first words, and for each first word, a matching score of the first word is obtained based on the feature similarity between the weighted word expression and each second word expression. Specifically, for each first word included in the first question expression, the feature similarity between the weighted word expression of the first word and the second word expression of each second word in the latest target evidence text is calculated, and based on the feature similarity between the weighted word expression of the first word and the second word expression of each second word in the latest target evidence text, a representation of the degree to which the first word is matched by the latest target evidence text is obtained.

[0064] In one embodiment, the feature similarity between the weighted word representation of the first word and the second word representation of each second word in the latest target evidence text may be the cosine distance between the weighted word representation of the first word and the second word representation of each second word in the latest target evidence text. It is understandable that in other embodiments, the feature similarity between the weighted word representation of the first word and the second word representation of each second word in the latest target evidence text may also be calculated by other methods, which are not specifically limited here.

[0065] In one embodiment, for each first word included in the first question representation, the maximum feature similarity among the feature similarities between the weighted word representation of the first word and the second word representation of each second word in the latest target evidence text may be used as the matching score of the first word, which is used to indicate the degree to which the first word is matched by the target evidence text. For example, Figure 5 As shown, Figure 5It is a schematic diagram of an embodiment of the updating process of evidence retrieval weights provided by the present application. The first question representation includes 6 weighted word vectors of the first words, which are the weighted word vectors corresponding to the first words a, b, c, d, e and f respectively; the latest target evidence text includes 7 second word representations of the second words, which are the second word representations of the second words A, B, C, D, E, F and G respectively; in area α, the first row represents the feature similarity between the weighted word vector of the first word a and the second word representations of the second words A, B, C, D, E, F and G respectively, the second row represents the feature similarity between the weighted word vector of the first word b and the second word representations of the second words A, B, C, D, E, F and G respectively, the third row represents the feature similarity between the weighted word vector of the first word c and the second word representations of the second words A, B, C, D, E, F and G respectively, the fourth row represents the feature similarity between the weighted word vector of the first word d The feature similarities between the weighted word vector and the second word representations of the second words A, B, C, D, E, F and G, respectively; the fifth row represents the feature similarities between the weighted word vector of the first word e and the second word representations of the second words A, B, C, D, E, F and G, respectively; the sixth row represents the feature similarities between the weighted word vector of the first word f and the second word representations of the second words A, B, C, D, E, F and G, respectively; take the maximum feature similarity in the first row as the matching score of the first word a, take the maximum feature similarity in the second row as the matching score of the first word b, take the maximum feature similarity in the third row as the matching score of the first word c, take the maximum feature similarity in the fourth row as the matching score of the first word d, take the maximum feature similarity in the fifth row as the matching score of the first word e, and take the maximum feature similarity in the sixth row as the matching score of the first word f.

[0066] It can be understood that in other embodiments, for each first word included in the first question representation, the feature similarities between the weighted word representation of the first word and the second word representation of each second word in the latest target evidence text can also be averaged, and the average value of each feature similarity is used as the matching score of the first word, which is used to characterize the degree to which the first word is matched by the target evidence text.

[0067] Step S43: Based on the matching scores of the first words, at least one first word is selected as a target word.

[0068] In this embodiment, at least one first word is selected as a target word based on the matching scores of each first word. In one embodiment, according to the matching scores of each first word, the first word corresponding to the matching score greater than or equal to the matching threshold is used as the target word, wherein the matching threshold is not limited and can be specifically set according to actual use needs. If the matching score of the first word is greater than or equal to the matching threshold, it indicates that the first word is matched to a relatively high degree by the target evidence text, that is, the latest target evidence text refers to the first word to a large extent in this search process, so the first word is used as the target word to facilitate the subsequent adjustment of the first sub-weight corresponding to the first word, so that the first word will not be referenced too much when the new target evidence text is retrieved again in the subsequent search, thereby improving the value of the new target evidence text determined by the subsequent search, and increasing the amount of information in the target evidence set, which is conducive to the determination of the subsequent target answer text.

[0069] The number of first words selected as target words is not limited, and can be set according to actual usage needs. For example, 1, 2, 3, etc. first words are selected as target words. Figure 5 As shown, the six first words included in the first question representation are all taken as target words.

[0070] Step S44: weighting the latest first sub-weight of each target word based on the attenuation coefficient to obtain an updated evidence retrieval weight.

[0071] In this embodiment, the latest first sub-weight of each target word is weighted based on the decay coefficient to obtain the updated evidence retrieval weight. That is to say, the latest first sub-weight of each target word is weighted by the information decay coefficient (info decay) to obtain the new first sub-weight of each first word in the question text. The new first sub-weight of each first word in the question text constitutes the updated evidence retrieval weight, so that when the information amount of the target evidence set is determined to be insufficient and the candidate evidence set is searched again based on the question text, it is possible not to focus on the first word that was referenced to a large extent by the target evidence text in the last search, thereby improving the effectiveness and value of the target evidence text determined by the subsequent search, thereby increasing the information amount of the target evidence set, which is conducive to the determination of the subsequent target answer text.

[0072] Step S112: weighting the first word representations of the first words based on the first sub-weights of the first words respectively to obtain a first question representation of the question text.

[0073] In this embodiment, the first word representation of the first word is weighted based on the first sub-weight of each first word to obtain the first question representation of the question text. That is to say, the first sub-weight of the first word is multiplied by the first word representation of the corresponding first word; then, the first word representation of each first word after multiplication by the first sub-weight is spliced ​​to obtain the first question representation of the question text. Since the first sub-weight of each first word is updated and different from that before the update, the first question representation of the question text obtained based on the first sub-weight of each first word is also different from the original first question representation. Therefore, the target evidence text subsequently determined based on the first question representation of the updated question text will not be very similar to the previously determined target evidence text, and is valid. Including it in the target evidence set corresponding to the question text will increase the amount of information in the target evidence set, which is conducive to the subsequent determination of the target answer text.

[0074] In one embodiment, the first word representation may be a global word vector (Global Vectors for Word Representation, GloVo). Of course, in other embodiments, the first word representation may also be other word vectors, which are not specifically limited here.

[0075] Step S113: Score each candidate evidence text in the candidate evidence set based on the first question representation to determine the target evidence text.

[0076] In this implementation, each candidate evidence text in the candidate evidence set is scored based on the first question representation to determine the target evidence text. That is, the first question representation of the question text and each candidate evidence text in the candidate evidence set are scored one by one, and the target evidence text is determined from each candidate evidence text based on the score between the first question representation of the question text and each candidate evidence text. Among them, the scoring function used to score the first question representation of the question text and each candidate evidence text is not limited, and can be specifically set according to actual use needs.

[0077] In one embodiment, the candidate evidence text corresponding to the highest score can be used as the target evidence text; the higher the score, the higher the correlation between the candidate evidence text and the question text, which is conducive to the subsequent determination of the target answer text corresponding to the question text, and improves the accuracy of the determined target answer text. It can be understood that in other embodiments, the target evidence text can also be determined from the candidate evidence text by other methods, which are not specifically limited here.

[0078] It should be noted that when the question stem text and each option text constitute different question texts respectively, steps S111 to S113 will be executed accordingly for each question text to retrieve the corresponding target evidence text for each option text, thereby constructing a corresponding target evidence set for each option text.

[0079] See also Figure 6 , Figure 6 yes Figure 1 FIG. 1 is a flow chart of an embodiment of step S12. It should be noted that if there is substantially the same result, this embodiment does not necessarily use Figure 6 The process sequence shown is limited. Figure 6 As shown, in this embodiment, when the question stem text and each option text constitute different question texts respectively, the probability value of the option text of each question text as a candidate answer text is determined from the word granularity, specifically including:

[0080] Step S121: For the question text and each target evidence text in its target evidence set, reasoning is performed based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text to obtain semantic similarity features of each target evidence text.

[0081] In this embodiment, for each target evidence text in the question text and its target evidence set, reasoning is performed based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text, respectively, to obtain the semantic similarity features of each target evidence text. Among them, the semantic similarity feature represents the semantic similarity between the target evidence text and the question text. In other words, the question text and the target evidence text will be divided into word granularity, and the semantic similarity features representing the semantic similarity between the target evidence text and the question text are obtained by analyzing the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text.

[0082] In one embodiment, the question text is converted into a vector representation to obtain the first word representation of each first word in the question text; for each target evidence text in the target evidence set corresponding to the question text, the target evidence text is converted into a vector representation to obtain the second word representation of each second word in the target evidence text. Among them, the question text and the target evidence text can be processed using the word vector pre-trained using the Skip-Gram method to obtain the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text. Of course, in other embodiments, the question text and the target evidence text can also be processed using the word vector pre-trained using CBOW or Skip-Gram-SI, etc., to obtain the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text. Among them, it should be noted that the word vector pre-trained in the same way can be used to extract the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text; of course, different ways can also be used to extract the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text.

[0083] In one embodiment, for the question text and each target evidence text in its target evidence set, the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text are directly inferred to obtain the semantic similarity features of each target evidence text. Since the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text are obtained by extracting pre-trained word vectors, the first word representations of each first word in the extracted question text are isolated and the second word representations of each second word in the extracted target evidence text are also isolated, and no contextual connection and no connection between the question text and the target evidence text are established. Therefore, in order to improve the accuracy of the obtained semantic similarity features that characterize the semantic similarity between the target evidence text and the question text, in other embodiments, such as Figure 7 As shown, Figure 7 yes Figure 6 The flowchart of an embodiment of step S121 is shown. For each target evidence text in the target evidence set corresponding to the question text, context information, target evidence text related information, etc. are integrated on the basis of the first word representation of each first word in the question text to generate a fused word representation of each first word, so as to determine the semantic similarity feature of the target evidence text based on the fused word representation of each first word in the question text, specifically including the following sub-steps:

[0084] Step S71: construct a context based on the first word representation of the first word in the question text to obtain the first context representation of each first word, and construct a context based on the second word representation of the second word in the target evidence text to obtain the second context representation of each second word.

[0085] In this embodiment, for each first word in the question text, a context is constructed based on the first word representation of the first word in the question text to obtain a first context representation of the first word; and for each second word in the target evidence text, a context is constructed based on the second word representation of the second word in the target evidence text to obtain a second context representation of the second word. That is, based on the first word representation of each first word in the question text, a representation of each first word representation in the context is modeled to obtain the first context representation of each first word; and based on the second word representation of each second word in the target evidence text, a representation of each second word representation in the context is modeled to obtain the second context representation of each second word.

[0086] In a specific embodiment, the question answering model includes a context representation layer, which can capture information about adjacent words or even remote contexts. Therefore, by inputting the first word representation of each first word in the question text into the context representation layer of the question answering model, the first context representation of each first word containing the semantic information of the context and the semantic information of the question text at the overall level can be obtained, while maintaining the representation granularity at the word level; and, by inputting the second word representation of each second word in the target evidence text into the context representation layer of the question answering model, the second context representation of each second word containing the semantic information of the context and the semantic information of the target evidence text at the overall level can be obtained, while maintaining the representation granularity at the word level.

[0087] Optionally, in one embodiment, the context representation layer of the question answering model can be composed of a group of LSTM networks. An LSTM network can only capture information in one direction. The question answering model uses a group of LSTM networks, namely a bidirectional LSTM network, which can capture information in two directions, that is, it can capture context information. It can be understood that in other embodiments, the context representation layer of the question answering model can also be an RNN network or a GRU network, etc., which is not specifically limited here.

[0088] Step S72: construct word attention based on the first context representation of the first word and the second context representation of the second word to obtain the attention feature of the first word.

[0089] Since information related to the question text may be included in the corresponding target evidence text, it is necessary to establish a connection between the question text and the corresponding target evidence text to extract relevant information in the target evidence text. Therefore, in this embodiment, firstly, the word attention is constructed based on the first context representation of the first word and the second context representation of the second word to obtain the attention feature of the first word.

[0090] The attention feature includes the relevance of each second word to the first word, so in one embodiment, Figure 7 As shown, Figure 8 yes Figure 7 The flowchart of step S72 of an embodiment shown in FIG. 1 is a flowchart of determining the relevance of the first word to each second word in the target evidence text to obtain the attention feature of the first word, which specifically includes the following sub-steps:

[0091] Step S721: For each first word, obtain the similarity between the first context representation of the first word and the second context representation of each second word, and normalize the similarities corresponding to the first word to obtain the relevance of each second word to the first word.

[0092] In this embodiment, for each first word, the similarity between the first context representation of the first word and the second context representation of each second word is obtained. In one embodiment, the similarity between the first context representation of the first word and the second context representation of the second word can be the cosine distance between the first context representation of the first word and the second context representation of the second word. Of course, in other embodiments, the similarity between the first context representation of the first word and the second context representation of the second word can also be obtained by other methods.

[0093] In addition, in this embodiment, for each first word, each similarity corresponding to the first word is normalized to obtain the relevance of each second word to the first word. That is, the similarity between the first context representation of the first word and the second context representation of each second word is normalized to unify the similarity between the first context representation of the first word and the second context representation of each second word at the same order of magnitude, and the normalized similarity can be regarded as the relevance of the first word to each second word.

[0094] Step S722: Based on the relevance of each second word to the first word, obtain the attention feature of the first word.

[0095] In this embodiment, the attention feature of the first word is obtained based on the relevance of each second word to the first word. That is, the relevance of the first word to each second word constitutes the attention feature of the first word. Among them, the relevance of each second word to the first word can be expressed in the form of a matrix, that is, the relevance of the first word to each second word constitutes a matrix, and the matrix can be regarded as the attention feature of the first word.

[0096] In other embodiments, the question answering model includes a dual-path attention layer, which uses an attention mechanism to construct word attention based on a first context representation of a first word and a second context representation of a second word to obtain an attention feature of the first word.

[0097] Specifically, the attention mechanism is first used to process the first context representation of each first word in the question text and the second context representation of each second word in the target evidence text to obtain a matching matrix, where each element in the matching matrix corresponds to the dot product of the first context representation of the first word in the question text and the second context representation of the second word in the target evidence text. The specific formula is as follows:

[0098] M n (i,j)=QA(i)⊙D' n (j)

[0099] Where n represents the nth target evidence text; QA(i) represents the first context representation of the i-th first word in the question text; D' n (j) represents the second context representation of the j-th second word in the n-th target evidence text; M n (i,j) represents the similarity between the i-th first word in the question text and the j-th second word in the n-th target evidence text, that is, the similarity between the i-th first word and the j-th second word in the target evidence text is obtained by the dot product of the first context representation of the i-th first word and the second context representation of the j-th second word in the target evidence text.

[0100] Secondly, the elements in the matching matrix are normalized by column. The specific formula is as follows:

[0101] α n (i,j)=softmax(M n (1,j),…,M n (L Q ,j))(i)

[0102] Where n represents the nth target evidence text; α n(i, j) represents the column-normalized similarity between the i-th first word in the question text and the j-th second word in the n-th target evidence text. Each row of the column-normalized matching matrix represents the attention feature of a first word in the question text.

[0103] Step S73: Based on the attention feature of the first word and the second context representation of each second word, update the first attention representation of the first word.

[0104] In this embodiment, for each first word in the question text, the first attention representation of the first word is updated based on the attention feature of the first word and the second context representation of each second word. That is to say, based on the first context representation of each first word in the question text, the relevant information in the target evidence text, i.e., the second context representation of the second word, is extracted to obtain the attention feature of the first word.

[0105] Specifically, first, the relevance of the second word to the first word is used as a weighting coefficient to weight the second context representation of the corresponding second word; then the weighted second context representations of each second word are summed to obtain the weighted sum of the target evidence text corresponding to the first word. The specific formula is as follows:

[0106]

[0107] in, represents the weighted sum of the nth target evidence text corresponding to the i-th first word; α n (i,j) represents the column-normalized similarity between the first word of the i-th question text and the second word of the j-th target evidence text; D' n (j) represents the second context representation of the j-th second word in the n-th target evidence text.

[0108] Secondly, the weighted sum of the target evidence text corresponding to the first word is concatenated with the first context representation of the first word to update the first attention representation of the first word.

[0109] Step S74: Based on the first context representation of the first word, the attention feature and the first attention representation, a fused word representation of the first word is obtained.

[0110] In this embodiment, the first context representation, attention feature and first attention representation of the first word are fused to obtain a fused word representation of the first word. Specifically, the first context representation, attention feature and first attention representation of the first word are spliced ​​in the feature dimension, so as to achieve the fusion of the first context representation, attention feature and first attention representation of the first word. The fused fused word representation of the first word includes both the semantic information of the context and the semantic information of the target evidence text related to it, so that the semantic similarity features of the target evidence text determined based on the fused word representation of the first word are more accurate and reliable.

[0111] In order to determine the contribution of each first word in the semantic similarity judgment of the question text, in one embodiment, before the first context representation, attention feature and first attention representation of the first word are fused to obtain the fused word representation of the first word, a weight prediction is performed based on the first context representation of each first word in the question text to obtain the second sub-weight of each first word in the question text, that is, before the first context representation, attention feature and first attention representation of the first word are fused, a weight is assigned to each first word in the question text. In a specific embodiment, the question answering model includes a pooling layer, and the pooling layer is used to perform weight prediction on the first context representation of each first word in the question text to obtain the second sub-weight of each first word in the question text, wherein the specific formula is as follows:

[0112] g(i)=sigmoid(W·QA(i)+b)

[0113] Among them, g(i) represents the second sub-weight of the i-th first word in the question text, that is, the importance of the information at the position of the i-th first word in the question text; W and b both represent the network parameters of the pooling layer; QA(i) represents the first context representation of the i-th first word in the question text.

[0114] In a specific embodiment, Fig. 9 As shown, Fig. 9 yes Figure 7 The flowchart of step S74 of an embodiment shown in FIG. 1 shows that after obtaining the second sub-weight of each first word in the question text, determining the fused word representation of the first word specifically includes the following sub-steps:

[0115] Step S741: weight the first context representation, attention feature and first attention representation of the first word respectively based on the second sub-weight of the first word.

[0116] In this embodiment, the second sub-weight of the first word is used to weight the first context representation, the attention feature and the first attention representation of the first word respectively.

[0117] Step S742: concatenate the weighted first context representation of the first word, the attention feature, and the first attention representation to obtain a fused word representation of the first word.

[0118] In this embodiment, the weighted first context representation of the first word, the attention feature of the first word, and the first attention representation of the first word are spliced ​​to obtain a fused word representation of the first word. That is to say, the weighted first context representation of the first word, the attention feature of the first word, and the first attention representation of the first word are spliced ​​in the feature dimension, so as to achieve the fusion of the first context representation of the first word, the attention feature, and the first attention representation. The fused fused word representation of the first word includes both the semantic information of the context and the semantic information of the target evidence text related to it, so that the semantic similarity features of the target evidence text subsequently determined based on the fused word representation of the first word are more accurate and reliable. Among them, the specific formula for feature vector splicing is as follows:

[0119]

[0120] Among them, QAD(i) represents the fused word representation of the i-th first word in the question text; QA(i) represents the first context representation of the i-th first word in the question text; R(i) represents the attention feature of the i-th first word in the question text; F(i) represents the first attention representation of the i-th first word in the question text; g(i) represents the second sub-weight of the i-th first word in the question text; Represents feature vector concatenation.

[0121] Step S75: Perform semantic similarity modeling based on the fused word representation of each of the first words in the question text to obtain semantic similarity features of the target evidence text.

[0122] In this embodiment, semantic similarity modeling is performed based on the fused word representation of each first word in the question text to obtain the semantic similarity feature of the target evidence text. In a specific embodiment, the question answering model includes an inference layer, which uses a bidirectional LSTM network to perform semantic similarity modeling on the fused word representation of each first word in the question text to combine the fused word representation corresponding to each first word in the question text to obtain the semantic similarity feature of the target evidence text.

[0123] It should be noted that, when the target evidence set corresponding to the question text includes multiple target evidence texts, steps S71-S75, steps S721-S722 and steps S741-S742 will be executed accordingly for each target evidence text.

[0124] Step S122: Predictions are made based on the semantic similarity features of the target evidence texts in the target evidence sets of the question texts, respectively, to obtain first probability values ​​of the option texts constituting the question texts as candidate answer texts.

[0125] In this embodiment, predictions are made based on the semantic similarity features of the target evidence texts in the target evidence set of each question text, and the first probability values ​​of the option texts constituting each question cost as candidate answer texts are obtained. Since the semantic similarity features of the target evidence texts represent the semantic similarity between the target evidence texts and the question texts, by predicting the semantic similarity features of each target evidence text in the target evidence set corresponding to the question text, the correctness of the option texts constituting the question text can be determined, that is, the first probability value of the option texts constituting the question text as candidate answer texts can be determined.

[0126] It should be noted that, when the question stem text and each option text constitute different question texts respectively, step S121 - step S122 will be executed correspondingly for each question text.

[0127] Since each target evidence text in the target evidence set for the question text is not so important, in one embodiment, Fig.10 As shown, Fig.10 yes Figure 6 The flowchart of an embodiment of step S122 is shown. When predicting the semantic similarity features of the target evidence text in the target evidence set based on the question text to obtain the first probability value of the option text constituting the question text as the candidate answer text, it is also necessary to predict the relevance of each target evidence text in the target evidence set corresponding to the question text to highlight the truly relevant target evidence text and suppress the irrelevant target evidence text. Specifically, the following sub-steps are included:

[0128] Step S101: weighting the semantic similarity features of each target evidence text in the target evidence set of the question text based on the predicted relevance of each target evidence text, to obtain the weighted similarity features of each target evidence text in the target evidence set of the question text.

[0129] In this embodiment, for the target evidence set of each question text, the semantic similarity features of each target evidence text are weighted based on the predicted relevance of each target evidence text in the target evidence set of the question text, so as to obtain the weighted similarity features of each target evidence text in the target evidence set of the question text. Specifically, each target evidence text in the target evidence set of the question text corresponds to a predicted relevance; for each target evidence text in the target evidence set, the predicted relevance corresponding to the target evidence text is weighted to the semantic similarity features corresponding to the target evidence text, so as to obtain the weighted similarity features of the target evidence text. By weighting the predicted relevance of each target evidence text in the target evidence set of the question text to the semantic similarity features of the corresponding target evidence text, the truly relevant target evidence text in the target evidence set of the question text is highlighted, and the irrelevant target evidence text is suppressed.

[0130] In a specific embodiment, the question answering model includes a decision layer, and the decision layer includes a gating layer. The gating layer of the question answering model is applied to the semantic similarity features of each target evidence text in the target evidence set of the question text to obtain weighted similarity features corresponding to each target evidence text, thereby highlighting the relevant target evidence text in the target evidence set corresponding to the question text and suppressing irrelevant target evidence text.

[0131] Step S102: performing maximum pooling and average pooling on the weighted similar features of the target evidence text in the target evidence set based on the question text, respectively, to obtain the first pooling features and the second pooling features of the target evidence set.

[0132] In this embodiment, for the target evidence set corresponding to each question text, the weighted similar features of the target evidence texts in the target evidence set based on the question text are respectively subjected to maximum pooling and average pooling to obtain the first pooling features and the second pooling features of the target evidence set. That is to say, for the target evidence set corresponding to each question text, the weighted similar features of each target evidence text in the target evidence set corresponding to the question text are put together, and the maximum pooling process is performed to obtain the first pooling features of the target evidence set corresponding to the question text, and the maximum pooling represents the support level of the target evidence text with the highest support level; and the weighted similar features of each target evidence text in the target evidence set corresponding to the question text are put together, and the average pooling process is performed to obtain the second pooling features of the target evidence set corresponding to the question text, and the average pooling represents the average support level of all target evidence texts in the target evidence set.

[0133] Step S103: Predicting based on the first pooled features and the second pooled features of the target evidence set of the question text, to obtain a first probability value of the option text of the question text as a candidate answer text.

[0134] In this embodiment, for the first pooled features and the second pooled features of the target evidence set of each question text, prediction is performed based on the first pooled features and the second pooled features of the target evidence set of the question text to obtain a first probability value of the option text of the question text as a candidate answer text. In a specific embodiment, the question answering model includes a decision layer, and the decision layer includes a multi-layer feedforward network; the first pooled features and the second pooled features of the target evidence set of the question text are input into the multi-layer feedforward network of the decision layer of the question answering model, and the multi-layer feedforward network outputs a scalar value, which represents the comprehensive support degree of the target evidence set corresponding to the question text for the option text constituting the question text as a candidate answer text, that is, the scalar value represents the credibility of the option text constituting the question text as a candidate answer text.

[0135] In one embodiment, the scalar value corresponding to the question text obtained above can be directly used as the first probability value of the option text constituting the question text as the candidate answer text. It can be understood that in other embodiments, after obtaining the scalar values ​​corresponding to all question texts, each scalar value is normalized, and the normalized scalar value corresponding to the question text is used as the first probability value of the option text constituting the question text as the candidate answer text.

[0136] See also Fig.11 , Fig.11 yes Figure 1 It should be noted that if there is substantially the same result, this embodiment does not use Fig.11 The process sequence shown is limited. Fig.11 As shown, in this embodiment, when the question stem text and each option text constitute different question texts respectively, the candidate answer result includes each option text as a first probability value of the candidate answer text, and determining the evaluation result of the target evidence set specifically includes the following steps:

[0137] Step S111: obtaining a first evaluation parameter based on the first probability value of each option text as a candidate answer text.

[0138] In this embodiment, the first evaluation parameter is obtained based on the first probability value of each option text as a candidate answer text. In other words, the first evaluation parameter can be obtained based on the first probability value of each option text as a candidate answer text.

[0139] In one embodiment, the maximum first probability value among the first probability values ​​of each selected text as a candidate answer text is used as the first evaluation parameter. In a specific embodiment, the candidate answer result, i.e., the first probability value of each option text as a candidate answer text, is obtained by analyzing the question answering model, and the last layer output of the question answering model is a neural network classifier f(x) of a normalized exponential (Softmax) function, wherein the first probability value of the jth option text as a candidate answer text is f(x|j), and the specific formula for obtaining the first evaluation parameter is as follows:

[0140]

[0141] Among them, κ represents the first evaluation parameter; f(x|j) represents the first probability value of the jth option text as the candidate answer text; y represents the number of option texts.

[0142] Step S112: perform information quantity prediction based on the first evaluation parameter to obtain an evaluation result.

[0143] In this embodiment, the amount of information is predicted based on the first evaluation parameter to obtain an evaluation result that characterizes whether the amount of information of the target evidence set is sufficient. In one embodiment, the evidence evaluation model predicts the amount of information based on the first evaluation parameter to obtain an evaluation result that characterizes whether the amount of information of the target evidence set is sufficient; and the evidence evaluation model is obtained by training based on sample data after the question answering model training converges, the sample data includes a sample question text and a number of sample evidence texts, the sample data is annotated with a sample tag, and the sample tag characterizes whether the question answering model after training converges gives a correct answer to the sample question text based on the several sample evidence texts.

[0144] In one embodiment, if Fig.12 As shown, Fig.12 This is a flow chart of an embodiment of randomly deactivating a question answering model provided by the present application. The evaluation result of the target evidence set can also be determined by randomly deactivating the question answering model. At this time, before the information amount is predicted based on the first evaluation parameter to obtain the evaluation result, the following sub-steps are specifically included:

[0145] Step S121: the question answering model is randomly deactivated, and the question text and the target evidence set are analyzed based on the randomly deactivated question answering model to obtain a reference answer result for the question text.

[0146] In this embodiment, the question answering model is randomly deactivated. Random dropout is a common technique in deep learning to reduce overfitting and improve model generalization error. Some neural network units and corresponding connections are randomly discarded according to the Bernouolli distribution. Random dropout can be used as a method to model the prediction confidence of the neural network model. In addition, in this embodiment, the question text and the target evidence set are analyzed based on the question answering model after random deactivation to obtain a reference answer result of the question text. Among them, the reference answer result includes the second probability value of each option text as a candidate answer text.

[0147] Specifically, assume that the question answering model f applies K different random deactivation masks, each time for the input x to the question answering model f; calculate the corresponding predicted probability distribution Perform forward propagation on the neural network and get the result in, denotes the model parameters affected by dropout.

[0148] Step S122: Obtain a second evaluation parameter based on the probability distribution variance of the reference answer results obtained by several random inactivations.

[0149] In this embodiment, the second evaluation parameter is obtained based on the probability distribution variance of the reference answer results obtained by several random inactivations. The larger the probability distribution variance of the reference answer results, the less certain the question answering model is about the correct answer, which indirectly indicates that the information content of the target evidence set is not sufficient. Among them, the probability distribution variance of the reference answer results can be expressed as follows:

[0150]

[0151] At this time, the information amount is predicted based on the first evaluation parameter and the second evaluation parameter to obtain an evaluation result.

[0152] See also Fig.13 , Fig.13 is a flow chart of an embodiment of the evidence prediction model training method provided by the present application. It should be noted that if there are substantially the same results, the embodiments of the present application are not based on Fig.13 The process sequence shown is limited. Fig.13 As shown, this embodiment includes:

[0153] Step S131: obtaining the predicted answer result of the question answering model after training convergence based on several sample evidence texts to the sample question text.

[0154] In this embodiment, the predicted answer result of the question answering model after training convergence based on a number of sample evidence texts to the sample question text is obtained. Among them, the number of sample evidence texts is not limited and can be specifically set according to actual use needs.

[0155] In one embodiment, the predicted answer result of the question answering model after training convergence based on several sample evidence texts to the sample question text can be obtained from local storage or cloud storage. It can be understood that in other embodiments, the question answering model after training convergence can also be used to process several sample evidence texts and sample question texts in real time to obtain the predicted answer result.

[0156] The sample data consisting of the sample question text and several sample evidence texts are labeled with sample labels, and the sample labels are determined based on the predicted answer result. Specifically, the correct answer to the sample question text is known, and the correct answer to the sample question text is used to determine whether the predicted answer result is correct; if the predicted answer is correct, it is determined that the sample label represents that the question answering model after the training convergence gives the correct answer to the sample question based on several sample evidence texts, wherein 1 can be used to represent that the correct answer is given; if the predicted answer is wrong, it is determined that the sample label represents that the question answering model after the training convergence does not give the correct answer to the sample question based on several sample evidence texts, wherein 0 can be used to represent that the wrong answer is given.

[0157] Step S132: Based on the predicted answer result, the sample evaluation parameters are obtained, and the sample evaluation parameters are predicted based on the evidence evaluation model to obtain a predicted label.

[0158] In this embodiment, based on the predicted answer result, the sample evaluation parameter is obtained, and the sample evaluation parameter is predicted based on the evidence evaluation model to obtain the prediction mark, wherein the prediction mark indicates whether the information content of several sample evidence texts is sufficient.

[0159] In one embodiment, when the sample question stem text and each sample option text constitute different sample question texts, the first sample evaluation parameter can be determined based on the probability value of each sample option text as a sample answer text; then, the first sample evaluation parameter is predicted based on the evidence evaluation model to obtain a prediction mark. Of course, in other embodiments, on the basis of obtaining the first sample evaluation parameter, the second evaluation parameter is obtained by randomly deactivating the question answering model; then, the first evaluation parameter and the second evaluation parameter are predicted based on the evidence evaluation model to obtain a prediction mark.

[0160] Step S133: Based on the difference between the sample label and the prediction label, adjust the network parameters of the evidence evaluation model.

[0161] In this embodiment, the network parameters of the evidence evaluation model are adjusted based on the difference between the sample label and the prediction label. Specifically, the loss of the evidence evaluation model is determined by using the difference between the sample label and the prediction label; then, the network parameters of the evidence evaluation model are adjusted by using the loss of the evidence evaluation model.

[0162] Among them, the binary cross entropy loss function is used to determine the loss of the evidence evaluation model. The loss function is specifically as follows:

[0163]

[0164] Among them, z i Indicates whether the question answering model gives the correct answer; c(x) represents the confidence score given by the evidence evaluation model prediction; x represents the sample evaluation parameter.

[0165] See also Fig.14 , Fig.14 It is a structural diagram of an embodiment of a question answering device provided by the present application. The question answering device 140 includes an evidence retrieval module 141, a question analysis module 142, an evidence evaluation module 143 and a loop execution module 144. The evidence retrieval module 141 is used to retrieve the target evidence text in the candidate evidence set based on the question text, and incorporate the target evidence text into the target evidence set of the question text; the question analysis module 142 is used to analyze based on the question text and the target evidence set to obtain the candidate answer result of the question text; wherein the candidate answer result at least includes the candidate answer text; the evidence evaluation module 143 is used to evaluate the target evidence set based on the candidate answer result to obtain the evaluation result of the target evidence set; wherein the evaluation result characterizes whether the information amount of the target evidence set is sufficient; the loop execution module 144 is used to respond to the evaluation result not meeting the preset condition, re-execute the step of retrieving the target evidence text in the candidate evidence set based on the question text, until the evaluation result meets the preset condition, and obtain the target answer text based on the latest candidate answer result.

[0166] Among them, the evidence retrieval module 141 is used to retrieve the target evidence text in the candidate evidence set based on the question text, specifically including: obtaining the evidence retrieval weight updated after the last retrieval of the target evidence text; wherein the evidence retrieval weight includes the first sub-weight of each first word in the question text; based on the first sub-weight of each first word, the first word representation is weighted respectively to obtain the first question representation of the question text; based on the first question representation, each candidate evidence text in the candidate evidence set is scored to determine the target evidence text.

[0167] Among them, the first question representation includes a weighted word representation of the first word, and the evidence retrieval module 141 is used to re-execute when the evaluation result does not meet the preset conditions. The question answering method also includes: obtaining the second word representation of each second word in the latest target evidence text; for each first word, based on the feature similarity between the weighted word representation and each second word representation, obtaining the matching score of the first word; wherein the matching score indicates the degree to which the first word is matched by the target evidence text; based on the matching score of each first word, selecting at least one first word as the target word; based on the attenuation coefficient, weighting the latest first sub-weight of each target word to obtain an updated evidence retrieval weight.

[0168] Among them, when the question text and each option text constitute different question texts respectively, the candidate answer results include the first probability values ​​of each option text as a candidate answer text; the evidence evaluation module 143 is used to evaluate the target evidence set based on the candidate answer results to obtain the evaluation results of the target evidence set, specifically including: obtaining a first evaluation parameter based on the first probability value of each option text as a candidate answer text; and performing information amount prediction based on the first evaluation parameter to obtain an evaluation result.

[0169] Among them, the above-mentioned candidate answer results are obtained by analyzing the question answering model, and the evidence evaluation module 143 is also used to predict the amount of information based on the first evaluation parameter to obtain the evaluation result, specifically including: randomly deactivating the question answering model, and analyzing the question text and the target evidence set based on the question answering model after random deactivation to obtain a reference answer result of the question text; wherein the reference answer result includes a second probability value of each option text as a candidate answer text; the second evaluation parameter is obtained based on the probability distribution variance of the reference answer result obtained by several random deactivations; the evidence evaluation module 143 is used to predict the amount of information based on the first evaluation parameter to obtain the evaluation result, specifically including: predicting the amount of information based on the first evaluation parameter and the second evaluation parameter to obtain the evaluation result.

[0170] Among them, the above-mentioned candidate answer results are obtained by analyzing the question answering model, and the information amount prediction is obtained by predicting the evidence evaluation model. The evidence evaluation model is trained based on sample data after the question answering model training converges. The sample data includes a sample question text and several sample evidence texts. The sample data is annotated with sample tags, and the sample tags represent whether the question answering model after training convergence gives a correct answer to the sample question text based on several sample evidence texts.

[0171] Among them, the training steps of the above-mentioned evidence evaluation model include: obtaining the predicted answer results of the question answering model for the sample question text based on several sample evidence texts after training convergence; wherein, the sample label is determined based on the predicted answer result; based on the predicted answer result, the sample evaluation parameters are obtained, and the sample evaluation parameters are predicted based on the evidence evaluation model to obtain the predicted label; wherein the predicted label represents whether the amount of information of several sample evidence texts is sufficient; based on the difference between the sample label and the predicted label, the network parameters of the evidence evaluation model are adjusted.

[0172] Among them, when the question text and each option text constitute different question texts respectively, different question texts correspond to their own target evidence sets, and the candidate answer results include the first probability values ​​of each option text as a candidate answer text; the question analysis module 142 is used to perform analysis based on the question text and the target evidence set to obtain the candidate answer results of the question text, specifically including: for the question text and each target evidence text in its target evidence set, reasoning is performed based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text to obtain the semantic similarity features of each target evidence text; wherein the semantic similarity features represent the semantic similarity between the target evidence text and the question text; prediction is performed based on the semantic similarity features of the target evidence text in the target evidence set of each question text to obtain the first probability values ​​of the option texts constituting each question text as candidate answer texts.

[0173] Among them, the question analysis module 142 is used to perform reasoning based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text, respectively, to obtain the semantic similarity features of each target evidence text, specifically including: constructing a context based on the first word representation of the first word in the question text to obtain the first context representation of each first word, and constructing a context based on the second word representation of the second word in the target evidence text to obtain the second context representation of each second word; constructing word attention based on the first context representation of the first word and the second context representation of the second word to obtain the attention feature of the first word; wherein the attention feature includes the relevance of each second word to the first word; updating the first attention representation of the first word based on the attention feature of the first word and the second context representation of each second word; fusing the first context representation, attention feature and first attention representation of the first word to obtain the fused word representation of the first word; performing semantic similarity modeling based on the fused word representation of each first word in the question text to obtain the semantic similarity features of the target evidence text.

[0174] Among them, the question analysis module 142 is used to construct word attention based on the first context representation of the first word and the second context representation of the second word to obtain the attention feature of the first word, specifically including: for each first word, obtaining the similarity between the first context representation of the first word and the second context representation of each second word, and normalizing the similarities corresponding to the first word to obtain the relevance of each second word to the first word; based on the relevance of each second word to the first word, obtaining the attention feature of the first word.

[0175] Among them, the question analysis module 142 is also used to fuse the first context representation, attention feature and first attention representation of the first word to obtain the fused word representation of the first word, specifically including: performing weight prediction based on the first context representation of each first word in the question text to obtain the second sub-weight of each first word in the question text; the question analysis module 142 is used to fuse the first context representation, attention feature and first attention representation of the first word to obtain the fused word representation of the first word, specifically including: weighting the first context representation, attention feature and first attention representation of the first word respectively based on the second sub-weight of the first word; splicing the weighted first context representation, attention feature and first attention representation of the first word to obtain the fused word representation of the first word.

[0176] Among them, the question analysis module 142 is used to make predictions based on the semantic similarity features of the target evidence texts in the target evidence set of each question text, respectively, to obtain the first probability values ​​of the option texts constituting each question text as candidate answer texts, specifically including: weighting the semantic similarity features of each target evidence text in the target evidence set of the question text based on the predicted relevance of each target evidence text in the target evidence set of the question text, to obtain the weighted similarity features of each target evidence text in the target evidence set of the question text; performing maximum pooling and average pooling based on the weighted similarity features of the target evidence texts in the target evidence set of the question text, to obtain the first pooling features and the second pooling features of the target evidence set; making predictions based on the first pooling features and the second pooling features of the target evidence set of the question text, to obtain the first probability values ​​of the option texts of the question text as candidate answer texts.

[0177] See also Fig.15 , Fig.151 is a schematic diagram of the structure of an embodiment of an electronic device provided by the present application. The electronic device 150 includes a memory 151 and a processor 152 coupled to each other, and the processor 152 is used to execute program instructions stored in the memory 151 to implement the steps of any of the above-mentioned question answering method embodiments. In a specific implementation scenario, the electronic device 150 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 150 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0178] Specifically, the processor 152 is used to control itself and the memory 151 to implement the steps of any of the above-mentioned question answering method embodiments. The processor 152 may also be referred to as a CPU (Central Processing Unit). The processor 152 may be an integrated circuit chip having the ability to process signals. The processor 152 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 152 may be implemented by an integrated circuit chip.

[0179] See also Fig.16 , Fig.16 It is a structural diagram of an embodiment of a computer-readable storage medium provided by the present application. The computer-readable storage medium 160 of the embodiment of the present application stores a program instruction 161, and when the program instruction 161 is executed, it implements the method provided by any embodiment of the question answering method of the present application and any non-conflicting combination. Among them, the program instruction 161 can form a program file and be stored in the above-mentioned computer-readable storage medium 160 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of each implementation method of the present application. The aforementioned computer-readable storage medium 160 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, or a terminal device such as a computer, a server, a mobile phone, and a tablet.

[0180] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0181] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A question answering method, characterized in that: include: Obtaining a second word representation of each second word in the latest target evidence text; For each first word in the question text, a matching score of the first word is obtained based on the feature similarity between the weighted word representation of the first word and the feature similarity between the weighted word representation of the first word and the second word representation; wherein the matching score indicates the degree to which the first word is matched by the latest target evidence text; Selecting at least one of the first words as a target word based on the matching scores of the first words; Based on the decay coefficient, the latest first sub-weight of each target word is weighted respectively to obtain the evidence retrieval weight updated after the last retrieval of the target evidence text; wherein the evidence retrieval weight includes the first sub-weight of each first word in the question text; Based on the first sub-weights of the first words, the first word representations of the first words are weighted respectively to obtain a first question representation of the question text; Scoring each candidate evidence text in the candidate evidence set based on the first question representation, determining a target evidence text, and incorporating the target evidence text into the target evidence set of the question text; Analyzing the question text and the target evidence set to obtain candidate answer results for the question text; wherein the candidate answer results at least include candidate answer texts; Evaluating the target evidence set based on the candidate answer results to obtain an evaluation result of the target evidence set; wherein the evaluation result indicates whether the information content of the target evidence set is sufficient; In response to the evaluation result not satisfying the preset condition, the step of obtaining the second word representation of each second word in the latest target evidence text and subsequent steps are re-executed until the evaluation result satisfies the preset condition, and the target answer text is obtained based on the latest candidate answer result.

2. The method according to claim 1, characterized in that: In the case where the question stem text and each option text constitute different question texts respectively, the candidate answer result includes the first probability value of each option text as the candidate answer text respectively; The step of evaluating the target evidence set based on the candidate answer result to obtain the evaluation result of the target evidence set includes: Based on the first probability values ​​of each option text as the candidate answer text, a first evaluation parameter is obtained; An information quantity prediction is performed based on the first evaluation parameter to obtain the evaluation result.

3. The method according to claim 2, characterized in that The candidate answer result is obtained by analyzing the question answering model. Before performing information amount prediction based on the first evaluation parameter to obtain the evaluation result, the method further includes: The question answering model is randomly deactivated, and the question text and the target evidence set are analyzed based on the randomly deactivated question answering model to obtain a reference answer result of the question text; wherein the reference answer result includes a second probability value of each option text as the candidate answer text; A second evaluation parameter is obtained based on the probability distribution variance of the reference answer results obtained by several random dropouts; The performing information amount prediction based on the first evaluation parameter to obtain the evaluation result includes: The information amount is predicted based on the first evaluation parameter and the second evaluation parameter to obtain the evaluation result.

4. The method according to claim 2, characterized in that: The candidate answer result is obtained by analyzing the question answering model, and the information amount prediction is obtained by predicting the evidence evaluation model, and the evidence evaluation model is trained based on sample data after the question answering model training converges, and the sample data includes a sample question text and several sample evidence texts. The sample data is annotated with sample tags, and the sample tags represent whether the question answering model after training convergence gives a correct answer to the sample question text based on the several sample evidence texts.

5. The method according to claim 4, characterized in that The training steps of the evidence evaluation model include: Obtaining a predicted answer result of the question answering model after training convergence based on the plurality of sample evidence texts to the sample question text; wherein the sample label is determined based on the predicted answer result; Based on the predicted answer result, a sample evaluation parameter is obtained, and the sample evaluation parameter is predicted based on the evidence evaluation model to obtain a prediction mark; wherein the prediction mark indicates whether the information content of the plurality of sample evidence texts is sufficient; Based on the difference between the sample markers and the prediction markers, the network parameters of the evidence assessment model are adjusted.

6. The method according to claim 1, characterized in that In the case where the question stem text and each option text constitute different question texts respectively, each of the different question texts corresponds to its own target evidence set, and the candidate answer result includes each option text as a first probability value of the candidate answer text; The step of analyzing the question text and the target evidence set to obtain candidate answer results for the question text includes: For each of the target evidence texts in the question text and its target evidence set, reasoning is performed based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text to obtain a semantic similarity feature of each of the target evidence texts; wherein the semantic similarity feature represents the semantic similarity between the target evidence text and the question text; Predictions are made based on the semantic similarity features of the target evidence texts in the target evidence sets of the question texts, respectively, to obtain the first probability values ​​of the option texts constituting the question texts as the candidate answer texts.

7. The method according to claim 6, characterized in that The reasoning based on the first word representation of each first word in the question text and the second word representation of each second word in the target evidence text to obtain the semantic similarity features of each target evidence text includes: Constructing a context based on a first word representation of a first word in the question text to obtain a first context representation of each of the first words, and constructing a context based on a second word representation of a second word in the target evidence text to obtain a second context representation of each of the second words; constructing word attention based on a first context representation of the first word and a second context representation of the second word to obtain an attention feature of the first word; wherein the attention feature includes a relevance of each of the second words to the first word; Based on the attention feature of the first word and the second context representation of each of the second words, updating the first attention representation of the first word; Obtain a fused word representation of the first word based on a first context representation of the first word, the attention feature, and the first attention representation; Semantic similarity modeling is performed based on the fused word representation of each of the first words in the question text to obtain semantic similarity features of the target evidence text.

8. The method according to claim 7, characterized in that The constructing word attention based on the first context representation of the first word and the second context representation of the second word to obtain the attention feature of the first word includes: For each of the first words, obtaining the similarity between the first context representation of the first word and the second context representation of each of the second words, and normalizing the similarities corresponding to the first words to obtain the relevance of each of the second words to the first word; Based on the relevance of each of the second words to the first word, an attention feature of the first word is obtained.

9. The method according to claim 7, characterized in that: Before fusing the first context representation, the attention feature and the first attention representation based on the first word to obtain a fused word representation of the first word, the method further includes: Performing weight prediction based on the first context representation of each of the first words in the question text to obtain a second sub-weight of each of the first words in the question text; The fusing the first context representation, the attention feature and the first attention representation based on the first word to obtain a fused word representation of the first word includes: weighting the first context representation, the attention feature, and the first attention representation of the first word respectively based on the second sub-weight of the first word; The weighted first context representation, the attention feature, and the first attention representation of the first word are concatenated to obtain a fused word representation of the first word.

10. The method according to claim 6, characterized in that The step of respectively predicting the semantically similar features of the target evidence texts in the target evidence sets of the question texts to obtain the first probability values ​​of the option texts constituting the question texts as the candidate answer texts includes: Based on the predicted relevance of each target evidence text in the target evidence set of the question text, the semantic similarity features of each target evidence text are weighted respectively to obtain the weighted similarity features of each target evidence text in the target evidence set of the question text; Based on the weighted similarity features of the target evidence text in the target evidence set of the question text, maximum pooling and average pooling are performed respectively to obtain a first pooling feature and a second pooling feature of the target evidence set; Prediction is performed based on the first pooled features and the second pooled features of the target evidence set of the question text to obtain a first probability value of the option text of the question text as the candidate answer text.

11. A question answering device, characterized in that: include: An evidence retrieval module, used to obtain the second word representation of each second word in the latest target evidence text; For each first word in the question text, a matching score of the first word is obtained based on the feature similarity between the weighted word representation of the first word and the feature similarity between the weighted word representation of the first word and the feature similarity between the weighted word representation of the second word; wherein the matching score indicates the degree to which the first word is matched by the latest target evidence text; based on the matching score of each first word, at least one first word is selected as a target word; based on the attenuation coefficient, the latest first sub-weight of each target word is weighted to obtain an evidence retrieval weight updated after the last retrieval of the target evidence text; wherein the evidence retrieval weight includes the first sub-weight of each first word in the question text; based on the first sub-weight of each first word, the first word representation of the first word is weighted to obtain a first question representation of the question text; based on the first question representation, each candidate evidence text in the candidate evidence set is scored to determine the target evidence text, and the target evidence text is included in the target evidence set of the question text; A question analysis module, configured to analyze the question text and the target evidence set to obtain candidate answer results for the question text; wherein the candidate answer results at least include candidate answer texts; An evidence evaluation module, used to evaluate the target evidence set based on the candidate answer results to obtain an evaluation result of the target evidence set; wherein the evaluation result indicates whether the information content of the target evidence set is sufficient; A loop execution module is used to re-execute the second word representation of each second word in the latest target evidence text and its subsequent steps in response to the evaluation result not satisfying the preset condition, until the evaluation result satisfies the preset condition, and the target answer text is obtained based on the latest candidate answer result.

12. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the question answering method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the question answering method according to any one of claims 1 to 10.

Images