Automatic question answering method, apparatus, device and medium

Abstract

The application discloses an automatic question answering method and device, equipment and medium, and relates to the field of machine learning. The method comprises the following steps: obtaining an input question sentence vector of an input question and K groups of question and answer sentence vector templates, each group of the question and answer sentence vector templates comprising a question text sentence vector and a reply text sentence vector which appear in pairs; based on a first attention mechanism, fusing the input question sentence vector and the question text sentence vector to obtain a question comprehensive category vector, and based on a second attention mechanism, fusing the input question sentence vector and the reply text sentence vector to obtain a reply comprehensive category vector; obtaining a category probability according to the input question sentence vector, the question comprehensive category vector and the reply comprehensive category vector; and determining the reply text corresponding to the target sentence vector template as the output reply of the input question according to the category probability. The application can obtain a more accurate reply.

Description

Technical Field

[0001] This application relates to the field of machine learning, and in particular to an automatic question answering method, apparatus, device, and medium. Background Technology

[0002] Automated question answering refers to computer devices automatically analyzing and understanding user questions and directly returning the desired answer. For example, if a user enters the question "What is the weather like today?", the computer device will reply and display "It is sunny today".

[0003] After acquiring the input question, the relevant technology performs word segmentation on the question to obtain the segmentation results. Then, based on the entities and attributes extracted from the segmentation results, a semantic logic expression corresponding to the question is generated. A query statement is then generated based on the semantic logic expression. The query statement is used to query the knowledge graph to obtain the query results, and the corresponding answer to the question is obtained based on the query results.

[0004] In related technologies, the scope of knowledge that a knowledge graph can cover is limited. When the query statement does not exist in the knowledge graph, it is impossible to get a response through the knowledge graph. Summary of the Invention

[0005] This application provides an automatic question-answering method, apparatus, device, and medium. The method establishes connections between the input question and the question text, as well as between the input question and the answer text, mining information contained in the input question from different perspectives, which can greatly improve the accuracy of automatic question-answering responses. The technical solution is as follows:

[0006] According to one aspect of this application, an automatic question-answering method is provided, the method comprising:

[0007] Obtain the input question sentence vector and K sets of question-and-answer sentence vector templates. Each set of question-and-answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1.

[0008] Based on the first attention mechanism, the input question sentence vector and the question text sentence vector are fused to obtain a comprehensive question category vector; and based on the second attention mechanism, the input question sentence vector and the response text sentence vector are fused to obtain a comprehensive response category vector.

[0009] Based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector, a category probability is obtained. The category probability is used to represent the probability that the input question belongs to the target question and answer sentence vector template in the K groups of question and answer sentence vector templates.

[0010] Based on the category probability, the response text corresponding to the target sentence vector template is determined as the output response to the input question.

[0011] According to another aspect of this application, an automatic question-answering device is provided, the device comprising:

[0012] The acquisition module is used to acquire the input question sentence vector and K sets of question and answer sentence vector templates. Each set of question and answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1.

[0013] The fusion module is used to fuse the input question sentence vector and the question text sentence vector based on a first attention mechanism to obtain a comprehensive question category vector, and to fuse the input question sentence vector and the response text sentence vector based on a second attention mechanism to obtain a comprehensive response category vector.

[0014] The classification module is used to obtain the category probability based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector. The category probability is used to represent the probability that the input question belongs to the target question answer sentence vector template in the K groups of question answer sentence vector templates.

[0015] The classification module is further configured to determine the response text corresponding to the target sentence vector template as the output response to the input question based on the category probability.

[0016] In an optional design of this application, the fusion module is further configured to perform vector interaction between the input question sentence vector and the question text sentence vectors within the K sets of question-and-answer sentence vector templates to obtain K sets of question importance weights. The k1-th question importance weight in the K sets of question importance weights is used to represent the relevance between the input question and the question text corresponding to the k1-th question-and-answer sentence vector template, 1≤k1≤K, where k1 is a positive integer. Based on the K sets of question importance weights, K question attention weights are obtained. K question category vectors are obtained, where the k2-th question category vector in the K sets of question category vectors is used to represent the mean of the question text sentence vectors within the k2-th question-and-answer sentence vector template, 1≤k2≤K, where k2 is a positive integer. The K question attention weights and the K question category vectors are weighted and combined to calculate the comprehensive question category vector.

[0017] In an optional design of this application, the fusion module is further configured to take the top a values ​​of the largest question importance weight in each of the K groups of question importance weights, combine them to obtain a question importance weight sequence, the question importance weight sequence including a*K question importance weights; and extract the K question attention weights from the question importance weight sequence.

[0018] In an optional design of this application, the fusion module is further configured to perform vector interaction between the input question sentence vector and the response text sentence vectors in the K sets of question-and-answer sentence vector templates to obtain K sets of response importance weights. The k3rd response importance weight in the K sets of response importance weights is used to represent the relevance between the input question and the response text corresponding to the k3rd question-and-answer sentence vector template, 1≤k3≤K, where k3 is a positive integer. Based on the K sets of response importance weights, K response attention weights are obtained. K response category vectors are obtained, where the k4th response category vector in the K sets of response category vectors is used to represent the mean of the response text sentence vectors in the k4th question-and-answer sentence vector template, 1≤k4≤K, where k4 is a positive integer. The K response attention weights and the K response category vectors are weighted and combined to calculate the comprehensive response category vector.

[0019] In an optional design of this application, the fusion module is further configured to take the top b values ​​of the largest response importance weight in each of the K groups of response importance weights, combine them to obtain a response importance weight sequence, the response importance weight sequence including b*K response importance weights; and extract the K response attention weights from the response importance weight sequence.

[0020] In an optional design of this application, the acquisition module is further configured to invoke a first attention computing network to obtain the comprehensive question category vector based on the first attention mechanism and according to the input question sentence vector and the question text sentence vector; and invoke a second attention computing network to obtain the comprehensive response category vector based on the second attention mechanism and according to the input question sentence vector and the response text sentence vector.

[0021] In an optional design of this application, the classification module is further configured to concatenate the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector to obtain a concatenated vector; and call the classification network to convert the concatenated vector into the category probability.

[0022] In an optional design of this application, the acquisition module is further configured to invoke a sentence vector generation network to convert the input question into the input question sentence vector.

[0023] In an optional design of this application, the classification module is further configured to determine the sentence vector template corresponding to the highest category probability among the category probabilities as the target sentence vector template; and to use the response text corresponding to the target sentence vector template as the output response to the input question.

[0024] In an optional design of this application, the classification module is further configured to extract keywords from the input question; determine n key information corresponding to the keywords from the knowledge graph, and at least two candidate options corresponding to each key information, where n is a positive integer; in response to obtaining a target option from the at least two candidate options corresponding to each key information, extract the output response from the knowledge graph according to the target option; or, in response to the absence of the output response in the knowledge graph, execute the above-described automatic question answering method.

[0025] In an optional design of this application, the classification module is further configured to: determine the entities to be graphed and their relationships in the information to be graphed; obtain a knowledge graph to be fused based on the entities to be graphed and their relationships; align the knowledge graph to be fused with an existing knowledge graph to obtain a first structured knowledge graph and a second structured knowledge graph; and fuse the first structured knowledge graph and the second structured knowledge graph to obtain the knowledge graph.

[0026] In an optional design of this application, the classification module is further configured to display the input question and a first option corresponding to the input question; in response to a selection operation on a target option in the first option, determine a target entity from the knowledge graph based on the target option; in response to the target entity belonging to a response entity, display the output response corresponding to the target entity, wherein the response entity is the entity corresponding to the output response; or, in response to the absence of the target entity in the knowledge graph, execute the above-described automatic question answering method.

[0027] According to another aspect of this application, a method for training an automatic question-answering model is provided, the method comprising:

[0028] Obtain a training dataset, which includes sample input questions and the corresponding ground truth labels for the sample input questions;

[0029] Obtain the sample input question sentence vector;

[0030] Based on the first attention mechanism, a comprehensive category vector of sample questions is obtained according to the sample input question sentence vector and the question text sentence vector; and based on the second attention mechanism, a comprehensive category vector of sample responses is obtained according to the sample input question sentence vector and the response text sentence vector.

[0031] Based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample response comprehensive category vector, the sample category probability is obtained. The sample category probability is used to represent the probability that the sample input question belongs to the sample target sentence vector template in the K groups of question and answer sentence vector templates.

[0032] Based on the sample category probability, the response text corresponding to the sample target sentence vector template is determined as the sample output response to the sample input question;

[0033] The automatic question answering model is trained based on the sample output responses and the real annotations.

[0034] According to another aspect of this application, a training apparatus for an automatic question-answering model is provided, the apparatus comprising:

[0035] The sample acquisition module is used to acquire a training dataset, which includes sample input questions and the ground truth labels corresponding to the sample input questions.

[0036] The sample input module is used to obtain the sample input question sentence vector of the sample input question;

[0037] The sample fusion module is used to obtain a comprehensive category vector of sample questions based on the first attention mechanism and the sample input question sentence vector and the question text sentence vector, and to obtain a comprehensive category vector of sample responses based on the second attention mechanism and the sample input question sentence vector and the response text sentence vector.

[0038] The sample classification module is used to obtain the sample category probability based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample response comprehensive category vector. The sample category probability is used to represent the probability that the sample input question belongs to the sample target sentence vector template in the K groups of question and answer sentence vector templates.

[0039] The sample classification module is further configured to determine the response text corresponding to the sample target sentence vector template as the sample output response to the sample input question based on the sample category probability.

[0040] The sample training module is used to train the automatic question answering model based on the sample output responses and the real annotations.

[0041] According to another aspect of this application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the automatic question answering method as described above, or to implement the training method of the automatic question answering model as described above.

[0042] According to another aspect of this application, a computer storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the program code being loaded and executed by a processor to implement the automatic question answering method as described above, or to implement the training method of the automatic question answering model as described above.

[0043] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the automatic question-answering method described above, or to implement the automatic question-answering model training method described above.

[0044] The beneficial effects of the technical solutions provided in this application include at least the following:

[0045] By fusing the input question sentence vector and the question text sentence vector, a comprehensive question category vector is obtained; similarly, by fusing the input question sentence vector and the response text sentence vector, a comprehensive response category vector is obtained. Then, based on the comprehensive question category vector, the comprehensive question category vector, and the input question sentence vector, a category probability is calculated, which determines the output response corresponding to the input question. Because this method determines the relationship between the input question and the question text, as well as the relationship between the input question and the response text, it fuses matching results from two different perspectives, thus improving the accuracy of automatic question answering. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application;

[0048] Figure 2 This is a schematic diagram of the structure of an automatic question-answering model provided in an exemplary embodiment of this application;

[0049] Figure 3 This is a flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application;

[0050] Figure 4 This is a flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application;

[0051] Figure 5 This is a flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application;

[0052] Figure 6 This is a schematic diagram of the interface of an automatic question-answering method provided in an exemplary embodiment of this application;

[0053] Figure 7 This is a schematic diagram illustrating the generation of a knowledge graph provided in an exemplary embodiment of this application;

[0054] Figure 8 This is a flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application;

[0055] Figure 9 This is a schematic diagram of the interface of an automatic question-answering method provided in an exemplary embodiment of this application;

[0056] Figure 10 This is a flowchart illustrating a training method for an automatic question-answering model provided in an exemplary embodiment of this application;

[0057] Figure 11 This is a flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application;

[0058] Figure 12 This is a schematic diagram of the structure of an automatic question-answering device provided in an exemplary embodiment of this application;

[0059] Figure 13 This is a schematic diagram of the structure of a training device for an automatic question-answering model provided in an exemplary embodiment of this application;

[0060] Figure 14 This is a schematic diagram of the structure of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0062] First, let's introduce the terms used in the embodiments of this application:

[0063] Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0064] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0065] Machine Learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0066] Knowledge graphs, known in library and information science as knowledge domain visualization or knowledge domain mapping maps, are a series of various graphics that display the development process and structural relationships of knowledge. They use visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, draw, and display knowledge and the interrelationships between them. Policy knowledge graphs are a type of knowledge graph, which are knowledge graphs that are generated by breaking down policy documents and transforming them into graph data.

[0067] Automatic question answering system: It is an advanced form of information retrieval system that can answer user questions in natural language with accurate and concise natural language.

[0068] Figure 1 A schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application is shown. The computer system 100 includes a terminal 120 and a server 140.

[0069] Terminal 120 runs an application related to automatic question answering. This application can be a small app within an app, a dedicated application, or a web client. For example, a user performs an operation related to automatic question answering on terminal 120; for instance, the user enters a question, and terminal 120 automatically answers the question through the aforementioned application. Terminal 120 is at least one of a smartphone, tablet computer, e-book reader, MP3 player, MP4 player, laptop computer, and desktop computer.

[0070] Terminal 120 is connected to server 140 via a wireless network or a wired network.

[0071] Server 140 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 140 provides background services for the automated question-answering application and sends the output answer to the input question to terminal 120. Optionally, server 140 undertakes the main computing work, and terminal 120 undertakes the secondary computing work; or, server 140 undertakes the secondary computing work, and terminal 120 undertakes the main computing work; or, server 140 and terminal 120 jointly perform computing using a distributed computing architecture.

[0072] Figure 2 A schematic diagram of the structure of an automatic question answering model provided in an exemplary embodiment of this application is shown. The automatic question answering model includes: a sentence vector generation network 21, an attention calculation network 22, and a classification network 23.

[0073] The sentence vector generation network 21 is used to convert the input question into sentence vectors. The input to the sentence vector generation network 21 is the input question, and the output is the sentence vector q of the input question. Optionally, the model structure of the sentence vector generation network 21 can be at least one of the following: ALBERT (A Lite Bidirectional Encoder Representations from Transformers), BERT (Bidirectional Encoder Representations from Transformers), or FastText. This application does not limit the specific model structure of the sentence vector generation network 21.

[0074] Attention computation network 22 is used to compute category vectors based on an attention mechanism. Attention computation network 22 includes a first attention computation network 201 and a second attention computation network 202. The first attention computation network 201 computes a comprehensive category vector for the question text, and correspondingly, the second attention computation network 202 computes a comprehensive category vector for the response text. The input to the first attention computation network 201 is the question sentence vector q, and the output is the comprehensive question category vector u. The input to the second attention computation network 202 is the question sentence vector q, and the output is the comprehensive response category vector v.

[0075] For example, after inputting the input question sentence vector q into the first attention computation network 201, the dot product calculation of the input question sentence vector q is carried out. get This represents the vector of the i-th question text in the k-th question-and-answer vector template. Let β′ represent the question importance weight of the i-th question text vector in the k-th question-and-answer sentence vector template. Through max-pooling, β′ is... k Convert to β k ,β k This represents the attention weight of the k-th question-and-answer sentence vector template based on the question text at the category level. β is calculated through combination. k and l k This yields the comprehensive problem category vector u, l k This represents the mean of the question text vectors in the k-th question-and-answer vector template.

[0076] For example, after inputting the input question sentence vector q into the second attention computation network 202, the dot product is calculated between the input question sentence vector q and... get This represents the vector of the i-th reply text sentence in the k-th question-and-answer sentence vector template. Let γ′ represent the question importance weight of the i-th response text vector in the k-th question-and-answer vector template. Through max-pooling, γ′ is... k Convert to γ k γ k This represents the attention weight of the k-th question-and-answer sentence vector template based on the response text at the category level. γ is calculated through combination. k and l′ k The response is a comprehensive category vector v, l′ k This represents the mean of the response text vectors in the k-th question-and-answer vector template.

[0077] Classification network 23 is used to determine the probability that the input question belongs to the target question-answer vector template. The input to classification network 23 is the input question vector q, the comprehensive question category vector u, and the comprehensive answer category vector v; the output is the category probability q. For example, classification network 23 includes a classifier that outputs the three vectors q, u, and v as category probabilities.

[0078] Figure 3 A flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0079] Step 302: Obtain the input question sentence vector and K sets of question and answer sentence vector templates. Each set of question and answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1.

[0080] The input question sentence vector is obtained by converting the input question into a sentence vector. Optionally, a sentence vector generation network is invoked to convert the input question into an input question sentence vector. The sentence vector generation network includes at least one of the ALBERT model, BERT model, and FastText model.

[0081] The question text vectors and answer text vectors in each question-and-answer vector template can have a one-to-one, one-to-many, or many-to-many relationship. For example, a question-and-answer vector template may include a pair of question text vectors and one answer text vector. Alternatively, it may include a pair of question text vectors and multiple answer text vectors. Or, it may include a pair of multiple question text vectors and one answer text vector. Or, it may include a pair of multiple question text vectors and multiple answer text vectors. In this embodiment, the example of an answer vector template including pairs of multiple question text vectors and multiple answer text vectors will be used for illustration.

[0082] The question text vectors are obtained from the question text, and the answer text vectors are obtained from the answer text, with a corresponding relationship between the question and answer texts. Different question-and-answer vector templates correspond to different question types. For example, the first set of question-and-answer vector templates corresponds to the question type "medical insurance," and the second set corresponds to the template "pension insurance."

[0083] Optionally, each set of question-and-answer sentence vector templates corresponds to one answer text and multiple question texts. The answer results for the multiple question texts are the aforementioned answer texts. One answer text corresponds to multiple answer text sentence vectors, and the multiple question texts correspond to multiple question text sentence vectors respectively. For example, the first sentence vector generation network is invoked to convert r question texts into question text sentence vectors. The answer text is divided into t answer statements; the second sentence vector generation network is invoked to convert the t answer statements into answer text sentence vectors, where r and t are integers greater than 1.

[0084] For example, when responding to policy documents, the question texts are different ways of asking questions about the policy. In an automated question-answering scenario, the same question can be phrased in multiple ways. For instance, "How much more do I need to pay for my health insurance?" and "What is the amount of health insurance?" both express the same semantics. Therefore, r questions with the same semantics are counted as question texts and converted into question text sentence vectors. On the other hand, responses in policy documents are sometimes very long. Therefore, the policy text is split into t sentences, and each t sentence is converted into a response text sentence vector.

[0085] In this embodiment of the application, in order to improve efficiency, the K sets of question and answer vector templates are optionally preset.

[0086] Optionally, if the input question is a policy issue, then the question text is a policy issue and the response text is a policy response.

[0087] Step 304: Based on the first attention mechanism, fuse the input question sentence vector and the question text sentence vector to obtain the comprehensive question category vector; and based on the second attention mechanism, fuse the input question sentence vector and the response text sentence vector to obtain the comprehensive response category vector.

[0088] The comprehensive category vector for a question is based on the question text. It is a category vector of the input question obtained by combining the sentence vector of the input question and the sentence vector of the question text, using the question text as the standard.

[0089] The response category vector represents a comprehensive category vector based on the response text. It is the category vector of the input question obtained by combining the sentence vector of the input question and the sentence vector of the response text, using the response text as the standard.

[0090] The first attention mechanism and the second attention mechanism can be the same or different. Optionally, the first attention mechanism and / or the second attention mechanism includes at least one of the following: self-attention, additive attention, multiplicative attention, key-value attention, and joint attention.

[0091] Optionally, the first attention computation network is invoked, and based on the first attention mechanism, a comprehensive question category vector is obtained according to the input question sentence vector and question text sentence vector.

[0092] Optionally, a second attention computation network is invoked. Based on the second attention mechanism, a comprehensive category vector of the response is obtained according to the input question sentence vector and the response text sentence vector.

[0093] Step 306: Based on the input question sentence vector, the question comprehensive category vector, and the answer comprehensive category vector, obtain the category probability. The category probability is used to represent the probability that the input question belongs to the target question and answer sentence vector template in the K groups of question and answer sentence vector templates.

[0094] Optionally, a classifier can be invoked to obtain the category probability based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector.

[0095] The target question-and-answer vector template is any one of the K sets of question-and-answer vector templates.

[0096] Step 308: Based on the category probability, determine the response text corresponding to the target sentence vector template as the output response to the input question.

[0097] Since there are K category probabilities, it is necessary to determine the target sentence vector template from the K sets of question-and-answer sentence vector templates corresponding to the K category probabilities. Optionally, the sentence vector template corresponding to the largest category probability is determined as the target sentence vector template; the response text corresponding to the target sentence vector template is used as the output response to the input question.

[0098] In summary, this embodiment obtains a comprehensive question category vector by fusing the input question sentence vector and the question text sentence vector, and a comprehensive response category vector by fusing the input question sentence vector and the response text sentence vector. Then, based on the comprehensive question category vector, the comprehensive question category vector, and the input question sentence vector, a category probability is obtained, and the output response corresponding to the input question is determined by the category probability. Because this method determines the relationship between the input question and the question text, as well as the relationship between the input question and the response text, it fuses matching results from two different perspectives, thus improving the accuracy of automatic question answering.

[0099] In the following embodiments, detailed first and second attention calculation mechanisms are provided. The resulting comprehensive question category vector and comprehensive answer category vector reflect the category characteristics of the input question from different perspectives. Therefore, the category probabilities obtained from the comprehensive question category vector and comprehensive answer category vector are more accurate and comprehensive, resulting in a more accurate final output answer. On the other hand, in the process of calculating attention weights, all question text sentence vectors and answer text sentence vectors from the K sets of question-and-answer sentence vector templates are taken into consideration to ensure the accuracy of attention weights, resulting in a more accurate final output answer.

[0100] Figure 4 A flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0101] Step 401: Obtain the input question sentence vector and K sets of question and answer sentence vector templates.

[0102] Each of the K sets of question-and-answer vector templates includes a pair of question text vectors and answer text vectors.

[0103] The input question sentence vector is obtained by converting the input question into a sentence vector. Optionally, a sentence vector generation network is invoked to convert the input question into an input question sentence vector. The sentence vector generation network includes at least one of the ALBERT model, BERT model, and FastText model.

[0104] Optionally, each set of question-and-answer sentence vector templates corresponds to one answer text and multiple question texts. The answer results for the multiple question texts are the aforementioned answer texts. One answer text corresponds to multiple answer text sentence vectors, and the multiple question texts correspond to multiple question text sentence vectors respectively. For example, the first sentence vector generation network is invoked to convert r question texts into question text sentence vectors. The answer text is divided into t answer statements; the second sentence vector generation network is invoked to convert the t answer statements into answer text sentence vectors, where r and t are integers greater than 1.

[0105] For example, the question sentence vector generation network is invoked to convert the question text into question text sentence vectors; the response sentence vector network is invoked to convert the response text into response text sentence vectors; and the question sentence vector generation network is invoked again to convert the input question into an input question sentence vector. The question sentence vector generation network is denoted as ALBERT. q The response vector generation network is denoted as ALBERT. a Then we have:

[0106]

[0107]

[0108] q = ALBERT q ;

[0109] in, This represents the vector of the i-th question text in the k-th question-and-answer vector template. Let represent the i-th response text vector in the k sets of question-and-answer vector templates, and q represent the input question vector.

[0110] Step 402: Perform vector interaction between the input question sentence vector and the question text sentence vectors in the K sets of question and answer sentence vector templates to obtain the K sets of question importance weights.

[0111] The k1-th question importance weight in the K-group question importance weights is used to represent the relevance between the input question and the question text corresponding to the k1-th question-answer vector template, where 1≤k1≤K and k1 is a positive integer.

[0112] Optionally, the input question sentence vector is multiplied by the question text sentence vectors in the K sets of question and answer sentence vector templates to obtain the K sets of question importance weights.

[0113] For example, the vector dot product process of each question text vector and the input question vector within the K sets of question-and-answer sentence vector templates is as follows:

[0114]

[0115] Where q represents the input question vector, This represents the vector of the i-th question text in the k-th question-and-answer vector template. This represents the question importance weight corresponding to the i-th question text vector in the k-th question-and-answer vector template.

[0116] Step 403: Based on the importance weights of the K groups of questions, obtain the attention weights of the K questions.

[0117] Optionally, this step includes the following sub-steps:

[0118] 1. Take the top a values ​​of the largest problem importance weight in each of the K groups of problem importance weights, and combine them to obtain the problem importance weight sequence, where a is a positive integer.

[0119] For example, the maximum value of the problem importance weight in each of the K groups is taken, and combined to obtain a problem importance weight sequence, which includes K problem importance weights. Then, the i-th problem importance weight in the problem importance weight sequence is: k represents the vector template of the k-th question-and-answer sentence corresponding to the importance weight of the question.

[0120] Optionally, by using a max-pool operation, the maximum value of the problem importance weight in each of the K groups of problem importance weights is taken, and the groups are combined to obtain a sequence of problem importance weights.

[0121] 2. Extract K problem attention weights from the problem importance weight sequence.

[0122] Optionally, if the problem importance weight sequence is obtained by taking the maximum value of each of the K sets of problem importance weights, the normalized problem importance weight sequence yields K problem attention weights. For example, the problem attention weight β = softmax(β′), where β′ represents the problem importance weight sequence and softmax() represents the normalized exponential function.

[0123] Optionally, the problem importance weight sequence is obtained by taking the largest a values ​​of each of the K groups of problem importance weights, where a is a positive integer greater than 1, and averaging the largest a values ​​of each group of problem importance weights to obtain K average values; normalizing the K average values ​​yields K problem attention weights.

[0124] Optionally, the problem importance weight sequence is obtained by taking the largest a values ​​of each of the K groups of problem importance weights, where a is a positive integer greater than 1, and taking the median of the largest a values ​​of each group of problem importance weights to obtain K medians; and normalizing the K medians to obtain K problem attention weights.

[0125] Step 404: Obtain K problem category vectors.

[0126] The k2th question category vector in the K question category vectors is used to represent the mean of the question text sentence vectors in the k2th question and answer sentence vector template, 1≤k2≤K, where k2 is a positive integer.

[0127] To improve computational efficiency, similar to the question-and-answer sentence vector template, the K question category vectors can be pre-set.

[0128] Step 405: Combine the K question attention weights and K question category vectors in a weighted manner to calculate the comprehensive question category vector.

[0129] The comprehensive category vector for a question is based on the question text. It is a category vector of the input question obtained by combining the sentence vector of the input question and the sentence vector of the question text, using the question text as the standard.

[0130] For example, a comprehensive problem category vector Where, β k Let l represent the question attention weight corresponding to the k-th question-and-answer sentence vector template. k This represents the question attention weight corresponding to the k-th question-and-answer vector template.

[0131] Step 406: Perform vector interaction between the input question sentence vector and the response text sentence vector in the K sets of question and answer sentence vector templates to obtain the importance weights of the K sets of responses.

[0132] The importance weight of the k3rd group of responses in the K-group response importance weights is used to represent the relevance between the input question and the response text corresponding to the k3rd group of question-and-answer sentence vector templates, where 1≤k3≤K and k3 is a positive integer.

[0133] Optionally, the input question sentence vector is multiplied by the response text sentence vectors in the K sets of question and answer sentence vector templates to obtain the importance weights of the K sets of responses.

[0134] For example, the vector dot product process for each response text vector and the input question vector within the K sets of question-and-answer sentence vector templates is as follows:

[0135]

[0136] Where q represents the input question vector, This represents the vector of the i-th reply text sentence in the k-th question-and-answer sentence vector template. This represents the importance weight of the response corresponding to the i-th response text vector in the k-th question-and-answer vector template.

[0137] Step 407: Obtain the attention weights of K responses based on the importance weights of the K groups of responses.

[0138] Optionally, this step includes the following sub-steps:

[0139] 1. Take the top b values ​​of the largest response importance weight in each of the K response importance weight groups, and combine them to obtain the response importance weight sequence, where b is a positive integer.

[0140] For example, the maximum value of the response importance weight in each of the K groups is taken, and combined to obtain a response importance weight sequence, which includes K response importance weights. Then, the i-th response importance weight in the response importance weight sequence is: k represents the vector template of the k-th question and answer statement corresponding to the importance weight of the response.

[0141] Optionally, by using a max-pool operation, the maximum value of the question importance weight in each of the K groups of response importance weights is taken, and the result is combined to obtain the response importance weight sequence.

[0142] 2. Extract K question attention weights from the response importance weight sequence.

[0143] Optionally, if the response importance weight sequence is obtained by taking the maximum value of each of the K response importance weights, the normalized response importance weight sequence yields K question attention weights. For example, the response attention weight γ = softmax(γ′), where β′ represents the response importance weight sequence and softmax() represents the normalized exponential function.

[0144] Optionally, the response importance weight sequence is obtained by taking the top a values ​​of the largest response importance weight in each of the K groups of response importance weights, where a is a positive integer greater than 1, and taking the average of the top a values ​​of the largest response importance weight in each group of response importance weights to obtain K average values; normalizing the K average values ​​yields K response attention weights.

[0145] Optionally, the response importance weight sequence is obtained by taking the first a values ​​of the largest response importance weight in each of the K groups of response importance weights, where a is a positive integer greater than 1, and taking the median of the first a values ​​of the largest response importance weight in each group of response importance weights to obtain K medians; and normalizing the K medians to obtain K response attention weights.

[0146] Step 408: Obtain K response category vectors.

[0147] The k4th group of response category vectors in the K response category vectors is used to represent the mean of the question text sentence vectors in the k4th group of question and answer sentence vector templates, 1≤k4≤K, where k4 is a positive integer.

[0148] To improve computational efficiency, similar to the question-and-answer sentence vector template, the K response category vectors can be pre-set.

[0149] Step 409: Combine the K response attention weights and K response category vectors in a weighted manner to calculate the comprehensive response category vector.

[0150] The response category vector represents a comprehensive category vector based on the response text. It is the category vector of the input question obtained by combining the sentence vector of the input question and the sentence vector of the response text, using the response text as the standard.

[0151] For example, the response is a comprehensive category vector. Where, β k Let l′ represent the response attention weight corresponding to the k-th question-and-answer vector template. k This represents the response attention weight corresponding to the k-th question-and-answer vector template.

[0152] It should be noted that steps 402-405 and 406-409 are not in any particular order. You can execute steps 402-405 first, followed by steps 406-409. Alternatively, you can execute steps 406-409 first, followed by steps 402-405. Or, you can execute steps 402-405 and 406-409 simultaneously.

[0153] Step 410: Concatenate the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector to obtain the concatenated vector.

[0154] For example, by concatenating vectors, the input question sentence vector, the question comprehensive category vector, and the answer comprehensive category vector are concatenated to obtain the concatenated vector. For example, the concatenated vector z = Concat[u, v, q], where u represents the question comprehensive category vector, v represents the answer comprehensive category vector, q represents the input question sentence vector, and Concat means vector concatenation.

[0155] Step 411: Call the classification network to convert the concatenated vectors into class probabilities.

[0156] Optionally, the classifier within the classification network can be invoked to convert the concatenated vector into class probabilities. For example, the class probability p = Softmax(FC(Concat[u, v, q])), where Concat[u, v, q] represents the concatenated vector, FC() represents the fully connected network layer, and Softmax() represents the normalized exponential function.

[0157] Step 412: Based on the category probability, determine the response text corresponding to the target sentence vector template as the output response to the input question.

[0158] Since there are K category probabilities, it is necessary to determine the target sentence vector template from the K sets of question-and-answer sentence vector templates corresponding to the K category probabilities. Optionally, the sentence vector template corresponding to the largest category probability is determined as the target sentence vector template; the response text corresponding to the target sentence vector template is used as the output response to the input question.

[0159] In summary, this embodiment obtains a comprehensive question category vector by fusing the input question sentence vector and the question text sentence vector, and a comprehensive response category vector by fusing the input question sentence vector and the response text sentence vector. Then, based on the comprehensive question category vector, the comprehensive question category vector, and the input question sentence vector, a category probability is obtained, and the output response corresponding to the input question is determined by the category probability. Because this method determines the relationship between the input question and the question text, as well as the relationship between the input question and the response text, it fuses matching results from two different perspectives, thus improving the accuracy of automatic question answering.

[0160] Because of the attention mechanism, the resulting question-comprehensive category vector and response-comprehensive category vector can reflect the category characteristics of the input question from different perspectives. Therefore, the category probabilities obtained from the question-comprehensive category vector and response-comprehensive category vector are more accurate and comprehensive, resulting in a more accurate output response.

[0161] The following embodiments will provide another automatic question-answering method, which is implemented through an indicator graph and can achieve automatic question-answering relatively quickly. Furthermore, by employing a bottom-up automatic question-answering method, the key information needed to answer the input question can be quickly located, and the output answer can be determined based on the user's response.

[0162] Figure 5 A flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0163] Step 501: Extract keywords from the input question.

[0164] Optionally, a keyword extraction network can be invoked to extract keywords from the input question.

[0165] Optionally, a word segmentation network can be invoked to segment the input question into words and obtain the segmentation results; based on an attention mechanism, keywords in the segmentation results can be determined.

[0166] Optionally, a semantic recognition network can be invoked to obtain the semantic recognition results of the input question; keywords can then be determined based on the semantic recognition results.

[0167] For example, such as Figure 6As shown, if the text content of the input question 601 is "How much does the individual pay?", then the keyword for the input question 601 is "individual payment".

[0168] Step 502: Identify n key pieces of information corresponding to the keywords from the knowledge graph, and at least two candidate options corresponding to each key piece of information.

[0169] A knowledge graph consists of entities and entity relationships, where entity relationships represent the relationship between two entities. For example, in a knowledge graph, the keyword entity corresponding to a keyword is identified; based on the entity relationships of the keyword entity, n key pieces of information corresponding to the keyword are identified, along with at least two candidate options for each key piece of information.

[0170] Optionally, a set of synonyms for the keywords is determined; based on the set of synonyms, n key pieces of information are determined from the knowledge graph, and at least two candidate options corresponding to each key piece of information.

[0171] For example, such as Figure 6 As shown, after the user inputs question 601, key information 602 is displayed based on the input question 601. Key information 602 includes "What is your insurance type?", with corresponding candidate options being "Resident Medical Insurance; Employee Medical Insurance; Urban Employee Commercial Supplementary Medical Insurance"; Key information 602 also includes "What is your policy status?", with corresponding candidate options being "Employed Employee; Urban and Rural Resident; Urban Employee Basic Medical Insurance Participant; Retiree; Flexible Employment Person"; Key information 602 also includes "What is your identity characteristic?", with corresponding candidate options being "New Urban Employee Basic Medical Insurance Participant; Under 45 years old; Old Urban Employee Basic Medical Insurance Participant; Retirement with outstanding payments; 45 years old and before retirement".

[0172] Optionally, the following steps are taken: First, determine the entities to be graphed and their relationships within the information to be graphed; second, obtain the knowledge graph to be fused based on the entities and their relationships; third, align the knowledge graph to be fused with the existing knowledge graph to obtain a first structured knowledge graph and a second structured knowledge graph; fourth, fuse the first and second structured knowledge graphs to obtain the final knowledge graph. Here, the first structured graph is the aligned knowledge graph to be fused, and the second structured graph is the aligned existing knowledge graph.

[0173] For example, let's take the construction of an indicator map of medical insurance policies as an example for illustration, such as... Figure 7As shown, through entity recognition, relation extraction, and event extraction, the entities to be graphed and their relationships in the medical insurance policy 701 to be graphed are determined, and a knowledge graph to be fused is established. Then, through graph alignment operation, the knowledge graph to be fused and the existing policy graph 702 are aligned to obtain the first structured policy graph 703 and the second structured policy graph 704, respectively. The first structured policy graph 703 and the second structured policy graph 704 are merged to obtain the knowledge graph 705.

[0174] Step 503: In response to obtaining the target option from at least two candidate options corresponding to each key information, extract the output response from the knowledge graph based on the target option.

[0175] In a knowledge graph, the target options corresponding to each key piece of information will point to the same entity, and that entity will output the response.

[0176] Optionally, in response to a click on the target option, the output answer corresponding to the input question is displayed.

[0177] Optionally, in response to input of the target option, the output answer corresponding to the input question is displayed.

[0178] For example, such as Figure 6 As shown, after the user enters the text "Resident Medical Insurance, Urban and Rural Residents, Under 45 Years Old" (603), the output response 604 is displayed. The text content of the output response 604 is "The personal contribution for urban and rural residents is 280 yuan / year".

[0179] Step 504: In response to the absence of an output answer in the knowledge graph, execute the automatic question answering method described above.

[0180] The scope of knowledge that a knowledge graph can cover is limited. When there is no corresponding output answer for an input question in the knowledge graph, a method such as... Figure 3 or Figure 4 The illustrated embodiment implements automatic question answering.

[0181] In summary, this embodiment provides an automatic question answering method based on knowledge graphs. By adopting a bottom-up automatic question answering approach, it can quickly locate the key information needed to answer the input question and determine the output answer based on the user's response, thus facilitating the user's query of the knowledge graph.

[0182] The following embodiments will provide another automatic question-answering method, which is implemented through an indicator graph and can achieve automatic question-answering relatively quickly. Furthermore, by employing a top-down automatic question-answering method, the user can continuously determine the desired options, ultimately obtaining a more accurate output response.

[0183] Figure 8A flowchart illustrating an exemplary embodiment of the automatic question-answering method provided in this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0184] Step 801: Display the input question and the first option corresponding to the input question.

[0185] Optionally, in a knowledge graph, the input question corresponds to an input entity, and the first option is an entity or entity relationship corresponding to the input entity.

[0186] Step 802: In response to the selection operation on the target option in the first option, determine the target entity from the knowledge graph based on the target option.

[0187] Optionally, when the first option represents an entity relation in the knowledge graph, the target entity is determined in the knowledge graph based on the input entity corresponding to the input question and the first entity relation corresponding to the first option. Therefore, in the knowledge graph, the input entity, the first entity relation, and the target entity form a triplet.

[0188] Optionally, the following steps are taken: First, determine the entities to be graphed and their relationships within the information to be graphed; second, obtain the knowledge graph to be fused based on the entities and their relationships; third, align the knowledge graph to be fused with the existing knowledge graph to obtain a first structured knowledge graph and a second structured knowledge graph; fourth, fuse the first and second structured knowledge graphs to obtain the final knowledge graph. Here, the first structured graph is the aligned knowledge graph to be fused, and the second structured graph is the aligned existing knowledge graph.

[0189] For example, let's take the construction of an indicator map of medical insurance policies as an example for illustration, such as... Figure 7 As shown, through entity recognition, relation extraction, and event extraction, the entities to be graphed and their relationships in the medical insurance policy 701 to be graphed are determined, and a knowledge graph to be fused is established. Then, through graph alignment operation, the knowledge graph to be fused and the existing policy graph 702 are aligned to obtain the first structured policy graph 703 and the second structured policy graph 704, respectively. The first structured policy graph 703 and the second structured policy graph 704 are merged to obtain the knowledge graph 705.

[0190] Step 803: In response to the target entity belonging to the reply entity, display the output reply corresponding to the target entity. The reply entity is the entity corresponding to the output reply.

[0191] The response entity refers to the entity consisting of the response text.

[0192] Optionally, in response to the target entity not being a responding entity, a sub-question and a second option are displayed.

[0193] For example, such as Figure 9 As shown, the user interface will first display "Types of insurance you may want to inquire about:" Urban employees Industrial and commercial supplementary medical insurance ; Major disease supplementary insurance for disadvantaged groups ; Major disease insurance for urban and rural residents ; Maternity insurance ; Resident medical insurance ; Employees Employee medical insurance The user clicked " Employee medical insurance ", further displaying "Employee Medical Insurance: Benefit policies ; Insurance participation policies The user clicked " Participation Insurance participation policies This further demonstrates the "Enrollment Policy:" Flexible employees ; Retirees ; On-the-job employees The user clicked " On-the-job staff Employees ", further showing "Current Employees: Employees of特困enterprises (It seems there is a misspelling here, should it be "special hardship enterprises"?) ; From 45 years old to before retirement ; Under 45 years old The user clicked " Full From 45 years old to before retirement ", further indicating "from age 45 to retirement: Unit contribution to personal account ; Personal contribution to personal account Contribution ; Unit contribution ; Personal contribution ; Lower limit of contribution base ; Upper limit of contribution base ; Contribution base The user clicked " Upper limit of contribution base Upper limit The output reply states that "the upper limit of the contribution base for employed workers aged 45 and above until retirement is 300% of the average social wage in the city in the previous year."

[0194] Step 804: In response to the absence of the target entity in the knowledge graph, execute the automatic question answering method described above.

[0195] The scope of knowledge that a knowledge graph can cover is limited. When there is no corresponding output answer for an input question in the knowledge graph, a method such as... Figure 3 or Figure 4 The illustrated embodiment implements automatic question answering.

[0196] In summary, this embodiment provides a knowledge graph-based automatic question answering method. Employing a top-down approach, it can achieve relatively rapid automatic question answering. Users can continuously select the desired options, ultimately obtaining a more accurate output response.

[0197] Figure 10A flowchart illustrating the training method of the automatic question-answering model provided in an embodiment of this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0198] Step 1001: Obtain the training dataset.

[0199] The training dataset includes sample input questions and their corresponding ground truth annotations.

[0200] Step 1002: Obtain the sample input question sentence vector.

[0201] Optionally, a sentence vector generation network is invoked to convert the sample input question into a sentence vector. The sentence vector generation network includes at least one of the ALBERT model, BERT model, and FastText model.

[0202] Step 1003: Based on the first attention mechanism, obtain the sample question comprehensive category vector according to the sample input question sentence vector and question text sentence vector, and based on the second attention mechanism, obtain the sample response comprehensive category vector according to the sample input question sentence vector and response text sentence vector.

[0203] The sample question comprehensive category vector represents the comprehensive category vector based on the question text. It is the category vector of the sample input question obtained by combining the sentence vector of the sample input question and the sentence vector of the question text, using the question text as the standard.

[0204] The sample response comprehensive category vector represents the comprehensive category vector based on the response text. It is the category vector of the sample input question obtained by combining the sentence vector of the sample input question and the sentence vector of the response text, using the response text as the standard.

[0205] The first attention mechanism and the second attention mechanism may be the same or different. Optionally, the first attention mechanism and / or the second attention mechanism include at least one of the following: self-attention mechanism, additive attention mechanism, dot product attention mechanism, key value attention mechanism, and joint attention mechanism.

[0206] Optionally, the first attention computation network is invoked. Based on the first attention mechanism, the comprehensive category vector of the sample question is obtained according to the sample input question sentence vector and question text sentence vector.

[0207] Optionally, a second attention computation network is invoked. Based on the second attention mechanism, a comprehensive category vector of the sample response is obtained according to the sample input question sentence vector and the response text sentence vector.

[0208] Step 1004: Obtain the sample category probability based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample response comprehensive category vector.

[0209] The sample category probability is used to represent the probability that the input question belongs to the sample target question and answer vector template in the K groups of question and answer vector templates.

[0210] Optionally, a classifier can be invoked to obtain the sample category probability based on the sample input question sentence vector, question comprehensive category vector, and response comprehensive category vector.

[0211] The sample target question-and-answer vector template is any one of the K sets of question-and-answer vector templates.

[0212] Step 1005: Based on the sample category probability, determine the response text corresponding to the sample target sentence vector template as the sample output response to the sample input question.

[0213] Since there are K sample category probabilities, it is necessary to determine the target sentence vector template from the K sets of question and answer sentence vector templates corresponding to the K sample category probabilities. Optionally, the sentence vector template corresponding to the largest category probability in the sample category probabilities is determined as the target sentence vector template; the sample response text corresponding to the target sentence vector template is used as the sample output response to the sample input question.

[0214] Step 1006: Train the automatic question answering model based on the sample output responses and real annotations.

[0215] Optionally, the automatic question answering model can be trained using an error backpropagation algorithm based on sample output responses and real annotations.

[0216] In summary, this embodiment provides a training method for an automatic question answering model. This method can train the automatic question answering model relatively quickly. The trained automatic question answering model can determine the relationship between the input question and the question text, as well as the relationship between the input question and the answer text, and integrate the matching results from two different perspectives to improve the accuracy of automatic question answering.

[0217] Figure 11 A flowchart illustrating the training method of the automatic question-answering model provided in an embodiment of this application is shown. This method can be... Figure 1 The method, executed by the terminal 120, server 140, or other computer device shown, includes the following steps:

[0218] Step 1101: Obtain the policy question sentence vector and K sets of question and answer sentence vector templates. Each set of question and answer sentence vector templates includes a pair of policy question sentence vectors and policy response sentence vectors, where K is an integer greater than 1.

[0219] Policy question sentence vectors are obtained by converting policy questions into sentence vectors. Optionally, a sentence vector generation network is invoked to convert policy questions into policy question sentence vectors. The sentence vector generation network includes at least one of the ALBERT model, BERT model, and FastText model.

[0220] Step 1102: Based on the first attention mechanism, fuse the policy question sentence vector and the policy response sentence vector to obtain the comprehensive question category vector; and based on the second attention mechanism, fuse the policy question sentence vector and the policy response sentence vector to obtain the comprehensive response category vector.

[0221] The comprehensive category vector of policy issues represents a comprehensive category vector based on policy issues. It is a category vector of policy issues obtained by combining the sentence vectors of policy issues with the sentence vectors of policy issues.

[0222] The comprehensive category vector of responses represents a comprehensive category vector based on policy responses. It is a category vector of policy questions obtained by combining the sentence vector of policy questions and the sentence vector of policy responses, using policy responses as the standard.

[0223] Step 1103: Obtain the category probability based on the policy question sentence vector, the question comprehensive category vector, and the response comprehensive category vector.

[0224] Category probability is used to represent the probability that a policy question belongs to the target question-answer vector template in the K groups of question-answer vector templates.

[0225] Optionally, a classifier can be invoked to obtain the category probabilities based on the policy question sentence vector, the question comprehensive category vector, and the response comprehensive category vector.

[0226] The target question-and-answer vector template is any one of the K sets of question-and-answer vector templates.

[0227] Step 1104: Based on the category probability, determine the policy response corresponding to the target sentence vector template as the output response to the policy question.

[0228] Since there are K category probabilities, it is necessary to determine the target sentence vector template from the K sets of question and answer sentence vector templates corresponding to the K category probabilities. Optionally, the sentence vector template corresponding to the largest category probability is determined as the target sentence vector template; the policy response corresponding to the target sentence vector template is taken as the output response to the policy question.

[0229] In summary, this embodiment obtains a comprehensive question category vector by fusing policy question sentence vectors and policy response sentence vectors, and obtains a comprehensive response category vector by fusing policy question sentence vectors and policy response sentence vectors. Then, based on the comprehensive question category vector, the comprehensive question category vector, and the policy question sentence vector, a category probability is obtained, and the output response corresponding to the policy question is determined by the category probability. Because this method determines the relationships between policy questions and between policy questions and policy responses, it fuses matching results from two different perspectives, improving the accuracy of automatic question answering. This facilitates users' access to government policies and meets the rapidly growing and changing question-and-answer needs of users online.

[0230] Optionally, the computer system involved in the embodiments of this application can be a distributed system formed by connecting a client and multiple nodes (any form of computing device in the network, such as a server or user terminal) through network communication.

[0231] Taking a distributed system as an example, blockchain is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying platform, a platform product service layer, and an application service layer.

[0232] The underlying blockchain platform can include processing modules such as user management, basic services, smart contracts, and operational monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Furthermore, under authorization, it monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic services module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic services first perform interface adaptation parsing and authentication (interface adaptation), and then encrypt the business information through a consensus algorithm (consensus management). After encryption, the data is transmitted completely and consistently to the shared ledger (network communication) and recorded and stored. The smart contract module is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). According to the contract terms, the key or other events are invoked to trigger execution and complete the contract logic. It also provides functions for contract upgrades and cancellations. The operation monitoring module is mainly responsible for deployment, configuration modification, contract settings, cloud adaptation, and real-time status visualization output during product release, such as alarms, monitoring network conditions, and monitoring the health status of node devices.

[0233] The platform's product service layer provides the basic capabilities and implementation frameworks for typical applications. Developers can leverage these basic capabilities, along with the specific characteristics of their business needs, to implement blockchain-based business logic. The application service layer provides blockchain-based application services to business stakeholders.

[0234] The following are device embodiments of this application. For details not described in detail in the device embodiments, please refer to the corresponding descriptions in the above method embodiments. They will not be repeated here.

[0235] Figure 12 A schematic diagram of an automatic question-answering device provided in an exemplary embodiment of this application is shown. This device can be implemented as all or part of a computer device through software, hardware, or a combination of both. The device 1200 includes:

[0236] The acquisition module 1201 is used to acquire the input question sentence vector of the input question and K sets of question and answer sentence vector templates. Each set of question and answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1.

[0237] The fusion module 1202 is used to fuse the input question sentence vector and the question text sentence vector based on a first attention mechanism to obtain a comprehensive question category vector, and to fuse the input question sentence vector and the response text sentence vector based on a second attention mechanism to obtain a comprehensive response category vector.

[0238] The classification module 1203 is used to obtain a category probability based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector. The category probability is used to represent the probability that the input question belongs to the target question and answer sentence vector template in the K groups of question and answer sentence vector templates.

[0239] The classification module 1203 is further configured to determine the response text corresponding to the target sentence vector template as the output response to the input question based on the category probability.

[0240] In an optional design of this application, the fusion module 1202 is further configured to perform vector interaction between the input question sentence vector and the question text sentence vectors in the K sets of question-answer sentence vector templates to obtain K sets of question importance weights. The k1-th question importance weight in the K sets of question importance weights is used to represent the relevance between the input question and the question text corresponding to the k1-th question-answer sentence vector template, 1≤k1≤K, where k1 is a positive integer; K question attention weights are obtained based on the K sets of question importance weights; K question category vectors are obtained, the k2-th question category vector in the K sets of question category vectors is used to represent the mean of the question text sentence vectors in the k2-th question-answer sentence vector template, 1≤k2≤K, where k2 is a positive integer; and the K question attention weights and the K question category vectors are weighted and combined to calculate the comprehensive question category vector.

[0241] In an optional design of this application, the fusion module 1202 is further configured to take the first a values ​​of the largest problem importance weight in each of the K groups of problem importance weights, combine them to obtain a problem importance weight sequence, the problem importance weight sequence including a*K problem importance weights; and extract the K problem attention weights from the problem importance weight sequence.

[0242] In an optional design of this application, the fusion module 1202 is further configured to perform vector interaction between the input question sentence vector and the response text sentence vectors in the K sets of question-and-answer sentence vector templates to obtain K sets of response importance weights. The k3rd set of response importance weights in the K sets of response importance weights is used to represent the relevance between the input question and the response text corresponding to the k3rd set of question-and-answer sentence vector templates, 1≤k3≤K, where k3 is a positive integer; based on the K sets of response importance weights, K response attention weights are obtained; K response category vectors are obtained, the k4th set of response category vectors in the K sets of response category vectors is used to represent the mean of the response text sentence vectors in the k4th set of question-and-answer sentence vector templates, 1≤k4≤K, where k4 is a positive integer; the K response attention weights and the K response category vectors are weighted and combined to calculate the comprehensive response category vector.

[0243] In an optional design of this application, the fusion module 1202 is further configured to take the top b values ​​of the largest response importance weight in each of the K groups of response importance weights, combine them to obtain a response importance weight sequence, the response importance weight sequence including b*K response importance weights; and extract the K response attention weights from the response importance weight sequence.

[0244] In an optional design of this application, the acquisition module 1201 is further configured to invoke a first attention computing network to obtain the comprehensive question category vector based on the first attention mechanism and according to the input question sentence vector and the question text sentence vector; and invoke a second attention computing network to obtain the comprehensive response category vector based on the second attention mechanism and according to the input question sentence vector and the response text sentence vector.

[0245] In an optional design of this application, the classification module 1203 is further configured to concatenate the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector to obtain a concatenated vector; and call the classification network to convert the concatenated vector into the category probability.

[0246] In an optional design of this application, the acquisition module 1201 is further configured to invoke a sentence vector generation network to convert the input question into the input question sentence vector.

[0247] In an optional design of this application, the classification module 1203 is further configured to determine the sentence vector template corresponding to the highest category probability among the category probabilities as the target sentence vector template; and to use the response text corresponding to the target sentence vector template as the output response to the input question.

[0248] In an optional design of this application, the classification module 1203 is further configured to extract keywords from the input question; determine n key information corresponding to the keywords from the knowledge graph, and at least two candidate options corresponding to each key information, where n is a positive integer; and, in response to obtaining a target option from the at least two candidate options corresponding to each key information, extract the output response from the knowledge graph according to the target option.

[0249] In an optional design of this application, the classification module 1203 is further configured to determine the entities to be graphed and the relationships between entities to be graphed in the information to be graphed; obtain the knowledge graph to be fused based on the entities to be graphed and the relationships between entities to be graphed; align the knowledge graph to be fused with the existing knowledge graph to obtain a first structured knowledge graph and a second structured knowledge graph; and fuse the first structured knowledge graph and the second structured knowledge graph to obtain the knowledge graph.

[0250] In an optional design of this application, the classification module 1203 is further configured to display the input question and the first option corresponding to the input question; in response to a selection operation on the target option in the first option, determine the target entity from the knowledge graph according to the target option; in response to the target entity belonging to the response entity, display the output response corresponding to the target entity, wherein the response entity is the entity corresponding to the output response.

[0251] In summary, this embodiment obtains a comprehensive question category vector by fusing the input question sentence vector and the question text sentence vector, and a comprehensive response category vector by fusing the input question sentence vector and the response text sentence vector. Then, based on the comprehensive question category vector, the comprehensive question category vector, and the input question sentence vector, a category probability is obtained, and the output response corresponding to the input question is determined by the category probability. Because this method determines the relationship between the input question and the question text, as well as the relationship between the input question and the response text, it fuses matching results from two different perspectives, thus improving the accuracy of automatic question answering.

[0252] Figure 13 A schematic diagram of a training apparatus for an automatic question-answering model provided in an exemplary embodiment of this application is shown. This apparatus can be implemented as all or part of a computer device through software, hardware, or a combination of both. The apparatus 1300 includes:

[0253] The sample acquisition module 1301 is used to acquire a training dataset, which includes a sample input question and the ground truth annotations corresponding to the sample input question.

[0254] The sample input module 1302 is used to obtain the sample input question sentence vector of the sample input question;

[0255] The sample fusion module 1303 is used to obtain a comprehensive category vector of sample questions based on the first attention mechanism and the sample input question sentence vector and the question text sentence vector, and to obtain a comprehensive category vector of sample responses based on the second attention mechanism and the sample input question sentence vector and the response text sentence vector.

[0256] The sample classification module 1304 is used to obtain the sample category probability based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample answer comprehensive category vector. The sample category probability is used to represent the probability that the sample input question belongs to the sample target sentence vector template in the K groups of question and answer sentence vector templates.

[0257] The sample classification module 1304 is further configured to determine the response text corresponding to the sample target sentence vector template as the sample output response to the sample input question based on the sample category probability.

[0258] The sample training module 1305 is used to train the automatic question answering model based on the sample output responses and the real annotations.

[0259] In summary, this embodiment provides a training method for an automatic question answering model. This method can train the automatic question answering model relatively quickly. The trained automatic question answering model can determine the relationship between the input question and the question text, as well as the relationship between the input question and the answer text, and integrate the matching results from two different perspectives to improve the accuracy of automatic question answering.

[0260] Figure 14 This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device 1400 includes a Central Processing Unit (CPU) 1401, a system memory 1404 including Random Access Memory (RAM) 1402 and Read-Only Memory (ROM) 1403, and a system bus 1405 connecting the system memory 1404 and the CPU 1401. The computer device 1400 also includes a basic input / output system (I / O system) 1406 to facilitate information transfer between various devices within the computer device, and a mass storage device 1407 for storing an operating system 1413, application programs 1414, and other program modules 1415.

[0261] The basic input / output system 1406 includes a display 1408 for displaying information and an input device 1409 for user input, such as a mouse or keyboard. Both the display 1408 and the input device 1409 are connected to the central processing unit 1401 via an input / output controller 1410 connected to the system bus 1405. The basic input / output system 1406 may also include the input / output controller 1410 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1410 also provides output to a display screen, printer, or other types of output devices.

[0262] The mass storage device 1407 is connected to the central processing unit 1401 via a mass storage controller (not shown) connected to the system bus 1405. The mass storage device 1407 and its associated computer device-readable media provide non-volatile storage for the computer device 1400. That is, the mass storage device 1407 may include computer device-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0263] Without loss of generality, the computer device readable medium may include computer device storage media and communication media. Computer device storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer device readable instructions, data structures, program modules, or other data. Computer device storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, digital video disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer device storage media are not limited to the above-mentioned types. The system memory 1404 and mass storage device 1407 described above can be collectively referred to as memory.

[0264] According to various embodiments of this disclosure, the computer device 1400 can also be connected to a remote computer device on a network, such as the Internet. That is, the computer device 1400 can be connected to the network 1411 via a network interface unit 1412 connected to the system bus 1405, or the network interface unit 1412 can be used to connect to other types of networks or remote computer device systems (not shown).

[0265] The memory also includes one or more programs stored in the memory. The central processing unit 1401 executes the one or more programs to implement all or part of the steps of the above-mentioned automatic question answering method or the training method of the automatic question answering model.

[0266] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the automatic question-answering method provided in the above-described method embodiments, or to implement the training method for the automatic question-answering model as described above.

[0267] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the automatic question answering method provided in the above method embodiments, or to implement the training method for the automatic question answering model as described above.

[0268] Optionally, this application also provides a computer program product containing instructions that, when run on a computer device, causes the computer device to execute the automatic question-answering method described in the foregoing aspects, or to implement the training method for the automatic question-answering model as described in the foregoing aspects.

[0269] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0270] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0271] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An automatic question-answering method, characterized in that, The method includes: Obtain the input question sentence vector and K sets of question-and-answer sentence vector templates. Each set of question-and-answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1. Based on the first attention mechanism, the input question sentence vector and the question text sentence vector are fused to obtain a comprehensive question category vector; and based on the second attention mechanism, the input question sentence vector and the response text sentence vector are fused to obtain a comprehensive response category vector. Based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector, a category probability is obtained. The category probability is used to represent the probability that the input question belongs to the target question and answer sentence vector template in the K groups of question and answer sentence vector templates. Based on the category probability, the response text corresponding to the target question-and-answer vector template is determined as the output response to the input question.

2. The method according to claim 1, characterized in that, The first attention mechanism, based on the input question sentence vector and the question text sentence vector, yields a comprehensive question category vector, including: The input question sentence vector is interacted with the question text sentence vectors in the K sets of question-answer sentence vector templates to obtain K sets of question importance weights; the k1-th question importance weight in the K sets of question importance weights is used to represent the relevance between the input question and the question text corresponding to the k1-th question-answer sentence vector template, 1≤k1≤K, where k1 is a positive integer; Based on the K sets of question importance weights, K question attention weights are obtained; Obtain K question category vectors. The k2th question category vector in the K question category vectors is used to represent the mean of the question text sentence vectors in the k2th question and answer sentence vector template, 1≤k2≤K, and k2 is a positive integer; The K question attention weights and the K question category vectors are weighted and combined to calculate the comprehensive question category vector.

3. The method according to claim 2, characterized in that, The process of obtaining K problem attention weights based on the K sets of problem importance weights includes: Take the top a values ​​of each of the K groups of question importance weights, sorted from largest to smallest, and combine them to obtain a question importance weight sequence. The question importance weight sequence includes a... K importance weights for each question, where a is a positive integer; Extract the K problem attention weights from the problem importance weight sequence.

4. The method according to claim 1, characterized in that, The second attention mechanism fuses the input question sentence vector and the response text sentence vector to obtain a comprehensive response category vector, including: The input question sentence vector is interacted with the response text sentence vectors in the K sets of question-and-answer sentence vector templates to obtain K sets of response importance weights. The k3th response importance weight in the K sets of response importance weights is used to represent the relevance between the input question and the response text corresponding to the k3th question-and-answer sentence vector template, where 1≤k3≤K and k3 is a positive integer. Based on the K sets of response importance weights, K response attention weights are obtained; Obtain K response category vectors. The k4th group of response category vectors in the K response category vectors is used to represent the mean of the response text sentence vectors in the k4th group of question and answer sentence vector templates, where 1≤k4≤K and k4 is a positive integer. The comprehensive category vector of the responses is calculated by weighting and combining the attention weights of the K responses and the category vectors of the K responses.

5. The method according to claim 4, characterized in that, The process of obtaining K response attention weights based on the K sets of response importance weights includes: Take the top b values ​​of each of the K groups of response importance weights, sorted from largest to smallest, and combine them to obtain a response importance weight sequence. The response importance weight sequence includes b... K importance weights for each response, where b is a positive integer; Extract the K response attention weights from the response importance weight sequence.

6. The method according to any one of claims 1 to 5, characterized in that, The first attention mechanism, based on the input question sentence vector and the question text sentence vector, yields a comprehensive question category vector, including: The first attention computing network is invoked, and based on the first attention mechanism, the comprehensive question category vector is obtained according to the input question sentence vector and the question text sentence vector; The second attention mechanism, based on the input question sentence vector and the response text sentence vector, yields a comprehensive response category vector, including: The second attention computation network is invoked, and based on the second attention mechanism, the comprehensive category vector of the response is obtained according to the input question sentence vector and the response text sentence vector.

7. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the category probability based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector includes: The input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector are concatenated to obtain the concatenated vector; The classification network is invoked to convert the concatenated vector into the class probabilities.

8. The method according to any one of claims 1 to 5, characterized in that, The process of obtaining the input question sentence vector includes: The sentence vector generation network is invoked to convert the input question into a sentence vector.

9. The method according to any one of claims 1 to 5, characterized in that, The step of determining the response text corresponding to the target question-answer vector template as the output response to the input question based on the category probability includes: The sentence vector template corresponding to the highest category probability among the category probabilities is determined as the target question-and-answer sentence vector template; The response text corresponding to the target question-and-answer vector template is used as the output response to the input question.

10. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Extract keywords from the input question; From the knowledge graph, determine n key information corresponding to the keyword, and at least two candidate options corresponding to each key information, where n is a positive integer; In response to obtaining a target option from the at least two candidate options corresponding to each key information, the output response is extracted from the knowledge graph based on the target option; or, in response to the absence of the output response in the knowledge graph, the automatic question answering method according to any one of claims 1 to 5 is executed.

11. The method according to claim 10, characterized in that, The method further includes: Determine the entities to be mapped and their relationships within the information to be mapped; Based on the entities to be graphed and their relationships, a knowledge graph to be fused is obtained. Align the knowledge graph to be fused with the existing knowledge graph to obtain a first structured knowledge graph and a second structured knowledge graph; The knowledge graph is obtained by fusing the first structured knowledge graph and the second structured knowledge graph.

12. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Display the input question and the first option corresponding to the input question; In response to the selection operation on the target option in the first option, the target entity is determined from the knowledge graph based on the target option; In response to the target entity belonging to the responding entity, the output response corresponding to the target entity is displayed, wherein the responding entity is the entity corresponding to the output response; or, in response to the target entity not existing in the knowledge graph, the automatic question answering method according to any one of claims 1 to 5 is executed.

13. A training method for an automatic question-answering model, characterized in that, The method includes: Obtain a training dataset, which includes sample input questions and the corresponding ground truth labels for the sample input questions; Obtain the sample input question sentence vector; Based on the first attention mechanism, a comprehensive category vector of sample questions is obtained according to the sample input question sentence vector and the question text sentence vector; and based on the second attention mechanism, a comprehensive category vector of sample responses is obtained according to the sample input question sentence vector and the response text sentence vector. Based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample response comprehensive category vector, the sample category probability is obtained. The sample category probability is used to represent the probability that the sample input question belongs to the sample target sentence vector template in K groups of question and answer sentence vector templates, where K is an integer greater than 1. Based on the sample category probability, the response text corresponding to the sample target sentence vector template is determined as the sample output response to the sample input question; The automatic question answering model is trained based on the sample output responses and the real annotations.

14. An automatic question-and-answer device, characterized in that, The device includes: The acquisition module is used to acquire the input question sentence vector and K sets of question and answer sentence vector templates. Each set of question and answer sentence vector templates includes a pair of question text sentence vectors and answer text sentence vectors, where K is an integer greater than 1. The fusion module is used to fuse the input question sentence vector and the question text sentence vector based on a first attention mechanism to obtain a comprehensive question category vector, and to fuse the input question sentence vector and the response text sentence vector based on a second attention mechanism to obtain a comprehensive response category vector. The classification module is used to obtain the category probability based on the input question sentence vector, the question comprehensive category vector, and the response comprehensive category vector. The category probability is used to represent the probability that the input question belongs to the target question answer sentence vector template in the K groups of question answer sentence vector templates. The classification module is further configured to determine the response text corresponding to the target question-answer vector template as the output response to the input question based on the category probability.

15. A training device for an automatic question-answering model, characterized in that, The device includes: The sample acquisition module is used to acquire a training dataset, which includes sample input questions and the ground truth labels corresponding to the sample input questions. The sample input module is used to obtain the sample input question sentence vector of the sample input question; The sample fusion module is used to obtain a comprehensive category vector of sample questions based on the first attention mechanism and the sample input question sentence vector and the question text sentence vector, and to obtain a comprehensive category vector of sample responses based on the second attention mechanism and the sample input question sentence vector and the response text sentence vector. The sample classification module is used to obtain the sample category probability based on the sample input question sentence vector, the sample question comprehensive category vector, and the sample response comprehensive category vector. The sample category probability is used to represent the probability that the sample input question belongs to the sample target sentence vector template in K groups of question and answer sentence vector templates, where K is an integer greater than 1. The sample classification module is further configured to determine the response text corresponding to the sample target sentence vector template as the sample output response to the sample input question based on the sample category probability. The sample training module is used to train the automatic question answering model based on the sample output responses and the real annotations.

16. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the automatic question answering method as described in any one of claims 1 to 12, or to implement the training method for the automatic question answering model as described in claim 13.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the automatic question answering method as described in any one of claims 1 to 12, or to implement the training method for the automatic question answering model as described in claim 13.

Images