Text interaction multiple-choice intelligent answering method and system fusing three-dimensional attention

Abstract

The application provides a text interaction multiple-choice intelligent answering method and system fusing three-dimensional attention, data set collection, including collecting question data on the Internet, forming a multiple-choice data set after collation and filtering; option difference interaction, including simulating human beings in solving reading comprehension multiple-choice questions in the text interaction multiple-choice answering model fusing three-dimensional attention, seeking help by comparing all options, modeling the difference information between options; text interaction between questions, options and articles, including simulating the need of human beings to understand the connection between different types of text segments for reading reasoning in the reading comprehension process in the text interaction multiple-choice answering model fusing three-dimensional attention, and using three-dimensional attention to supplement the missing context information when the question and option interact; model fine-tuning, including using the data set, inputting into the text interaction multiple-choice answering model fusing three-dimensional attention for pre-training, and realizing intelligent answering.

Description

Technical Field

[0001] This invention belongs to the field of natural language intelligent processing, and more specifically, relates to a text-based interactive intelligent multiple-choice question-answering scheme based on machine reading comprehension and three-dimensional attention. Background Technology

[0002] In recent years, with the development of neural networks, teaching machines to read and understand texts has become one of the core tasks in NLP. In 2018, the Ministry of Science and Technology of China established the "Key Technologies and Systems for Human-like Intelligence Based on Big Data" project, aiming to develop intelligent robots capable of understanding the meaning of articles and accurately answering corresponding questions. Multiple-choice questions, as a basic reading comprehension question type, have become an important research direction in machine reading comprehension tasks under the impetus of this project. Since 2018, BERT and its modified versions have shown remarkable improvements in various NLP tasks, providing new ideas for completing machine reading comprehension multiple-choice tasks.

[0003] In reading comprehension multiple-choice tasks, the focus is on modeling the differences between different options and the pairwise relationships between questions, options, and the passage. The textual interactions between options simulate how humans might seek help by comparing all options when solving multiple-choice questions. For example, choosing an option isn't because it's the most likely correct answer, but because all other options are impossible. Inspired by this human experience, comparative information between answer options is introduced, making each option non-independent. A bilinear representation is established between any two options, and a gating mechanism is used to fuse the interaction representation into the original answer option representation, allowing for a clearer selection of the correct answer. The textual interactions between questions, options, and the passage simulate the human reading comprehension process of understanding the connections between different types of text passages for reading reasoning. Previous work introduced a dual collaborative matching network to bidirectionally simulate the relationship representation between questions, options, and the passage, and then used a gating mechanism to fuse them from both directions, achieving the effect of considering the influence of other parts from the perspectives of questions, options, and the passage. Different textual interaction mechanisms produce different effects in machine reading comprehension multiple-choice tasks, but essentially they use different attention mechanisms. When modeling the relationship between questions and options, ordinary two-dimensional attention mechanisms only treat the question and option as two dimensions of interaction, failing to explicitly simulate the interaction between the question and option under the influence of contextual information. This results in the loss of important contextual information and poor attention performance. Therefore, a three-dimensional attention mechanism is needed to extend the standard two-dimensional attention mechanism. By explicitly incorporating context as a third dimension in calculating the relevance score, the interaction between the question and option is obtained, resulting in an interaction representation of the question and option with integrated context, which can be used for multiple-choice reasoning.

[0004] The statistics of multiple choice models that have emerged in recent years, based on text interaction methods, attention dimensions, etc., are shown in Table 1.

[0005] Table 1 Comparison of Multiple Choice Models

[0006] In machine reading comprehension multiple-choice tasks, each data point includes an article, at least one question, and four candidate options for each question. There is only one correct answer, and the model needs to understand the meaning of the article and select the correct option. Therefore, this task is modeled as inputting a question, options, and text, and then classifying the input text into four categories.

[0007] Patents such as CN109492227A ("A Machine Reading Comprehension Method Based on Multi-Head Attention Mechanism and Dynamic Iteration"), CN110083682B ("A Machine Reading Comprehension Answer Acquisition Method Based on Multi-Turn Attention Mechanism"), CN110334184A ("Intelligent Question Answering System Based on Machine Reading Comprehension"), and CN115659947A ("Multiple-Choice Question Answering Method and System Based on Machine Reading Comprehension and Text Summarization") typically only calculate the question-aware article representation when using attention mechanisms to model the relationship between articles and questions, while ignoring the article-aware question representation. This results in an inability to effectively capture the relationship between articles and questions. Summary of the Invention

[0008] Existing reading comprehension multiple-choice models mostly use two-dimensional attention mechanisms to model the differences between options and the pairwise relationships between questions, options, and the text. These methods simulate some details of the human reading comprehension process, improving machine performance in reading comprehension multiple-choice tasks. However, the ordinary two-dimensional attention mechanism framework only treats the question and options as two dimensions of attention interaction, which leads to the neglect of the influence of context when the question and options interact. Especially when the question involves understanding sentences and words in the text, the lack of contextual information can make the relationship between the question and the options unclear. Therefore, this invention proposes to use a general three-dimensional attention mechanism, incorporating context as the third dimension for calculating relevance scores. This approach is more suitable for this scenario and can improve the accuracy of multiple-choice models.

[0009] To achieve the above objectives, this invention proposes a text-based interactive multiple-choice intelligent question-answering method that integrates three-dimensional attention, comprising the following steps: Step 1, Data Collection, includes collecting question data from the Internet. Each data point contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple-choice dataset is formed. Step 2, option difference interaction, includes simulating human behavior when solving multiple-choice reading comprehension questions by comparing all options in a text interaction multiple-choice answering model that integrates 3D attention, and modeling the difference information between options; Step 3 involves textual interaction between questions, options, and text. This includes simulating the human reading comprehension process in a multi-choice question model that integrates three-dimensional attention, where the human needs to understand the connections between different types of text passages and make reading inferences. Three-dimensional attention is also used to supplement the missing contextual information when questions and options interact. Step 4, model fine-tuning, includes using the dataset from Step 1 to pre-train a text interaction multiple-choice answering model that incorporates 3D attention, and using the results to support intelligent answering.

[0010] Furthermore, when collecting the dataset, web crawlers are used to collect raw HTML text from relevant websites, extract articles, questions, options, and answers, and remove redundant JavaScript code, CSS code, and comment text; then the structure of the web page text after removing the code is parsed, and HTML tags contained in the web page text are removed; finally, some symbols in the text are converted.

[0011] Furthermore, the model is based on BERT. It first reads the questions, options, and articles from the dataset in batches, and then encodes the question, option, and article text using BERT to obtain the question part vector. Vectors in the options section Vectors related to the article section , , , For text length, The dimension of the hidden layer.

[0012] Furthermore, in order to integrate the differences between the various options, After bilinear interaction, a gating mechanism is used to integrate the interaction representation into the original answer option representation.

[0013] Furthermore, after incorporating the difference information in the options, a vector encoding the question is used to model the relationship between the question, options, and the article. Vectors after option difference interaction The encoded vector of the article Bidirectional matching is performed, and the representation is fused from both directions using a gating mechanism. Therefore, multiple three-dimensional attention layers are used for interaction during the interaction between questions and options.

[0014] Moreover, the text interaction multiple-choice intelligent answering model that integrates three-dimensional attention reads questions, options, articles and standard answers from the dataset in batches. After reading and understanding the articles, questions and options, it makes predictions, calculates the model loss based on the predicted values ​​and the standard answers, and then trains the model based on the loss.

[0015] On the other hand, the present invention also provides a text-based interactive multiple-choice intelligent question-answering system that integrates three-dimensional attention, for implementing the text-based interactive multiple-choice intelligent question-answering method that integrates three-dimensional attention as described above.

[0016] Moreover, it includes the following modules, The first module is used for dataset collection, which includes collecting question data from the Internet. Each data entry contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple-choice dataset is formed. The second module is used for option difference interaction, including simulating human behavior when solving multiple-choice reading comprehension questions by comparing all options and modeling the difference information between options in a text interaction multiple-choice answering model that integrates three-dimensional attention. The third module is used for text interaction between questions, options, and articles. It includes simulating the human reading comprehension process in a text interaction multiple-choice answering model that integrates three-dimensional attention, which requires understanding the connections between different types of text passages and making reading inferences. It also uses three-dimensional attention to supplement the missing contextual information when questions and options interact. The fourth module is used for model fine-tuning, including using a dataset as input to pre-train a text-based interactive multiple-choice question-answering model that incorporates 3D attention, and supporting intelligent question answering based on the results.

[0017] Alternatively, it may include a processor and a memory, with the memory used to store program instructions and the processor used to call the stored instructions in the memory to execute a text-based interactive multiple-choice intelligent question-answering method that incorporates three-dimensional attention, as described above.

[0018] Alternatively, it may include a readable storage medium storing a computer program that, when executed, implements a text-based interactive multiple-choice intelligent question-answering method that integrates three-dimensional attention, as described above.

[0019] Compared to various two-dimensional attention-based text interaction reading comprehension multiple-choice methods, this invention delves deeper into the contextual information that needs attention when questions and options interact, making the interaction between questions, options, and text more comprehensive and the multiple-choice reasoning more accurate.

[0020] This patent employs a dual collaborative matching network to simulate the bidirectional relationship between questions, options, and the text within the model. Considering real-world scenarios involving multiple-choice reading comprehension tasks, some data samples involve understanding sentences or words within the text. Given that the bidirectional interaction between questions and options doesn't involve contextual information, the perceptual representation of questions and options can be incomplete. To compensate for this missing contextual information in the aforementioned application scenarios, this patent uses a three-dimensional attention mechanism to treat contextual information as a third dimension beyond the query and key. Context, query, and key directly interact through attention, making it more suitable for the task scenario described in this patent.

[0021] The present invention is simple and convenient to implement, highly practical, and solves the problems of low practicality and inconvenience in actual application of related technologies. It can improve user experience and has significant market value. Attached Figure Description

[0022] Figure 1 This is a flowchart of an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of the model in an embodiment of the present invention; Figure 3 This is a flowchart illustrating the two-dimensional attention mechanism according to an embodiment of the present invention. Figure 4 This is a flowchart of the three-dimensional attention mechanism in an embodiment of the present invention. Detailed Implementation

[0023] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0024] This invention discloses a text-based interactive multiple-choice question-answering method incorporating three-dimensional attention, drawing on detailed experiences of humans in reading comprehension multiple-choice questions. First, it introduces comparative information between answer options, ensuring each option is not independent. A bilinear representation is established between any two options, and a gating mechanism is used to fuse the interaction representation into the original answer option representation, resulting in clearer selection of the correct answer. Then, a dual-cooperative matching network is introduced to bidirectionally simulate the relationship between the question, options, and the text. A gating mechanism is then used to fuse these elements from both directions, achieving the effect of considering the influence of other parts from the perspectives of the question, options, and text. However, the dual-cooperative matching network uses a two-dimensional attention mechanism when interacting with the question and options, without considering the influence of context. This invention uses a three-dimensional attention mechanism to explicitly incorporate context as a third dimension into the interaction between the question and options, making the interaction more comprehensive, obtaining richer information, and improving the accuracy of answering questions.

[0025] First, the technical terminology of this invention will be explained and described: BERT: BERT stands for Bidirectional Encoder Representation from Transformers, a pre-trained language representation model. It emphasizes that instead of using traditional unidirectional language models or shallow concatenation of two unidirectional language models for pre-training, it adopts a new Masked Language Model (MLM) to generate deep bidirectional language representations.

[0026] Python: A computer programming language.

[0027] PyQuery: A library similar to jQuery (a JavaScript library) that uses lxml for fast XML and HTML manipulation.

[0028] HTML: HyperText Markup Language is a standard markup language used to create web pages.

[0029] JavaScript: A lightweight, interpreted or just-in-time (JIT) programming language with a function-first approach.

[0030] CSS: Cascading Style Sheets. CSS describes how HTML elements are displayed on screens, paper, or other media. CSS saves a lot of work.

[0031] Transformer: A novel architecture designed to solve sequence-to-sequence tasks while easily handling long-range dependencies.

[0032] Sigmoid: The sigmoid function is one of the most widely used activation functions. It has an exponential function shape and is physically closest to a biological neuron. It is a common sigmoid function in biology, also known as an sigmoid growth curve. Furthermore, the output of "(0,1)" can also be represented as a probability.

[0033] See Figure 1 This invention provides a text-based interactive multiple-choice question-answering method that integrates three-dimensional attention, comprising the following steps: Step 1: First, construct a machine reading comprehension multiple choice dataset, i.e., dataset collection, which includes collecting question data from the Internet. Each data entry contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple choice dataset is formed.

[0034] The example uses college entrance examination questions as examples to illustrate the implementation method of the preferred suggestion in step 1 as follows: Step 1.1: Use web scraping technology to collect college entrance examination simulation questions and past exam papers from the internet, and structure the HTML data to extract the article, question, options, and answer. This can be achieved by extracting fields such as "article," "question," "option," and "answer" from the raw HTML text collected from college entrance examination simulation question and past exam paper question bank websites. This part will contain redundant content such as JavaScript code, CSS code, and comment text. Then, the obtained web page text is processed to remove the code, that is, to remove JavaScript code, CSS code, and comment text. Then, the Python web page parsing library PyQuery is used to parse the structure of the web page text after the code is removed, and the HTML tags contained in the web page text are removed. Finally, some symbols in the text are converted.

[0035] Step 1.2: The content of the article, questions, options, and answers should be as concise and refined as possible, avoiding redundancy; Step 1.3: For examples in the question text that involve multiple selections, paragraph information, or explanations of underlined words, select "Discard". For examples in the option text that involve explanations of underlined words, select "Supplementary Information".

[0036] In practice, questions like "Which two of the following are consistent with the meaning of the article?", "Which of the following is the correct explanation of the underlined word in the text?", and "Which of the following is the correct understanding of the second paragraph in the text?" should be discarded. For questions like "Which of the following is the correct understanding of the underlined word in the text?", since plain text cannot directly display the "underlined" information, the original question needs to be modified to "Which of the following is the correct understanding of the underlined word in the text? Underlined word: handwritten curse." For questions like "Which of the following explanations of the underlined words in the sentence is incorrect? A. Father is gone, son is still fearful and dignified. 'Dignified and dignified' means a generous and magnanimous manner.", due to the handling of escape characters and special whitespace, the original spacing has been removed, and it should be changed to "A. Father is gone, son is still fearful and dignified. 'Dignified and dignified' means a generous and magnanimous manner." Table 2 shows a sample dataset.

[0037] Table 2. Examples of Long Text Multiple Choice Datasets

[0038] Step 2: Option Difference Interaction. This step simulates how people seek help by comparing all options when solving multiple-choice reading comprehension questions, modeling the differences between the options.

[0039] The MRC model (text-based interactive multiple-choice question-answering model with 3D attention integration) provided by this invention is based on BERT. It first reads the questions, options, and articles from the dataset in batches, and then processes the question, option, and article texts using BERT (…). Encoding yields the problem part vector. Vectors in the options section Vectors of the article section and, , , The length of the text (in characters). Let be the dimension of the hidden layer, where Represents the real number field.

[0040] In order to integrate the differences between the different options After bilinear interaction, a gating mechanism is used to integrate the interaction representation into the original answer option representation.

[0041] Step 2 of the embodiment is preferably implemented in the following way: Step 2.1, Input the sample into... Figure 2 Following the MRC model shown, the system first simulates how people seek help by comparing all options when solving multiple-choice reading comprehension questions, modeling the differences between the options. To obtain separate representations of the question, options, and text after encoding, the "[SEP]" symbol is used to separate the three types of text during the concatenation process. The four texts will then be processed by BERT (…). After that, we obtain the representation of each text. Each text representation is further divided into three parts, the vector of the problem part. Vectors in the options section Vectors related to the article section , , , The length of the text (in characters). The dimension of the hidden layer is 768 in this paper. The encoder will output each answer option. Encoded as Then the options and The comparison vector can be calculated as follows:

[0042]

[0043] in It is a learnable parameter. yes and The bilinear interaction matrix between them This represents the difference in interaction.

[0044] Step 2.2: Use a gating mechanism to integrate the differential interaction representation into the original answer option representation. The specific formula is shown below:

[0045]

[0046]

[0047]

[0048] In this embodiment and These are learnable parameters. It is a concatenation of all interactive representations, in this embodiment The value is 4. It is a reset gate that balances intermediate variables. and The impact, It is encoded using interactive information. The final option is indicated. (Final option is indicated.) As the final set of answer options, it incorporates comparative information from different answer options.

[0049] Step 3: Text interaction between questions, options, and articles.

[0050] After incorporating the discrepancies in the options, and drawing upon real-life experience, it's necessary to connect the questions, options, and text for effective reading. To model the relationships between these elements, this method encodes the vector containing the questions. Vectors after option difference interaction The encoded vector of the article Bidirectional matching is performed, and a gating mechanism is used to fuse the representations from both directions. Since direct interaction between questions and options does not take into account contextual information, it can lead to insufficient interaction and information loss. Therefore, a multi-layered three-dimensional attention layer is used for interaction between questions and options.

[0051] Step 3 of the embodiment is implemented as follows: Step 3.1, in Figure 2 In the MRC model, after modeling the option difference information, the model simulates the human reading comprehension process, which requires understanding the connections between different types of text passages for reading reasoning. It also uses three-dimensional attention to supplement the missing contextual information during the interaction between questions and options. After the options incorporate difference information, and based on real-life experience, it's necessary to connect the questions, options, and text for reading. To model the relationships between questions, options, and text, the vector encoded from the questions is... Vectors after option difference interaction The encoded vector of the article Bidirectional matching is performed, and a gating mechanism is used to fuse the representations from both directions. Since the interaction between questions and options in the two-dimensional attention text interaction method does not consider contextual information, it can lead to insufficient interaction and information loss. Therefore, a multi-layered three-dimensional attention layer is used for the interaction between questions and options. The flowchart of the two-dimensional attention mechanism in this embodiment is as follows: Figure 3 The flowchart of the three-dimensional attention mechanism is as follows: Figure 4 The standard two-dimensional attention mechanism first calculates the similarity F(query, key) between the query and the key, then obtains a relevance score. This is followed by softmax (normalization) to obtain attention weights, and finally, a weighted sum of the weights and values ​​is performed to obtain the attention embedding. In three-dimensional attention, the mechanism first extends the similarity calculation by incorporating contextual information, resulting in a context-dependent query-key relevance score, i.e., the context relevance score. Next, the context relevance score is normalized, and the normalized context and value are integrated to produce a context-dependent value, i.e., the context value. Ultimately, the context value and the context relevance score reside in the same semantic space; both capture additional contextual information, resulting in a richer and semantically consistent attention embedding representation.

[0052] Before entering the attention layer, to avoid the attention process consuming too much video memory (GPU memory) resources, the problem vector... Option vector Article Vector It will first go through the average pooling layer ( Lower the parameters, the formula is as follows:

[0053] In this embodiment , , , and of It is 60. of It is 256.

[0054] Step 3.2: The pooled question and option vectors interact using a three-dimensional attention mechanism. The overall framework of this mechanism is similar to that of the multi-head attention mechanism, and the specific formula is as follows:

[0055]

[0056]

[0057]

[0058] In this embodiment, The number is 12, each The focus of attention is different, and the attention received is indicated. There are also some differences. , , First, compare with the corresponding learnable weights. , , , Perform a dot product operation to obtain the query vector. key vector value vector Context vector ,in Then During operation, and will with The Hadama product operation incorporates contextual information, where Representative at Add a dimension before the 0th dimension of the original vector. Representative at Add a dimension to the beginning of the original vector's first dimension. Finally, calculate the current... Attention embedding vector, multiple The attention vectors are concatenated to form the final question-aware option representation. ,exchange and Similarly, the option perception problem representation is obtained by following the sequence. .

[0059] Step 3.3: The interaction between the question and the article, and between the option and the article, uses a common two-way matching strategy. Taking the interaction between the option and the article as an example, it is shown in formulas (12) to (15):

[0060]

[0061]

[0062]

[0063] In this embodiment , , , These are learnable parameters. and It is a weight matrix between questions and options. , These represent option-aware article representation and article-aware option representation, respectively.

[0064] Step 3.4: The final representation of the question-and-answer pair is calculated as follows:

[0065]

[0066]

[0067]

[0068] In this embodiment , , These are three learnable parameters. After performing max pooling operations step by step, it will reach... and Aggregate representation. It is a reset gate. This represents the sigmoid function. This is the final bidirectional matching representation of the option and passage sequence pairs. (The final bidirectional matching representation of the question and option sequence pairs is also described.) The final bidirectional matching representation of question and article sequence pairs Calculate using the same formulas (16) to (19).

[0069] Step 3.5, loss calculation is as follows: Matching representation constructed using three sequence pairs , , These are concatenated to form the final representation of each question-option-article triple. Z will calculate the probability distribution of the four options A, B, C, and D. Represents each Triplets. If If this is the correct option, the loss function can be calculated as follows ( The base is ):

[0070]

[0071] In this embodiment These are learnable parameters.

[0072] Step 4: Fine-tune the model and use the results to support intelligent question answering.

[0073] The question, options, and short text are concatenated and input into the MRC model established in this invention for training. The answer to the current question is the training label. The cross-entropy loss is calculated between the MRC predicted classification value and the label, as shown in the following formula:

[0074]

[0075]

[0076] The training samples are divided into mixed sequences of relevant blocks and continuous unrelated blocks, and mixed sequences of relevant blocks and random unrelated blocks. The correct answer to this question is [answer name]. This represents the probability distribution of the options inferred by the inference engine in this instance. This is the calculated inference loss.

[0077] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0078] In some possible embodiments, a text-based interactive multiple-choice intelligent question-answering system incorporating three-dimensional attention is provided, including the following modules: The first module is used for dataset collection, which includes collecting question data from the Internet. Each data entry contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple-choice dataset is formed. The second module is used for option difference interaction, including simulating human behavior when solving multiple-choice reading comprehension questions by comparing all options and modeling the difference information between options in a text interaction multiple-choice answering model that integrates three-dimensional attention. The third module is used for text interaction between questions, options, and articles. It includes simulating the human reading comprehension process in a text interaction multiple-choice answering model that integrates three-dimensional attention, which requires understanding the connections between different types of text passages and making reading inferences. It also uses three-dimensional attention to supplement the missing contextual information when questions and options interact. The fourth module is used for model fine-tuning, including using a dataset as input to pre-train a text-based interactive multiple-choice question-answering model that incorporates 3D attention, and using the results to support intelligent question answering. In some possible embodiments, a text-based interactive multiple-choice intelligent question-answering system incorporating three-dimensional attention is provided, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a text-based interactive multiple-choice intelligent question-answering method incorporating three-dimensional attention as described above.

[0079] In some possible embodiments, a text-based interactive multiple-choice intelligent question-answering system incorporating three-dimensional attention is provided, including a readable storage medium on which a computer program is stored. When the computer program is executed, it implements the text-based interactive multiple-choice intelligent question-answering method incorporating three-dimensional attention as described above.

[0080] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A text-based interactive multiple-choice intelligent question-answering method integrating three-dimensional attention, characterized in that, Includes the following steps: Step 1, Data Collection, includes collecting question data from the Internet. Each data point contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple-choice dataset is formed. Step 2, option difference interaction, includes simulating human behavior when solving multiple-choice reading comprehension questions by comparing all options in a text interaction multiple-choice answering model that integrates 3D attention, and modeling the difference information between options; including performing bilinear interaction between each answer option and other answer options to obtain an interaction matrix, and then using a gating mechanism to integrate the interaction representation into the original answer option representation, wherein the gating mechanism balances the influence of the original option representation and the interaction representation by resetting the gate; Step 3 involves textual interaction between the questions, options, and the text. This includes simulating the human reading comprehension process—understanding the connections between different types of text passages and making reading inferences—in a textual interaction multiple-choice question-answering model that integrates three-dimensional attention. A three-dimensional attention mechanism is also used to supplement the missing contextual information during the interaction between the questions and options. The implementation is as follows: After incorporating the discrepancy information in the options, a vector encoding the question is used to model the relationship between the question, options, and the article. Vectors after option difference interaction The encoded vector of the article Bidirectional matching is performed, and the representation is fused from both directions using a gating mechanism. Therefore, multiple three-dimensional attention layers are used for interaction during the interaction between questions and options. Including the problem vector before entering the attention layer Option vector Article Vector The parameters will first be reduced by an average pooling layer; The three-dimensional attention mechanism is implemented in the following way: in, The number is 12; , , First, compare with the corresponding learnable weights. , , , Perform a dot product operation to obtain the query vector. key vector value vector Context vector Then During operation, and will with The Hadama product operation incorporates contextual information, where Representative at Add a dimension before the 0th dimension of the original vector. Representative at Add a dimension to the beginning of the original vector; finally calculate the current... Attention embedding vector, multiple The attention vectors are concatenated to form the final question-aware option representation. ; Step 4, model fine-tuning, includes using the dataset from Step 1 to pre-train a text interaction multiple-choice answering model that incorporates 3D attention, and using the results to support intelligent answering.

2. The intelligent text-based interactive multiple-choice question-answering method integrating three-dimensional attention as described in claim 1, characterized in that: When collecting the dataset, web crawlers are used to collect raw HTML text from relevant websites, extract articles, questions, options, and answers, and remove redundant JavaScript code, CSS code, and comment text; then the structure of the web page text after removing the code is parsed, and HTML tags contained in the web page text are removed; finally, some symbols in the text are converted.

3. The intelligent text-based interactive multiple-choice question-answering method integrating three-dimensional attention as described in claim 1, characterized in that: The model is based on BERT. It first reads questions, options, and articles from the dataset in batches, and then encodes the question, option, and article texts using BERT to obtain the question portion vector. Vectors in the options section Vectors related to the article section , , , For text length, The dimension of the hidden layer.

4. The intelligent answering method for multiple-choice questions in text interaction fused with three-dimensional attention as described in claim 3, characterized in that: To integrate the differences between the different options, After bilinear interaction, a gating mechanism is used to integrate the interaction representation into the original answer option representation.

5. The intelligent text-based interactive multiple-choice question-answering method integrating three-dimensional attention as described in claim 1, 2, 3, or 4, characterized in that: The text interaction multiple-choice intelligent answering model that integrates three-dimensional attention reads questions, options, articles and standard answers from the dataset in batches. After reading and understanding the articles, questions and options, it makes predictions, calculates the model loss based on the predicted values ​​and the standard answers, and then trains the model based on the loss.

6. A text-based interactive multiple-choice intelligent question-answering system integrating three-dimensional attention, characterized in that: This method is used to implement a text-based interactive multiple-choice intelligent answering method that integrates three-dimensional attention, as described in any one of claims 1-5.

7. The intelligent text-based interactive multiple-choice question-answering system integrating three-dimensional attention as described in claim 6, characterized in that: Includes the following modules, The first module is used for dataset collection, which includes collecting question data from the Internet. Each data entry contains an article, multiple single-choice questions, and several options under each question. After sorting and filtering, a multiple-choice dataset is formed. The second module is used for option difference interaction, including simulating human behavior when solving multiple-choice reading comprehension questions by comparing all options and modeling the difference information between options in a text interaction multiple-choice answering model that integrates three-dimensional attention. The third module is used for text interaction between questions, options, and articles. It includes simulating the human reading comprehension process in a text interaction multiple-choice answering model that integrates three-dimensional attention, which requires understanding the connections between different types of text passages and making reading inferences. It also uses three-dimensional attention to supplement the missing contextual information when questions and options interact. The fourth module is used for model fine-tuning, including using a dataset as input to pre-train a text-based interactive multiple-choice question-answering model that incorporates 3D attention, and supporting intelligent question answering based on the results.

8. The intelligent text-based interactive multiple-choice question-answering system integrating three-dimensional attention as described in claim 7, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a text interaction multiple-choice intelligent answering method incorporating three-dimensional attention as described in any one of claims 1-5.

9. The intelligent text-based interactive multiple-choice question-answering system integrating three-dimensional attention as described in claim 7, characterized in that: It includes a readable storage medium on which a computer program is stored, and when the computer program is executed, it implements a text-based interactive multiple-choice intelligent question-answering method that integrates three-dimensional attention as described in any one of claims 1-5.

Images