Question answering scoring method and system based on pre-training language model and gated convolution unit

Abstract

The invention relates to a scoring method and system for question answering of national general languages and characters. The national general language character question answering scoring method comprises the steps that based on a public English data set Mohler, the English data set Mohler is translated into Chinese, wrong translations in the English data set Mohler are corrected and then trained, and a question answering model used for automatic scoring is obtained. The method mainly comprises a feature extraction part, a semantic similarity evaluation part and a problem score calculation part. According to the feature extraction part, a question text to be scored and an answer text to be scored are obtained and then sent to a BERT encoder, so that a question text vector and an answer text vector containing deep text semantic features are obtained; in the semantic similarity evaluation part, a multi-head attention mechanism is used for calculating the questions and the answer texts subjected to BERT fine tuning so as to determine the similarity between the questions and the answer texts; and the answer score calculation part is used for filtering irrelevant parts in answers and questions by applying a gating convolution unit, and scoring the answers of the user. According to the national general language character question answering and scoring method and system, the question answering and scoring accuracy of farmers and herdsmen in the aspect of learning national general language characters can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of language scoring, and in particular relates to a method and system for scoring answering questions based on a pre-trained language model and a gated convolutional unit. Background Art

[0002] Scoring of spoken Mandarin question answers is an important part of the Automated Speech Assessment System. Traditional automatic scoring of question answers requires human participation, which is time-consuming, labor-intensive, and subjective, and it is difficult to meet the needs of large-scale scoring. To address this problem, some automatic scoring systems have been proposed, but there are still some limitations. Current technologies are mainly based on keyword matching, syntactic analysis, or statistical methods, which cannot well consider semantic and contextual factors, and the scoring accuracy is limited. Users usually use different free texts to answer the same question, and users' answers may differ significantly in sentence structure, language style, and text length. Therefore, it is necessary to use advanced deep learning technology to combine different deep neural networks in the automatic scoring task to achieve a deeper semantic understanding of students' answers. In reality, we also hope to have a single model that can quickly, efficiently, and accurately score user answers.

[0003] At the same time, the deep learning method for automatic scoring is fully supervised machine learning, which requires assigning a label to each student's answer in the training corpus. Accurately assigning labels to free answers is very time-consuming. At present, the training corpus for automatic scoring of short answers is usually very small, usually only a few thousand, instead of tens of thousands in traditional deep learning tasks. Therefore, how to train a stable and effective deep neural network model on a small corpus is a major challenge facing the automatic scoring deep learning method; against the above background, the applicant has specially proposed a method and system for scoring answering questions based on a pre-trained language model and a gated convolutional unit, so as to improve the accuracy of scoring questions answered by farmers and herdsmen in learning the national common language and characters. Summary of the invention

[0004] The purpose of the present invention is to provide a method and system for scoring answers to questions in the national common language and characters that is simple in structure, reasonably designed, and practical in order to solve the above problems.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions: A first aspect of the present invention provides a method for scoring answer questions based on a pre-trained language model and a gated convolutional unit, the method comprising: Based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and trained it, and obtained a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

[0006] As a further optimization scheme of the present invention, based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and then trained it. The obtained automatic scoring model is a deep neural network question answering automatic scoring model (Grading). For any question Q and its answer A, let the scoring category set be Y. The scoring process of the user's answer to the question is converted to using the model to predict the most likely scoring category under the given reference answer P and answer A. ,in, During model training, any questions, reference texts, and user answer texts with different lengths are usually padded to the same data dimension. R is a real vector space, t is the length of the token sequence, and d is the dimension of the token. After any question, reference answer and answer are passed through the BERT encoder, the corresponding text content semantic vector is obtained. ; Where t is the length of the token sequence in the text, and d is the vector dimension represented by the token.

[0007] After the text data is encoded by BERT, deep semantic features are obtained. The formula is as follows, that is, ; ; As a further optimization solution of the present invention, in the BERT-Attention-GRU model, the text vector containing rich semantic information obtained by the previous module is It is fed into the bidirectional long short-term memory network and capsule network in turn to ensure that fine global semantic information and local context information are obtained. The calculation formula of text semantic information can be expressed by the following formula: is the number of hidden units in the current LSTM network, is the number of convolution kernels in the capsule network. ; ; As a further optimization scheme of the present invention, the global semantic information and local context information obtained by the previous module are integrated through the multi-head attention mechanism. The multi-head attention calculation formula is expressed by the following formula: , , The weight matrices representing the linear transformation of queries, keys, and values, respectively. ; ; ; As a further optimization scheme of the present invention, a gated convolutional unit (GRU) filters out irrelevant parts of the answer and question to obtain the final representation layer output. is the hidden state at the current moment, and the calculation formula of the gated convolution unit is as follows: ; Where z(t) is the output of the update gate, S t-1 is the input of the neuron at the previous moment, and h~(t) is the candidate hidden state.

[0008] As a further optimization scheme of the present invention, during the training phase, a multi-loss joint method is used for training; the similarity between the question and the user's answer is calculated, and the corresponding similarity loss is combined with the cross entropy loss used in the classification task, and the similarity loss is accumulated to obtain the final loss value Loss, and back propagation is performed. The calculation formula is as follows: As a further optimization scheme of the present invention, finally, the text semantic representation is fed into a linear transformation, and softmax is used to calculate the probability of each category: ; ; The second aspect of the present invention provides a scoring system for answering questions based on a pre-trained language model and a gated convolution unit, characterized in that it includes a database, a memory and a processor, the memory includes the scoring system for answering questions based on a pre-trained language model and a gated convolution unit, the database is used to store user data after model scoring, and the scoring system for answering questions based on a pre-trained language model and a gated convolution unit implements the following steps when executed by the processor: Based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and trained it, and obtained a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

[0009] Based on the public English dataset Mohler, we translated it into Chinese and corrected the incorrect translations before training. The resulting automatic scoring model is a deep neural network question answering automatic scoring model. The text to be scored obtains a deep semantic representation through the BERT encoder, and the multi-layer neural network model learns the similarities between texts. The GRU is used to filter out irrelevant information, and finally the probability value of belonging to each category is calculated through the linear layer.

[0010] The beneficial effects of the present invention are as follows: the present invention discloses a method for scoring answering questions based on a pre-trained language model and a gated convolutional unit, which uses a multi-loss joint approach to supplement the current lack of relevant research on scoring answering questions in the field of national common language oral assessment, and constructs a model called BERT-Attention-GRU by filtering information irrelevant to the question through a gated convolutional network. The model further improves the accuracy of automatic scoring of different aspects of Mandarin by fusing global and local semantic information. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A flow chart of a method for scoring answering questions based on a pre-trained language model and a gated convolutional unit is shown; Figure 2 A schematic diagram of the method flow of the BERT-Attention-GRU architecture is shown; Figure 3 A block diagram structure of a question answering scoring system based on a pre-trained language model and a gated convolutional unit is shown. Implementation

[0012] The present application is further described in detail below in conjunction with the accompanying drawings. It is necessary to point out here that the following specific implementation methods are only used to further illustrate the present application and cannot be understood as limiting the scope of protection of the present application. Technical personnel in this field can make some non-essential improvements and adjustments to the present application based on the above application content. Example

[0013] Figure 1A flow chart of a method for scoring answering questions based on a pre-trained language model and a gated convolutional unit is shown. A method for scoring answer questions based on a pre-trained language model and a gated convolutional unit, the method comprising: Based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and trained it, and obtained a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

[0014] Specifically, we first processed the dataset: we translated the public English dataset Mohler into Chinese, manually proofread the accuracy of the translated proper nouns, and replaced inaccurate terms. At the same time, we supplemented a dataset containing farmers and herdsmen answering questions, part of which contained reference texts provided by experts.

[0015] Use BERT to extract features from text: BERT uses stacked Transformers to capture very deep semantics and encodes the hidden state of a word using the weighted sum of all other word embeddings. Its definition is as follows: Among them, HiddenState represents the hidden state of the i-th word, W ij represents the weight between the i-th word and the j-th word, and n represents the total number of words in the sequence.

[0016] Compared with traditional static word vectors, the dynamic word embedding provided by BERT contains richer general knowledge, which can promote the training and convergence of upper-layer neural networks. In order to solve the problem of sequence and distance, we use the gated unit in LSTM to generate more refined global semantic information, and use the capsule network to extract local context information of each hidden state. Here, we set the number of convolution kernels of the capsule network to 400, the size of each convolution kernel to 3, and adopt dynamic routing to iterate 3 times. Each LSTM network uses 200 hidden units. Then, we use the attention mechanism to fuse the refined global vector and local context vector, and each attention mechanism module uses 2 attention heads. The fused semantic vector is sent to the gated convolutional network to filter out the parts that are not relevant to the problem, thereby improving the accuracy of subsequent classification tasks. Finally, in the prediction layer, we perform a maximum pooling operation on the obtained semantic representation information, and then use the softmax function to calculate the probability of each category.

[0017] Finally, the model outputs the score of the user's answer by mapping the classification label to a score. The final score ranges from 0 to 6.

[0018] Specifically, Figure 2 A schematic diagram of the method flow of the BERT-Attention-GRU architecture is shown; Furthermore, based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations, and trained it to obtain a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

[0019] Furthermore, based on the public English dataset Mohler, we translated it into Chinese and corrected the incorrect translations before training. The resulting automatic scoring model is a deep neural network question answering automatic scoring model (Grading). For any question Q and its answer A, let the scoring category set be Y. The scoring process for the user's answer to the question is converted to using the model to predict the most likely scoring category given the reference answer P and the answer A. ,in, During model training, any questions, reference texts, and user answer texts with different lengths are usually padded to the same data dimension. R is a real vector space, t is the length of the token sequence, and d is the dimension of the token. After any question, reference answer and answer are passed through the BERT encoder, the corresponding text content semantic vector is obtained. ; Wherein, is the length of the token sequence in the t text, and d is the vector dimension represented by the token. After the text data is encoded by BERT, the deep semantic features are obtained. The formula is as follows, that is, ; ; Furthermore, in the BERT-Attention-GRU model, the text vector containing rich semantic information obtained in the previous module is It is fed into the bidirectional long short-term memory network and capsule network in turn to ensure that fine global semantic information and local context information are obtained. The calculation formula of text semantic information can be expressed by the following formula: is the number of hidden units in the current LSTM network, is the number of convolution kernels in the capsule network. ; ; Furthermore, the global semantic information and local context information obtained by the previous module are integrated through the multi-head attention mechanism. The multi-head attention calculation formula is expressed by the following formula. , , The weight matrices representing the linear transformation of queries, keys, and values, respectively. ; ; ; Furthermore, the gated convolutional unit (GRU) filters out irrelevant parts of the answer and question to obtain the final representation layer output. is the hidden state at the current moment, and the calculation formula of the gated convolution unit is as follows: ; Where z(t) is the output of the update gate, S t-1 is the input of the neuron at the previous moment, and h~(t) is the candidate hidden state.

[0020] Furthermore, during the training phase, a multi-loss joint method is used for training; the similarity between the question and the user's answer is calculated, and the corresponding similarity loss is combined with the cross entropy loss used in the classification task, and they are accumulated to obtain the final loss value Loss, which is then back-propagated. The calculation formula is as follows: Finally, the text semantic representation is fed into a linear transformation and softmax is used to calculate the probability of each category: ; ; Figure 3 A block diagram structure of a question answering scoring system based on a pre-trained language model and a gated convolutional unit is shown.

[0021] A scoring system for answering questions based on a pre-trained language model and a gated convolution unit comprises a database, a memory and a processor. The memory comprises the scoring system for answering questions based on a pre-trained language model and a gated convolution unit, the database is used to store user data after model scoring, and the scoring system for answering questions based on a pre-trained language model and a gated convolution unit implements the following steps when executed by the processor: Based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and trained it, and obtained a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

[0022] It should be noted that when using the answer question scoring method and system, we first translate the public English dataset Mohler into Chinese, correct the incorrect translations and then train it to obtain a question answering model for automatic scoring. It mainly includes a feature extraction part, a semantic similarity evaluation part and a question scoring calculation part. Among them, the feature extraction part obtains the question text and answer text to be scored and then sends them to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; the semantic similarity evaluation part uses a multi-head attention mechanism to calculate the question and answer text after BERT fine-tuning to determine the similarity between them; the answer scoring calculation part filters out irrelevant parts of the answer and question by applying a gated convolution unit, and scores the user's answer.

[0023] It should also be noted that the present invention fully utilizes the advantages of BERT in text feature extraction, and uses the ability of LSTM and capsule network to extract global information and local information respectively, to obtain richer text representation. The accuracy, recall rate and F1 score on the public dataset reached 92.2%, 87.5% and 85.8% respectively.

[0024] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for scoring answer questions based on a pre-trained language model and a gated convolutional unit, characterized in that: The method includes, Based on the public English dataset Mohler, we translated it into Chinese, corrected the incorrect translations and trained it, and obtained a question answering model for automatic scoring; After obtaining the question text and answer text to be scored, they are sent to the BERT encoder to obtain the question text vector and answer text vector containing deep text semantic features; Use a multi-head attention mechanism to calculate the similarity between the question and answer texts after BERT fine-tuning; By applying a gated convolutional unit (GRU), we filter out irrelevant parts of the answer and question, and score the user's answer by mapping the scores to categories.

2. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 1, characterized in that: Based on the public English dataset Mohler, we translated it into Chinese and corrected the incorrect translations before training. The resulting automatic scoring model is a deep neural network question answering automatic scoring model (Grading). For any question Q and its answer A, let the scoring category set be Y. The scoring process for the user's answer to the question is converted to using the model to predict the most likely scoring category given the reference answer P and the answer A. ,in, During model training, any questions, reference texts, and user answer texts with different lengths are usually padded to the same data dimension. R is a real vector space, t is the length of the token sequence, and d is the dimension of the token.

3. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 2, characterized in that: After any question, reference answer and answer are passed through the BERT encoder, the corresponding text content semantic vector is obtained. ; Wherein, is the length of the token sequence in the t text, and d is the vector dimension represented by the token. After the text data is encoded by BERT, the deep semantic features are obtained. The formula is as follows, that is, ; ; 4. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 3, characterized in that: In the BERT-Attention-GRU model, the text vector containing rich semantic information obtained in the previous module is It is fed into the bidirectional long short-term memory network and capsule network in turn to ensure that fine global semantic information and local context information are obtained. The calculation formula of text semantic information can be expressed by the following formula: is the number of hidden units in the current LSTM network, is the number of convolution kernels in the capsule network. ; ; 5. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 4, characterized in that: The multi-head attention mechanism integrates the global semantic information and local context information obtained from the previous module. The multi-head attention calculation formula is expressed by the following formula. , , The weight matrices representing the linear transformation of queries, keys, and values, respectively. ; ; ; 6. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 5, characterized in that: The Gated Convolutional Unit (GRU) filters out irrelevant parts of the answer and question to obtain the final representation layer output. is the hidden state at the current moment, and the calculation formula of the gated convolution unit is as follows: ; Where z(t) is the output of the update gate, S t-1 is the input of the neuron at the previous moment, and h~(t) is the candidate hidden state.

7. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 1, characterized in that: During the training phase, a multi-loss joint method is used for training; the similarity between the question and the user's answer is calculated, and the corresponding similarity loss is combined with the cross entropy loss used in the classification task, and they are accumulated to obtain the final loss value Loss, which is then back-propagated. The calculation formula is as follows:

8. The method for scoring answer questions based on a pre-trained language model and a gated convolutional unit according to claim 7, characterized in that: Finally, the text semantic representation is fed into a linear transformation and softmax is used to calculate the probability of each category: ; ; 9. A question answering scoring system based on a pre-trained language model and a gated convolutional unit, characterized in that: The invention comprises a database, a memory and a processor, wherein the memory comprises a scoring system for answering questions based on a pre-trained language model and a gated convolution unit, the database is used to store user data after being scored by the model, and the scoring system for answering questions based on a pre-trained language model and a gated convolution unit implements the following steps when executed by the processor: based on the public English data set Mohler, we translate it into Chinese, correct the incorrect translations therein and then train it, and obtain a model for answering questions for automatic scoring, which mainly comprises a feature extraction part, a semantic similarity evaluation part and a question scoring calculation part. Among them, the feature extraction part obtains the question text and the answer text to be scored and then sends them to the BERT encoder to obtain the question text vector and the answer text vector containing deep text semantic features; the semantic similarity evaluation part uses a multi-head attention mechanism to calculate the question and answer text after BERT fine-tuning to determine the similarity between them; the answer scoring calculation part filters out the irrelevant parts of the answer and question by applying the gated convolution unit, and scores the user's answer. The scoring method and system for answering questions on the national common language and characters can improve the accuracy of scoring for farmers and herdsmen's answers to questions about learning the national common language and characters.

Images