Question correction method and device, electronic equipment and storage medium

Abstract

This invention provides a question error correction method, apparatus, electronic device, and storage medium. It employs a question error correction model, which includes an encoding layer and a decoding layer. The encoding layer comprises a first-type convolutional neural network layer and the encoding end of a Transformer layer. The decoding layer comprises a second-type convolutional neural network layer and the decoding end of a Transformer layer. The question text samples and their corresponding corrected question text samples used during training can be selected based on the required accuracy of the question error correction model, thus addressing the problem of excessively large parameter space in N-gram models when N is too large. Furthermore, since the question error correction model utilizes word vectors, which are a distributed feature representation, they take into account the semantics of words, effectively mitigating the high sparsity of traditional statistical word representations.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a question error correction method, apparatus, electronic device, and storage medium. Background Technology

[0002] Grammatical error correction (GEC) is one of the more challenging tasks in syntactic analysis within natural language processing. Current methods for GEC in natural language processing primarily address spelling errors and grammatical errors.

[0003] Existing Chinese spelling error detection and correction technologies mainly include rule-based methods and statistical machine learning methods, with the latter being most commonly used in language models and classification. For example, the weighted channel noise model-based error correction method, improved from the N-gram statistical language model, draws on the Chinese spelling verification algorithm based on the N-gram language model and proposes a weighted noise channel model-based Chinese spelling error correction algorithm. This algorithm uses a Trigram language model as the language model, word frequency probability as the transformation model, and incorporates the weights of confused characters, while using the Beam Search algorithm as the decoding algorithm.

[0004] Since the number of parameters in an N-gram language model increases exponentially with the increase of N, this leads to an excessively large parameter space, reducing error correction efficiency. Furthermore, N-gram statistical language models may suffer from data smoothing issues due to the sparsity of word data in the text to be corrected, further reducing the accuracy of error correction. Summary of the Invention

[0005] This invention provides a question error correction method, apparatus, electronic device, and storage medium to address the deficiencies in the prior art.

[0006] In a first aspect, the present invention provides a question error correction method, comprising:

[0007] Obtain the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and determine the word vector and byte pair encoded text of the question text to be corrected;

[0008] The word vectors and the byte-pair encoded text are input into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer;

[0009] The error attribute features are input into the decoding layer of the question correction model to obtain the corrected question text output by the decoding layer.

[0010] The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples.

[0011] In one embodiment, determining the word vectors and byte-pair encoded text of the question text to be corrected specifically includes:

[0012] The text of the question to be corrected is converted into token form to obtain a word sequence;

[0013] Based on the byte-pair encoding algorithm, the character sequence is segmented to obtain the word vector corresponding to the character sequence and the byte-pair encoded text.

[0014] In one embodiment, the step of inputting the word vectors and the byte-pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer specifically includes:

[0015] The byte-pair encoded text is input into the first type of convolutional neural network layer to obtain the position encoding features of the word segmented text corresponding to the word vector output by the first type of convolutional neural network layer;

[0016] Based on the positional encoding features of the segmented text corresponding to the word vectors, the aggregation features corresponding to the word vectors are determined;

[0017] The aggregated feature is input to the encoding end to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end.

[0018] In one embodiment, determining the aggregation feature corresponding to the word vector based on the positional encoding features of the segmented text corresponding to the word vector specifically includes:

[0019] The positional encoding feature is concatenated with the word vector to obtain the concatenated feature corresponding to the word vector;

[0020] The splicing features are input into the first type of convolutional neural network layer to obtain the error location features of the question text to be corrected output by the first type of convolutional neural network layer;

[0021] The aggregation feature is determined based on the error location feature.

[0022] In one embodiment, the step of inputting the aggregated feature to the encoding end to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end specifically includes:

[0023] The aggregated features are input to the encoding end to obtain the deep interaction features corresponding to the aggregated features output by the encoding end;

[0024] The deep interaction features are input to the encoding end to obtain the error attribute features output by the encoding end.

[0025] In one embodiment, the question correction model is trained in the following manner:

[0026] The question text sample and the corrected question text sample are input into the question error correction model to be trained to obtain the first type of sample features corresponding to the question text sample and the second type of sample features corresponding to the corrected question text sample.

[0027] Calculate the first-class matching score of the first-class sample features and the second-class matching score of the second-class sample features, and calculate the loss function value based on the first-class matching score and the second-class matching score;

[0028] Based on the loss function value, the question correction model to be trained is trained to obtain the question correction model.

[0029] In one embodiment, the context text is elliptical text.

[0030] Secondly, the present invention provides a question correction device, comprising:

[0031] The acquisition module is used to acquire the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and to determine the word vector of the question text to be corrected;

[0032] The encoding module is used to input the word vectors into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer;

[0033] The decoding module is used to input the error attribute features into the decoding layer of the question correction model to obtain the corrected question text output by the decoding layer;

[0034] The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples.

[0035] Thirdly, the present invention provides an electronic device, including a memory and a memory storing a computer program, wherein the processor executes the program to implement the steps of the question correction method described in the first aspect.

[0036] Fourthly, the present invention provides a processor-readable storage medium storing a computer program for causing the processor to perform the steps of the question correction method described in the first aspect.

[0037] The question correction method, apparatus, electronic device, and storage medium provided by this invention employ a question correction model, which includes an encoding layer and a decoding layer. The encoding layer includes the encoding ends of a first-type convolutional neural network layer and a Transformer layer, while the decoding layer includes the decoding ends of a second-type convolutional neural network layer and a Transformer layer. The question text samples and their corresponding corrected question text samples used during training can be selected based on the required accuracy of the question correction model, thus addressing the problem of excessively large parameter space in N-gram models when N is too large. Furthermore, because the question correction model utilizes word vectors, which are a distributed feature representation, they take into account word semantics, effectively mitigating the high sparsity of traditional statistical word representations and incorporating semantic information into specific scenarios, thereby improving the ability to detect and correct spelling errors. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the question correction method provided by the present invention;

[0040] Figure 2 This is a schematic diagram of the structure of the question correction model provided by the present invention;

[0041] Figure 3 This is a schematic diagram of the question correction device provided by the present invention;

[0042] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] Existing Chinese spelling error detection and correction technologies mainly include rule-based methods and statistical machine learning methods, with the latter being most commonly used in language modeling and classification. These methods are often not used independently, but rather work together in combination.

[0045] (1) Rule-based spelling correction methods: These methods use conventional grammatical rules such as chunking, syntax, and grammar to determine whether there are spelling errors in a sentence. These methods require word segmentation, chunking recognition, and phrase recognition of Chinese text. If the segmentation results cannot meet the grammatical rules such as syntax or grammar, they are considered to be potentially erroneous.

[0046] (2) Classification-based methods. These methods are similar to the Real-word method in English spelling correction. The main idea is to collect a large-scale corpus of correct Chinese text and use feature engineering to select a set of effective features as training samples to train a classification model. One researcher proposed a Max Entity-based method, training a binary maximum entropy model for each Chinese character using a large-scale corpus. The idea is that each character in a sentence can be considered a binary classification problem containing two categories: correct and incorrect, correcting the characters in the sentence to be corrected one at a time. Another researcher proposed a logistic regression (LR) method, HANSpeller++. This method extracts features from all new sentences obtained after replacing characters in the sentence to be corrected using a confusion set, and then uses LR for second-order classification. The first-order classification is based on word segmentation features, language model features, edit distance features, and dictionary features to obtain the 20 optimal candidate sentences. These 20 candidate sentences are then introduced with new features: Web-based features, translation features, and translation rationality features for second-order classification, obtaining the 5 optimal candidate sentences. Finally, the corrected sentence is obtained through grammatical rules.

[0047] (3) Statistical language model-based methods. These methods use a mixin set to replace characters in the original sentence, resulting in many candidate sentences. A language model is then used to score the replaced sentences, and the highest-scoring sentence is compared with the original sentence to identify misspelled characters and provide correction suggestions. This method uses conditional probability analysis of the closeness of relationships between characters or words for error correction. For example, in Chinese error correction methods based on statistical language models used in search engines, the statistical analysis results of the language model are combined with TF / IDF for error correction. Some researchers have proposed a spelling correction method combining Chinese word segmentation, bi-gram, and tri-gram language models. Other researchers have proposed a method combining language models and pattern matching. This method first uses the longest matching method for Chinese word segmentation, and then combines the segmentation results with n-gram grammar to obtain the co-occurrence frequency of the word with its left and right adjacent words. If the frequency is below a certain threshold, the word is considered potentially misspelled.

[0048] In existing technologies, the weighted channel noise model error correction method based on the N-gram statistical language model is based on the following idea: It borrows from the Chinese spelling verification algorithm based on the N-gram language model and proposes a Chinese spelling error correction algorithm based on the weighted noise channel model. This algorithm uses a Trigram language model as the language model, word frequency probability as the transformation model, and incorporates the weights of confused words, while using the Beam Search algorithm as the decoding algorithm.

[0049] The specific implementation steps are as follows:

[0050] Step 1: Model preparation. First, the training corpus is segmented into Unigram, Bigram, and Trigram segments, and word frequencies are counted.

[0051] 1.1 Construct a character frequency table charDict based on Unigram statistics;

[0052] 1.2. Train a trigram language model (LM) based on the given N-gram language model formula. This LM serves as the N-gram probability dictionary and language model P(I) for the error detection module. The language model format follows the ARPA format given by the SRILM framework (containing probabilities of unary, binary, and trigram languages ​​simultaneously). The first line of this format is the model identifier. The following rows represent the order of the model and the number of model parameters for each order, separated by blank lines. Then, the specific parameters of each order model are listed, separated by blank lines. Each row represents one parameter and is divided into three columns: L (logarithm of the probability of the n-gram), the n-gram itself, and the backoff weight of the n-gram (no backoff weight for the highest order).

[0053] 1.3. Calculate the confusion set conversion model according to the character frequency table char Dict and the confusion set conSet according to the following formula:

[0054]

[0055] Among them, the character frequency table is a table that counts the number of occurrences of each character in a text, and the confusion set is a set constructed for each character, and the characters in this set have the characteristics of the same pronunciation or spelling. For example, some of the confusing characters of "与" are "雨鱼云丐语", etc.

[0056] 1.4. According to the confusion set conSet, convert the confusing characters into pinyin, and calculate the weight of each confusing character according to the minimum edit distance of pinyin according to the following formula, and construct the weight confusion set weightSet:

[0057]

[0058] Step 2. Input. The original sentence Sentence, the position information error List of the misspelled characters obtained by the error detection module based on CBNP for error detection according to trigram LM, and the initialized error correction result Correct Result.

[0059] Step 3. Correction. Traverse each position information in error List, obtain the confusion character set con List according to the marked characters, and replace them with each confusing character in con List respectively to obtain all new sentences. Calculate the probability of the sentence according to the following formula to obtain the sentence new Sen with the highest score.

[0060]

[0061] Step 4. Output. Judge whether new Sen is the same as Sentence. If they are the same, use CorrectResult to record all the modified characters and position information.

[0062] The above technical solutions have the following disadvantages:

[0063] (1) First of all, a relatively large-scale training text is required to determine the model parameters. When N is very large, the parameter space of the model is too large; as shown in Table 1, it is the parameter quantity table of the N-ary language model

[0064] Table 1 Parameter Quantity of N-ary Language Model

[0065] Model Number of parameters Unigram (a unigram language model) 20000 Bnigram (a binary language model) <![CDATA[20000 2 =4×10 8 ]]> Tnigram (a language model for ternary words) <![CDATA[20000 3 =8×10 12 ]]> FOUR-gram (four-gram language model) <![CDATA[20000 4 =1.6×10 17 ]]>

[0066] (2) Secondly, there is the problem of data smoothing that may be caused by data sparsity;

[0067] (3) Finally, the N-gram model is constructed based on discrete unit words that have no genetic attributes among themselves, thus it does not have the semantic advantage of word vectors in continuous space: words with similar meanings have similar word vectors, so when the system model adjusts the parameters for a certain word or word sequence, words and word sequences with similar meanings will also change.

[0068] (4) Furthermore, existing technologies do not have corresponding completion techniques for situations where questions are omitted or pronouns are used, resulting in ambiguous semantic information.

[0069] Therefore, this invention provides a question error correction method to solve the above-mentioned technical problems. Figure 1 This is a flowchart illustrating a question correction method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method includes:

[0070] S1, obtain the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and determine the word vector and byte pair encoded text of the question text to be corrected;

[0071] S2, input the word vector and the byte pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer;

[0072] S3, input the error attribute features into the decoding layer of the question correction model to obtain the corrected question text of the question text to be corrected output by the decoding layer;

[0073] The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples.

[0074] Specifically, the question correction method provided in this embodiment of the invention is executed by a server, which can be a local server or a cloud server. The local server can be a computer, etc., and this embodiment of the invention does not make specific limitations on this.

[0075] First, step S1 is executed to obtain the question text to be corrected, which contains the erroneous question text and its context text. The question text to be corrected refers to the question text containing the erroneous question text and its context text. The erroneous question text is the question text containing incorrect information, and the context text is the text adjacent to the erroneous question text. It can be before or after the erroneous question text, and can be a complete sentence or an elliptical sentence. The error type of the erroneous question text can be a word error, a sentence error, etc. When the context text is an elliptical sentence, it can also be considered an erroneous question text, and its error type is an omission error. For example, the question text to be corrected could be "Where in Beijing can I cancel my account in a different location? What about Sanya?", where "Where in Beijing can I cancel my account in a different location?" is the erroneous question text, and "What about Sanya?" is the context text, and this context text is an elliptical sentence.

[0076] After obtaining the text of the question to be corrected, word embeddings and byte-pair encoded text can be determined using word segmentation technology. A word embedding is a distributed representation of a word, where each word is represented as a continuous, dense vector of a fixed length. In this embodiment of the invention, the word embedding can be represented as E = {e1, e2, e3, ..., e...} i ,…,e m}, where E represents the set of word vectors, e i Let represent the i-th word vector, and m be the number of word vectors in the question text to be corrected. Byte-pair encoding (BPE) text refers to the text obtained by segmenting the question text to be corrected using the BPE algorithm.

[0077] Then, step S2 is executed, where word vectors and byte-pair encoded text are input into the encoder layer of the question correction model to obtain the error attribute features of the question text to be corrected, output by the encoder layer. The question correction model can include an encoder layer and a decoder layer. The encoder layer extracts the error attribute features of the question text to be corrected. These error attribute features can include error location features and error type features. Error location features characterize the location of the error information in the question text, while error type features characterize the type of error information, which can be word errors, sentence structure errors, omission errors, etc.

[0078] Finally, step S3 is executed, inputting the error attribute features into the decoding layer of the question correction model to obtain the corrected question text output by the decoding layer. The corrected question text refers to the correct and complete text corresponding to the question text to be corrected, which can include the correct text corresponding to the erroneous question text and the complete text corresponding to the context text when the context text is an abbreviated sentence. For example, the corrected question text could be "Where in Beijing can I process a cancellation in a different location? Where in Sanya can I process a cancellation in a different location?"

[0079] In this embodiment of the invention, the encoding layer of the question correction model includes the encoding ends of a first-type Convolutional Neural Network (CNN) layer and a Transformer layer, and the decoding layer also includes the decoding ends of a second-type Convolutional Neural Network (CNN) layer and a Transformer layer. CNN is a type of feedforward neural network with a deep structure that includes convolutional computation, and is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities, enabling them to perform shift-invariant classification of input information according to their hierarchical structure. The Transformer abandons traditional CNNs and RNNs; the entire network structure is composed entirely of attention mechanisms. The Transformer consists of only self-attention and a Feed Forward Neural Network.

[0080] The number of first-type convolutional neural network (CNN) layers and Transformer layers in the encoding layer can be set as needed, as can the number of second-type CNN layers and Transformer layers in the decoding layer. For example, the first-type CNN layers can be 4, and the number of Transformer layers in the encoding layer can be 8. That is, the encoding layer can be viewed as a stacked model of the encoding ends of 4 first-type CNN layers and 8 Transformer layers, i.e., a Transformer-CNN model. The second-type CNN layers in the decoding layer can be 4, and the number of Transformer layers in the decoding layer can be 6. That is, the decoding layer can be viewed as a stacked model of the decoding ends of 4 first-type CNN layers and 6 Transformer layers, i.e., a Transformer-CNN model. In this embodiment of the invention, the word vector dimension can be set to 512, the hidden layer dimension to 1024, and the convolutional window size to 3 in the encoding and decoding layers.

[0081] In this embodiment of the invention, the question error correction model can be obtained by training a question error correction model to be trained using question text samples containing error information and corresponding corrected question text samples. The question error correction model to be trained can include an encoding layer and a decoding layer, both of which can be Transformer-CNN models. The question text samples and corresponding corrected question text samples can be selected according to the accuracy of the question error correction model required for actual training.

[0082] The question correction method provided in this embodiment of the invention employs a question correction model, which includes an encoding layer and a decoding layer. The encoding layer includes the encoding ends of a first-type convolutional neural network layer and a Transformer layer, while the decoding layer includes the decoding ends of a second-type convolutional neural network layer and a Transformer layer. The question text samples and their corresponding corrected question text samples used during training can be selected based on the required accuracy of the question correction model, thus addressing the problem of excessively large parameter space in N-gram models when N is too large. Furthermore, since the question correction model utilizes word vectors, which are a distributed feature representation, they can consider the semantics of words, effectively mitigating the high sparsity of traditional statistical word representations and incorporating semantic information into specific scenarios, thereby improving the ability to detect and correct spelling errors.

[0083] Furthermore, in this embodiment of the invention, the context text can be elliptical sentence text. Moreover, when the context text is elliptical sentence text, the question correction model can output the correct text corresponding to the erroneous question text and the complete text corresponding to the context text. This embodiment of the invention addresses the shortcomings of existing technologies in lacking corresponding completion techniques for semantically ambiguous information such as omitted or pronoun-related information in questions. These are categorized under question correction, and a multi-turn QA retrieval model is used to consider multiple rounds of queries when returning the answer. A Transformer is introduced into the question correction model, learning error types and similar errors during the learning process to facilitate rapid detection of similar errors, exhibiting strong scalability and robustness. Ultimately, this enables the question correction model to locate and focus on errors, correcting and completing them, thereby improving question retrieval speed and response accuracy. In other words, the question correction method provided in this embodiment of the invention can be applied to scenarios such as intelligent question answering systems where users input questions with spelling errors, or multiple questions asked simultaneously with omitted pronouns or other words.

[0084] Based on the above embodiments, the question correction method provided in this embodiment of the invention, wherein determining the word vector and byte-pair encoded text of the question text to be corrected specifically includes:

[0085] The text of the question to be corrected is converted into token form to obtain a word sequence;

[0086] Based on the byte-pair encoding algorithm, the character sequence is segmented to obtain the word vector corresponding to the character sequence and the byte-pair encoded text.

[0087] Specifically, in this embodiment of the invention, when determining the word vector of the question text to be corrected, to address the problem of sparse words that may contain spelling errors in the question, the question text to be corrected can first be converted into token form to obtain the word sequence X = {x1, x2, ..., x...} j ,…,x n}. x j Let X represent the j-th character in the text to be corrected, and n be the number of characters in the text. Then, according to the byte-pair encoding algorithm, the character sequence is segmented into words to obtain the word vector E corresponding to the character sequence X. Since a word is composed of several characters, m ≤ n. Simultaneously with obtaining the word vector E corresponding to the character sequence X, the BPE text BPE can also be obtained using the BPE algorithm. x ={x e1 ,x e2 ,x e3 ,…,x em}

[0088] BPE (Bottom-Up Formatting) is a compression algorithm, a bottom-up approach. It treats words as word pieces to facilitate handling words that haven't appeared in the vocabulary list. In Non-Mean Transformation (NMT) tasks, BPE first divides the training set words into pieces. Then, these pieces are randomly assigned values ​​and fed into RNNs or CNNs to train their embeddings. Finally, the pieces are combined to obtain word vectors, which are then used for NMT. ​​This way, if uncommon or out-of-vocabulary words are encountered in the training set or elsewhere, the fragments can be directly combined for NMT.

[0089] In this embodiment of the invention, word vectors are determined by encoding using a byte-to-byte encoding algorithm, which can solve the problem of potentially sparse words.

[0090] Based on the above embodiments, the question correction method provided in this embodiment of the invention, wherein the step of inputting the word vector byte pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer, specifically includes:

[0091] The byte-pair encoded text is input into the first type of convolutional neural network layer to obtain the position encoding features of the word segmented text corresponding to the word vector output by the first type of convolutional neural network layer;

[0092] Based on the positional encoding features of the segmented text corresponding to the word vectors, the aggregation features corresponding to the word vectors are determined;

[0093] The aggregated feature is input to the encoding end to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end.

[0094] Specifically, in this embodiment of the invention, the encoding layer may include a first type of convolutional neural network layer and the encoding end of a Transformer. To address the problem of the lack of temporal feature information in the input of the question correction model, the first type of convolutional neural network layer can perform positional encoding on the byte-to-byte encoded text to obtain positional encoding features. The first type of convolutional neural network layer can perform positional encoding on the byte-to-byte encoded text using the following formula:

[0095] or

[0096]

[0097] Where POS is the position index of the i-th word vector obtained after processing by the BPE algorithm, PE is the positional encoding function, and d model Let be the dimension of the i-th word vector.

[0098] The obtained positional coding features can be represented as: P = {p1, p2, p3, ..., p m}

[0099] Then, based on the positional encoding features P of the segmented text corresponding to the word vectors, the aggregated features corresponding to the word vectors can be determined. The aggregated features refer to the features obtained by concatenating the word vectors and positional encoding features, performing error localization and feature representation again, and then performing max pooling. These features can be used to characterize whether there are errors in the words corresponding to the word vectors.

[0100] Finally, the aggregated features are input to the encoder of the Transformer. The encoder of the Transformer can then be used to extract deeper interactive features and finally output the error attribute features corresponding to the aggregated features.

[0101] In this embodiment of the invention, error attribute features of the question text to be corrected are obtained through the encoding layer. In the encoding layer, shallow error features are extracted through a first type of convolutional neural network layer, and deep interactive features are extracted through the encoding end of the Transformer, which can make the error attribute features more accurate.

[0102] Based on the above embodiments, the question correction method provided in this embodiment of the invention, which determines the aggregation feature corresponding to the word vector based on the positional encoding feature of the segmented text corresponding to the word vector, specifically includes:

[0103] The positional encoding feature is concatenated with the word vector to obtain the concatenated feature corresponding to the word vector;

[0104] The splicing features are input into the first type of convolutional neural network layer to obtain the error location features of the question text to be corrected output by the first type of convolutional neural network layer;

[0105] The aggregation feature is determined based on the error location feature.

[0106] Specifically, in this embodiment of the invention, when determining the aggregation feature corresponding to the word vector, the positional encoding feature is first concatenated with the word vector to obtain the concatenated feature corresponding to the word vector. This concatenated feature is rich in positional information and can ultimately be represented as W(x i )=W(p i +BPE(x i )+b), BPE(x) i ) = x ei The splicing method can be either direct splicing or splicing according to the corresponding positions.

[0107] The resulting concatenated features are then input into the first type of CNN layer, which again performs error localization and feature representation on the question. The feature representation is then used to generate the final feature representation. i =f(w·x) i：i+h-1 )+b locates the incorrect phrase. Where, c i is the error location feature of the i-th word vector obtained from the convolutional layer in the first type of CNN layer, b is the threshold of each neuron in the output layer of the first type of CNN layer, and f is the activation function. Since it is necessary to specifically identify the range of erroneous words, this embodiment of the invention only considers surrounding words or characters of size h, where h is a set value. For the question error correction model, after setting h, c i The input parameters are within the range of h.

[0108] Finally, by concatenating all the error location features, we have:

[0109] c = [c1, c2, ..., c m ]

[0110] Where c represents the splicing feature.

[0111] The aggregated features can be determined by performing max-pooling on the concatenated features.

[0112] In this embodiment of the invention, a method for determining aggregated features is provided, which can make the obtained aggregated features more accurate by using a first type of CNN layer.

[0113] Based on the above embodiments, the question error correction method provided in this embodiment of the invention, wherein the aggregated feature is input to the encoding end to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end, specifically includes:

[0114] The aggregated features are input to the encoding end to obtain the deep interaction features corresponding to the aggregated features output by the encoding end;

[0115] The deep interaction features are input to the encoding end to obtain the error attribute features output by the encoding end.

[0116] Specifically, in this embodiment of the invention, when determining error attribute features, the aggregated features are first input to the encoder of the Transformer to obtain the deep interaction features corresponding to the aggregated features output by the encoder, such as error attribute features. Then, the deep interaction features are input to the encoder of the Transformer again, and the error location features and error type features can be found more accurately after passing through the encoder of the Transformer again. For example, the features output by the first type of CNN layer are encoded by the encoder of the Transformer. For the output of each first type of CNN layer, starting from the word level, the corresponding encoder of the Transformer performs word-to-word interactions, amplifying each word vector while simultaneously performing a self-attention mechanism to detect errors, thus enabling the detection of erroneous words and obtaining error attribute features.

[0117] In this embodiment of the invention, a method for determining error attribute features is provided, which can make the obtained error attribute features more accurate by using the encoding end of the Transformer.

[0118] Based on the above embodiments, the question correction method provided in this embodiment of the invention, wherein the question correction model is trained in the following manner:

[0119] The question text sample and the corrected question text sample are input into the question error correction model to be trained to obtain the first type of sample features corresponding to the question text sample and the second type of sample features corresponding to the corrected question text sample.

[0120] Calculate the first-class matching score of the first-class sample features and the second-class matching score of the second-class sample features, and calculate the loss function value based on the first-class matching score and the second-class matching score;

[0121] Based on the loss function value, the question correction model to be trained is trained to obtain the question correction model.

[0122] Specifically, in this embodiment of the invention, when training the question error correction model, the question text samples are first... And corrected question text samples The input is fed into the question correction model to be trained, and the first type of sample features corresponding to the question text sample and the second type of sample features corresponding to the corrected question text sample are obtained.

[0123] Calculate the first-class matching score p for the features of the first-class samples and the second-class matching score u for the features of the second-class samples. Based on the first-class matching score p and the second-class matching score u, calculate the loss function value. The formula for the loss function can be expressed as:

[0124]

[0125] in, S represents the loss function value. * It is a set of matching scores, θ crt These are the training parameter values ​​in the question correction model to be trained, P(p|μ;θ). crt ) indicates that at θ crt The probability of p under condition μ.

[0126] Based on the loss function value, the question correction model is trained. Through multiple rounds of correction until the loss function converges, the question correction model can be obtained.

[0127] Figure 2 This is a schematic diagram of the structure of a question error correction model provided in an embodiment of the present invention, such as... Figure 2 As shown, the question correction model includes a BPE layer 1, an encoder layer 2, and a decoder layer 3. The encoder layer 2 includes a first-class CNN layer 21 and an encoder end 22 of the Transformer. The encoder end 22 includes a multi-head self-attention layer 221 and a forward propagation layer 222. The decoder layer 3 includes a forward propagation layer 31, an encoder-decoder attention layer 32, and a self-attention layer 33.

[0128] In coding layer 2, coding end 22 may include stacking and normalization layers between multi-head self-attention layer 221 and forward propagation layer 222, and after forward propagation layer 222.

[0129] In summary, the question correction method provided in this embodiment addresses the shortcomings of traditional N-gram-based improved models, which use discrete unit words without considering sentence word vectors and semantic information. This embodiment uses a CNN model to obtain the temporal and positional features of words in the sentence, and finally concatenates and pools them to obtain aggregated features. For sentence spelling detection, the aggregated feature vector output by the CNN is encoded using a Transformer model, and the self-attention mechanism of the Transformer model is used to detect errors, enabling the detection of incorrect words. Furthermore, addressing the shortcomings of spelling correction and traditional sentence correction algorithms in considering potentially omitted information in questions, this embodiment continues to use a Transformer model combined with contextual questions for multi-round error correction and question information completion.

[0130] Figure 3 This is a schematic diagram of the structure of a question correction device provided in an embodiment of the present invention, such as... Figure 3 As shown, the device includes:

[0131] The acquisition module 31 is used to acquire the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and to determine the word vector of the question text to be corrected.

[0132] Encoding module 32 is used to input the word vectors into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer;

[0133] Decoding module 33 is used to input the error attribute features into the decoding layer of the question correction model to obtain the corrected question text of the question text to be corrected output by the decoding layer;

[0134] The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples.

[0135] Based on the above embodiments, the question correction device provided in this embodiment of the invention, wherein the acquisition module is specifically used for:

[0136] The text of the question to be corrected is converted into token form to obtain a word sequence;

[0137] Based on the byte-pair encoding algorithm, the character sequence is segmented to obtain the word vector corresponding to the character sequence and the byte-pair encoded text.

[0138] Based on the above embodiments, the question correction device provided in this embodiment of the invention, wherein the encoding module is specifically used for:

[0139] The byte-pair encoded text is input into the first type of convolutional neural network layer to obtain the position encoding features of the word segmented text corresponding to the word vector output by the first type of convolutional neural network layer;

[0140] Based on the positional encoding features of the segmented text corresponding to the word vectors, the aggregation features corresponding to the word vectors are determined;

[0141] The aggregated feature is input to the encoding end to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end.

[0142] Based on the above embodiments, the question correction device provided in this embodiment of the invention, wherein the encoding module is further specifically used for:

[0143] The positional encoding feature is concatenated with the word vector to obtain the concatenated feature corresponding to the word vector;

[0144] The splicing features are input into the first type of convolutional neural network layer to obtain the error location features of the question text to be corrected output by the first type of convolutional neural network layer;

[0145] The aggregation feature is determined based on the error location feature.

[0146] Based on the above embodiments, the question correction device provided in this embodiment of the invention, wherein the encoding module is further specifically used for:

[0147] The aggregated features are input to the encoding end to obtain the deep interaction features corresponding to the aggregated features output by the encoding end;

[0148] The deep interaction features are input to the encoding end to obtain the error attribute features output by the encoding end.

[0149] Based on the above embodiments, the question correction device provided in this embodiment of the invention further includes a training module, used for:

[0150] The question text sample and the corrected question text sample are input into the question error correction model to be trained to obtain the first type of sample features corresponding to the question text sample and the second type of sample features corresponding to the corrected question text sample.

[0151] Calculate the first-class matching score of the first-class sample features and the second-class matching score of the second-class sample features, and calculate the loss function value based on the first-class matching score and the second-class matching score;

[0152] Based on the loss function value, the question correction model to be trained is trained to obtain the question correction model.

[0153] Based on the above embodiments, the question correction device provided in this embodiment of the invention uses elliptical sentence text as the context text.

[0154] Specifically, the functions of each module in the question correction device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0155] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the computer program in the memory 430 to execute the steps of the question correction method provided in the above embodiments, such as: obtaining the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and determining the word vector and byte pair encoded text of the question text to be corrected; inputting the word vector and the byte pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer; inputting the error attribute features into the decoding layer of the question correction model to obtain the corrected question text of the question text to be corrected output by the decoding layer; wherein, the encoding layer includes the encoding end of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding end of a second type of convolutional neural network layer and a Transformer layer, and the question correction model is trained based on the question text sample containing error information and the corrected question text sample corresponding to the question text sample.

[0156] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer is able to execute the steps of the question correction method provided in the above embodiments, for example including: obtaining a question text to be corrected containing erroneous question text and context text of the erroneous question text, and determining the word vector and byte pair encoded text of the question text to be corrected; inputting the word vector and the byte pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer; inputting the error attribute features into the decoding layer of the question correction model to obtain the corrected question text of the question text to be corrected output by the decoding layer; wherein, the encoding layer includes the encoding end of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding end of a second type of convolutional neural network layer and a Transformer layer, and the question correction model is trained based on question text samples containing error information and the corrected question text samples corresponding to the question text samples.

[0158] On the other hand, embodiments of this application also provide a processor-readable storage medium storing a computer program. The computer program is used to cause the processor to execute the steps of the question correction method provided in the above embodiments, such as: obtaining a question text to be corrected containing erroneous question text and context text of the erroneous question text, and determining the word vector and byte pair encoded text of the question text to be corrected; inputting the word vector and the byte pair encoded text to the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer; inputting the error attribute features to the decoding layer of the question correction model to obtain the corrected question text of the question text to be corrected output by the decoding layer; wherein, the encoding layer includes the encoding end of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding end of a second type of convolutional neural network layer and a Transformer layer, and the question correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples.

[0159] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0160] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0161] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for correcting errors in questions, characterized in that, include: Obtain the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and determine the word vector and byte pair encoded text of the question text to be corrected; The word vectors and the byte-pair encoded text are input into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer; The error attribute features are input into the decoding layer of the question correction model to obtain the corrected question text output by the decoding layer. The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples. The step of inputting the word vectors and the byte-pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer specifically includes: The byte-pair encoded text is input into the first type of convolutional neural network layer to obtain the position encoding features of the word segmented text corresponding to the word vector output by the first type of convolutional neural network layer; Based on the positional encoding features of the segmented text corresponding to the word vectors, the aggregation features corresponding to the word vectors are determined; The aggregated feature is input to the encoding end of the Transformer layer to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end of the Transformer layer.

2. The question correction method according to claim 1, characterized in that, The process of determining the word vectors and byte-pair encoded text of the question text to be corrected specifically includes: The text of the question to be corrected is converted into token form to obtain a word sequence; Based on the byte-pair encoding algorithm, the character sequence is segmented to obtain the word vector corresponding to the character sequence and the byte-pair encoded text.

3. The question correction method according to claim 1, characterized in that, The step of determining the aggregation feature corresponding to the word vector based on the positional encoding features of the segmented text corresponding to the word vector specifically includes: The positional encoding feature is concatenated with the word vector to obtain the concatenated feature corresponding to the word vector; The splicing features are input into the first type of convolutional neural network layer to obtain the error location features of the question text to be corrected output by the first type of convolutional neural network layer; The aggregation feature is determined based on the error location feature.

4. The question correction method according to claim 1, characterized in that, The step of inputting the aggregated features into the encoder of the Transformer layer to obtain the error attribute features corresponding to the aggregated features output by the encoder of the Transformer layer specifically includes: The aggregated features are input to the encoder of the Transformer layer to obtain the deep interaction features corresponding to the aggregated features output by the encoder of the Transformer layer. The deep interaction features are input to the encoder of the Transformer layer to obtain the error attribute features output by the encoder of the Transformer layer.

5. The question correction method according to any one of claims 1-4, characterized in that, The question correction model is trained in the following manner: The question text sample and the corrected question text sample are input into the question error correction model to be trained to obtain the first type of sample features corresponding to the question text sample and the second type of sample features corresponding to the corrected question text sample. Calculate the first-class matching score of the first-class sample features and the second-class matching score of the second-class sample features, and calculate the loss function value based on the first-class matching score and the second-class matching score; Based on the loss function value, the question correction model to be trained is trained to obtain the question correction model.

6. The question correction method according to any one of claims 1-4, characterized in that, The context text is in the form of elliptical sentences.

7. A question correction device, characterized in that, include: The acquisition module is used to acquire the question text to be corrected, which contains the erroneous question text and the context text of the erroneous question text, and to determine the word vector and byte pair encoded text of the question text to be corrected. The encoding module is used to input the word vectors and the byte-pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer; The decoding module is used to input the error attribute features into the decoding layer of the question correction model to obtain the corrected question text output by the decoding layer; The encoding layer includes the encoding ends of a first type of convolutional neural network layer and a Transformer layer, the decoding layer includes the decoding ends of a second type of convolutional neural network layer and a Transformer layer, and the question error correction model is trained based on question text samples containing error information and corrected question text samples corresponding to the question text samples. The step of inputting the word vectors and the byte-pair encoded text into the encoding layer of the question correction model to obtain the error attribute features of the question text to be corrected output by the encoding layer specifically includes: The byte-pair encoded text is input into the first type of convolutional neural network layer to obtain the position encoding features of the word segmented text corresponding to the word vector output by the first type of convolutional neural network layer; Based on the positional encoding features of the segmented text corresponding to the word vectors, the aggregation features corresponding to the word vectors are determined; The aggregated feature is input to the encoding end of the Transformer layer to obtain the error attribute feature corresponding to the aggregated feature output by the encoding end of the Transformer layer.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the question correction method according to any one of claims 1 to 6.

9. A processor-readable storage medium, characterized in that, The processor-readable storage medium stores a computer program for causing the processor to perform the steps of the question correction method according to any one of claims 1 to 6.

Images