English grammar error correction method based on CNN and BERT model

Abstract

English grammar error correction is an important research direction in the field of natural language processing. A traditional grammar error correction system is mostly based on rule judgment, the types of detected errors are limited, and the expansion capacity is poor. An existing syntax error correction system based on a recurrent neural network is prone to losing head and tail information whenfacing long sentences, and due to the fact that characteristics cannot be extracted in parallel, the training period is long. The invention provides the English grammar error correction method based on the CNN and the BERT model. A CNN+Attention+BERT structure is adopted in the model, and an Encoder-Decoder framework is adopted in the implementation mode. Through convolution, the features of the context can be efficiently and accurately extracted; the Attention layer adds weights to different words, so that the model can learn more important features; the BERT adopts a Masked Lange Model modeto train a language model, (0, 1) classification tasks can be added to the language model through fin-tuning, the classification tasks are used for scoring sentences output by an error correction system, and the accuracy of the system is improved.

Description

technical field [0001] The present invention is a task in the field of natural language processing, mainly an English grammar error correction method based on CNN (Convolutional Neural Networks, Convolutional Neural Networks) and BERT (Bidirectional Encoder Representations from Transformers) models. Background technique [0002] For domestic English learners, grammar is often a difficult point in their English learning. Due to limited teaching resources, learners often cannot get correct correction opinions and examples in time for grammar problems encountered by learners. If there is a grammatical error correction system that enables learners to point out grammatical errors in the process of learning and using English in a timely manner, and give corresponding feedback and suggestions, it can greatly reduce the difficulty of learning for learners , and significantly improve their English proficiency. Most of the traditional English error correction systems are purely based...

AI Technical Summary

Problems solved by technology

Most of the traditional English error correction systems are purely based on the principle of statistical machine translation or based on certain grammatical rules. The accuracy is not high, the efficiency of training models is low, and the types of grammatical errors that can be detected are limited.

Images