BERT and feedforward neural network-based text error correction method

Abstract

The invention discloses a BERT and feedforward neural network-based text error correction method, which can be used for quickly and accurately identifying and correcting errors of large-scale corpora. The method comprises the following steps: firstly, preprocessing a text, then carrying out semantic coding on the text by using BERT, then judging whether the text is correct or not by using semantic information of the whole text, then finding out a specific position of an error in the text which is judged to be wrong by using a sequence labeling method, and finally, combining context information of the error to determine whether the error occurs in the text; and generating a corresponding correct text by using the feedforward neural network. The text error correction method constructed by the invention has the characteristics of high reasoning speed and good interpretability.

Description

technical field [0001] The invention belongs to the fields of artificial intelligence and natural language processing, and in particular relates to a text error correction method based on BERT and a feedforward neural network. [0002] technical background [0003] Text error correction is a natural language processing technology that corrects erroneous content in texts, specifically including spelling error correction, grammatical error correction, and semantic-pragmatic error correction in specific scenarios. Among them, spelling error correction is characterized by not changing the length of the text, but only one-to-one correction of typos in the text; grammatical error correction and semantic pragmatic error correction need to deal with multiple word errors, few word errors, and word usage in the text. Mistakes, such as mistakes and wrong word order, may change the length of the text. [0004] In recent years, large-scale deep pre-trained language models such as BERT ha...

AI Technical Summary

Problems solved by technology

[0005] Traditional text error correction mainly adopts the method based on rules or translation models. The rule-based method mainly relies on manually defining the replacement word dictionary, and can only correct several specific errors; using translation models to correct text errors is the current mainstream method , and the neural network-based translation model has replaced the statistical-based translation model for error correction. This method treats text error correction as a translation problem from wrong sentences to correct sentences. Although the effect is good and the sentences are fluent, it requires a lot of Training data, and there is a time-consuming problem when using it

In addition, if only spelling errors are corrected, the current sequence labeling method is mainly used, which can quickly correct typos, but it is not suitable for other errors

Images