Text error correction method based on XLNet-BiGRU

Abstract

The invention provides a text error correction method based on an XLNet-BiGRU, and the method is characterized in that the method comprises the following steps: S1, training an XLNet (Generalized Automation Network for Language Understanding) Chinese model based on a large-scale unlabeled corpus, wherein the XLNet model mainly comprises a permutation language model (Permutation Language Model), a double-flow attention mechanism (Twood-Stream Self-Attention), and a Transform-XL core component, S2, preprocessing and labeling the text error correction corpus data; and S3, on the basis of the XLNet pre-training Chinese model trained in the S1, constructing an XLNet-BiGRU neural network model, wherein the model is mainly composed of a detection network and an error correction network, and the model is trained by using the marked data in the S2. According to the method, the problem that a traditional error correction method based on a translation model is long in time consumption is solved, and text error correction is optimized into a parallel process that error correction is carried out only aiming at error contents by using the XLNet neural network from a string travel of generating correct sentences one by one.

Description

technical field [0001] The invention relates to the fields of artificial intelligence and natural language processing, in particular to an XLNet-BiGRU text error correction method. Background technique [0002] Text error correction is a natural language processing technology that corrects erroneous content in text, specifically including spelling error correction, grammatical error correction, and semantic-pragmatic error correction in specific scenarios. Among them, spelling error correction is characterized by not changing the length of the text, but only one-to-one correction of typos in the text; grammatical error correction and semantic pragmatic error correction need to deal with multiple word errors, few word errors, and word usage in the text. Mistakes, such as mistakes and wrong word order, may change the length of the text. [0003] In recent years, large-scale deep pre-trained language models such as BERT and XLNet have promoted the rapid development of the fiel...

AI Technical Summary

Problems solved by technology

[0004] Traditional text error correction mainly adopts rule-based or translation model-based methods. The rule-based method mainly relies on manual definitions of replacement word dictionaries, and can only correct certain types of errors; using translation models for text error correction is currently the mainstream method , and the neural network-based translation model has replaced the statistical-based translation model for error correction. This method treats text error correction as a translation problem from wrong sentences to correct sentences. Although the effect is good and the sentences are fluent, it requires a lot of Training data, and there is a time-consuming problem when using it

In addition, if only spelling errors are corrected, the current sequence labeling method is mainly used, which can quickly correct typos, but it is not suitable for other errors

Images