Text error correction method based on XLNet-BiGRU

A text error correction, Chinese technology, applied in the field of XLNet-BiGRU text error correction, can solve the problems of inapplicability, time-consuming, etc., and achieve the effect of improving time-consuming

CN114064856APending Publication Date: 2022-02-18JIANGSU FUTURE NETWORKS INNOVATION
0 Cites 3 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
Publication Date
2022-02-18

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention provides a text error correction method based on an XLNet-BiGRU, and the method is characterized in that the method comprises the following steps: S1, training an XLNet (Generalized Automation Network for Language Understanding) Chinese model based on a large-scale unlabeled corpus, wherein the XLNet model mainly comprises a permutation language model (Permutation Language Model), a double-flow attention mechanism (Twood-Stream Self-Attention), and a Transform-XL core component, S2, preprocessing and labeling the text error correction corpus data; and S3, on the basis of the XLNet pre-training Chinese model trained in the S1, constructing an XLNet-BiGRU neural network model, wherein the model is mainly composed of a detection network and an error correction network, and the model is trained by using the marked data in the S2. According to the method, the problem that a traditional error correction method based on a translation model is long in time consumption is solved, and text error correction is optimized into a parallel process that error correction is carried out only aiming at error contents by using the XLNet neural network from a string travel of generating correct sentences one by one.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the fields of artificial intelligence and natural language processing, in particular to an XLNet-BiGRU text error correction method. Background technique

[0002] Text error correction is a natural language processing technology that corrects erroneous content in text, specifically including spelling error correction, grammatical error correction, and semantic-pragmatic error correction in specific scenarios. Among them, spelling error correction is characterized by not changing the length of the text, but only one-to-one correction of typos in the text; grammatical error correction and semantic pragmatic error correction need to deal with multiple word errors, few word errors, and word usage in the text. Mistakes, such as mistakes and wrong word order, may change the length of the text.

[0003] In recent years, large-scale deep pre-trained language models such as BERT and XLNet have promoted the rapid development of the fiel...

Examples

Embodiment 1

[0049] Such as figure 1 Shown, a kind of text error correction method based on XLNet-BiGRU of the present invention comprises following steps:

[0050] S1. Training XLNet (Generalized Autoregressive Pretraining for Language Understanding) Chinese model based on large-scale unlabeled corpus.

[0051] The XLNet model mainly includes Permutation Language Model, Two-Stream Self-Attention and Transformer-XL core components.

[0052] Further, the permutation language model included in the XLNet model aims to randomly shuffle the Chinese characters in the sentences in the text. For the Chinese character x i , the Chinese {x that originally appeared behind it i+1 ,...,x n} can also appear in front of it, assuming that the text sequence of length T is [1,2,...,T] and all permutations are A T , a t is the tth element in the sequence, and aT The previous element of , this modeling process can be expressed as:

[0053]

[0054] where θ is the model parameter with training.

[00...