Chinese text automatic error correction method and device

Abstract

The invention provides a Chinese text automatic error correction method and device, and the method comprises the steps: carrying out the shallow error correction of a to-be-corrected text, and obtaining a first sentence sequence; performing deep neural network model correction on the first sentence sequence to obtain a fifth sentence sequence; post-processing the fifth sentence sequence to obtain a corrected sample; and outputting the correction sample and the error information. The device provided by the invention comprises a shallow error correction module, a deep neural network model correction module, a post-processing module and an integrated output module, the deep neural network model correction module is composed of an equal-length sequence error correction unit, a word redundancy error correction unit, a word missing error correction unit, a language model judgment unit and a three-model fusion unit, and the post-processing module is composed of a place name error detection unit and a sensitive word error detection unit. According to the method, automatic data set generation and deep neural network model correction can be realized, the Chinese error correction range is more comprehensive, and the error correction efficiency is higher.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a Chinese text automatic error correction method and device. Background technique [0002] Internet data dissemination is an important feature of the current era of big data. In the process of Internet data dissemination, there are a large number of electronic documents. The content quality of electronic documents will not only affect the reading experience of readers, but also affect the public influence of authors. Among them, the correct expression of Chinese is the key factor to improve the quality of article content, how to identify Chinese errors in electronic document data has become a necessary but time-consuming and laborious work. On the one hand, due to the variety of Chinese expressions, there are many types of errors. On the other hand, with the development of artificial intelligence technology, speech recognition and OCR recognition are likely to...

AI Technical Summary

Problems solved by technology

This method has the following disadvantages: 1) The construction of the confused word set requires a large amount of wrong data in real scenarios; 2) It also requires a lot of manual experience; 3) The confused word set depends on different data fields, and it is difficult to exhaust all errors Possibilities, so the ability to generalize to new scene data is very limited

[0005] The second idea is to apply the deep neural network model to Chinese error correction tasks and try Chinese error correction methods in different scenarios. The disadvantage is that the error correction scene is single, and there is still a large room for improvement in error correction accuracy.

Images