Method and device for enhancing grammar error correction data based on real error mode

Abstract

The invention discloses a method and a device for enhancing grammar error correction data based on a real error pattern. The method comprises the following steps: acquiring a to-be-noise-added statement and a noise adding strategy set; determining the noise adding probability of each word in the statement to be subjected to noise adding; randomly selecting a noise adding strategy from a noise adding strategy set according to the noise adding probability to carry out noise adding processing on the to-be-noise-added word; and constructing parallel statement pairs according to the error statements subjected to noise addition processing and the correct statements before noise addition processing. The noise adding strategy set comprises a real error pattern-based replacement strategy, a synonym replacement strategy, a function word replacement strategy, a similar spelling replacement strategy and a flexion replacement strategy. According to the embodiment of the invention, through introduction of real errors and simulation of various real errors, high-quality artificial error enhancement data which is more real and closer to real errors of learners can be generated; and various grammar errors can be manufactured through various types of noise schemes, and the method and the device can be widely applied to the technical field of data processing.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a syntax error correction data enhancement method and device based on a real error mode. Background technique [0002] Grammatical error correction refers to the process of grammatical detection and error correction for sentences with grammatical errors and generating natural, fluent and grammatically correct sentences. In recent years, grammatical error correction models based on neural networks have achieved good results, but these models usually require large-scale labeled parallel training data. In actual scenarios, due to the high cost of manually labeling large-scale massive data, researchers try to Another perspective is to improve the performance of the error correction model, that is, through the data enhancement method, noise is introduced into the correct sentence to obtain pseudo-labeled data with errors, as a supplement to the manually labeled data, thereby i...

AI Technical Summary

Problems solved by technology

[0004] Existing data enhancement technologies mainly include methods based on editing operations, methods based on machine translation and back-translation, etc. Data enhancement methods based on editing operations use editing operations such as randomly selecting words from the dictionary for replacement or insertion, random deletion, and random position exchange. , although a large amount of artificial error data can be generated, the generated errors include error data that will not appear to learners. This unreal and low-quality error data will further propagate errors into the grammatical error correction model, thereby reducing the accuracy of the model. overall performance

Although back-translation-based methods can achieve better results, they still require large-scale labeled training data

Images