Text Error Correction via Masked Fine-Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text error correction models struggle with generalization ability, failing to accurately correct errors that have not been seen during training due to differences in expressions and knowledge across various fields, leading to low accuracy on unseen errors.
Innovation Solution
The masked fine-tuning (MFT) method, which randomly adds a mask to the source text, requiring the model to learn context-based corrections and restore masked characters, thereby enhancing the model's ability to generalize across different fields without altering the raw data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the model is trained on existing text error correction data, then it can correct common errors effectively, but it fails to generalize to unseen errors in different fields
Solution Approach 1:
The patent applies preliminary action by performing masked fine-tuning during the training phase. The model is pre-exposed to masked tokens in training data, learning to predict missing text based on context. This preliminary learning of contextual relationships enables the model to generalize to unseen errors in different fields without requiring field-specific retraining.
Solution Approach 2:
The patent changes the training parameter by introducing masked tokens at random positions in the input text. This parameter modification transforms the standard text error correction training into a masked language modeling task, where the model learns to predict masked tokens using contextual information, thereby improving its ability to handle diverse error types across different fields.
2Measurement precision
If the model learns field-specific expressions and knowledge, then it achieves high accuracy in that field, but it cannot adapt to other fields with different expressions
Solution Approach 1:
The patent achieves universality by training the model on diverse training data from multiple fields with masked tokens. The model learns universal contextual relationships and language patterns that are applicable across different domains. This multi-functional training approach enables the single model to generalize effectively to unseen errors in various fields without requiring field-specific customization.
3Ease of manufacture
If traditional training methods are used, then the training process is simple, but the model performance on unseen errors is poor
Solution Approach 1:
The patent modifies the training parameter by introducing masked tokens into the training data. This change transforms the training objective from standard text classification to masked language modeling. The implementation maintains simplicity through random masking with a fixed probability (e.g., 15%), allowing the model to learn contextual relationships effectively while improving performance on unseen errors without complex training procedures.
Data Source
AI summary
Embodiments of the disclosure relate to a text error correction method and system, an electronic device, and a medium. The method includes: obtaining a sample label including a source and a target, and randomly adding a mask to the source in the sample label to obtain the source with the mask; inputting the source with the mask into a training model to obtain a prediction result; calculating a precision and a recall according to the source, the target, and the prediction result of the sample label; calculating an average precision and an average recall for the precisions and the recalls of a plurality of sample labels, and calculating a harmonic mean F1 of the precisions and the recalls according to the average precision and the average recall; and adjusting the training model according to the harmonic mean F1, and taking the adjusted training model as a text error correction model.