Typographical Error Correction Using Contextual Linguistic Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional text error correction programs based on language rules require significant human effort to summarize language rules, leading to low error recall and accuracy due to the complex structure of language and conflicts between rules.
Innovation Solution
A computer-implemented method that selects a target word in a sentence, acquires preceding and succeeding sequences of words, searches for candidate words from a database that separate these sequences, and determines the fittest replacement based on context information and similarity, using a linguistic model for error correction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If language rules are summarized preliminarily to correct text errors, then error correction can be performed automatically, but the error recall rate and accuracy remain low due to rule conflicts and complexity
Solution Approach 1:
The patent introduces a probabilistic context-free grammar (PCFG) model as an intermediary between the text to be corrected and the correction rules. Instead of directly applying conflicting language rules, the PCFG model evaluates candidate corrections based on statistical probabilities derived from training data, mediating the selection process to resolve rule conflicts and improve accuracy while maintaining automation.
2Extent of automation
If language rules are summarized preliminarily to correct text errors, then error correction can be performed automatically, but significant human effort is required to summarize the rules
Solution Approach 1:
The patent performs preliminary action by pre-training the PCFG model on a large corpus of text data before actual error correction tasks. During this offline training phase, the model learns statistical patterns and probabilities of language structures. When actual correction is needed, the pre-trained model can automatically evaluate candidates without requiring real-time human intervention for rule summarization, thus reducing time loss while maintaining automation.
3Measurement precision
If complex language rules are used to improve error correction accuracy, then more conflicts between rules occur, making the system harder to maintain
Solution Approach 1:
The patent transforms the complex system of conflicting qualitative language rules into a parameterized probabilistic model. Instead of managing multiple discrete rules with conflicts, the PCFG model uses statistical parameters (probabilities) to represent language patterns. This parameterization simplifies the system structure while maintaining or improving accuracy, as the model automatically handles what would otherwise be complex rule interactions through probability calculations.
Data Source
AI summary
A method is performed at a computer for automatically correcting typographical errors. The computer selects a target word in a target sentence and identifies a target word therein as having a typographical error and first and second sequences of words separated by the target word as context. After identifying, among a database of grammatically correct sentences, a set of sentences having the first and second sequences of words, each sentence including a replacement word, the computer selects a set of candidate grammatically correct sentences whose corresponding replacement words have similarities to the target word above a pre-set threshold, Finally, the computer chooses, among the set of candidate grammatically correct sentences, a fittest grammatically correct sentence according to a linguistic model and replaces the target word in the target sentence with the replacement word within the fittest grammatically correct sentence.