Character-Level NER for OCR Error-Resilient Document Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing named entity recognition (NER) methods are ineffective for text streams generated from images due to character-level translation errors introduced by optical character recognition (OCR), leading to incorrect entity expansion and inability to extract structured data from documents like receipts and tax forms.
Innovation Solution
A character-level bi-directional long short-term memory conditional random field (BiLSTM-CRF) model is employed for named entity recognition, utilizing character-level features and heuristic clues to train without high-quality labeled data, leveraging embeddings and spatial features to accurately identify entities despite OCR errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing named entity recognition methods are used on OCR-generated text, then the process is simple, but character-level translation errors cause incorrect entity expansion and inability to extract structured data
Solution Approach 1:
The patent segments the text processing into character-level features instead of treating entire words as units. By processing text at the character level, the model can identify and correct OCR errors within individual characters, preventing incorrect entity expansion while maintaining processing efficiency through the segmented BiLSTM-CRF architecture.
Solution Approach 2:
The patent introduces a new dimension of analysis by adding character-level features to the traditional word-level NER approach. This dimensional change allows the model to operate simultaneously at multiple granularities, using character-level information to correct OCR errors while preserving the contextual understanding provided by word-level processing.
2Reliability
If character-level features are used to improve entity recognition accuracy, then OCR error impact is reduced, but computational complexity increases
Solution Approach 1:
The patent performs preliminary action by pre-processing text at the character level before entity recognition. The BiLSTM-CRF model is trained in advance with character-level features to learn patterns of OCR errors and correct them proactively, rather than attempting to correct errors after entity recognition has already occurred. This preliminary processing improves reliability while managing computational resources efficiently.
Solution Approach 2:
The patent substitutes traditional mechanical NER approaches that rely on rule-based methods and manual error correction with a neural network-based BiLSTM-CRF system. This substitution automates the error correction process, improving reliability through learned patterns while the model's efficient architecture manages computational resource consumption.
3Productivity
If traditional NER methods are used, then processing speed is maintained, but character-level OCR errors lead to incorrect entity expansion
Solution Approach 1:
The patent maintains continuity of useful action by ensuring that character-level feature processing does not interrupt the overall NER workflow. The BiLSTM-CRF model processes character-level features in a continuous manner that integrates seamlessly with word-level processing, maintaining processing throughput while continuously correcting OCR errors to improve entity boundary accuracy.
Solution Approach 2:
The patent introduces character-level features as an intermediary between raw OCR text and final entity recognition. This intermediary layer processes and corrects OCR errors before they propagate to the entity recognition stage, acting as a mediator that preserves processing speed while improving entity boundary accuracy through intermediate error correction.
Data Source
Figure 1
Figure 2
Figure 3~5
AI summary
A system and method for information extraction character level features. The system and method may be used for data extraction for various types of content including a receipt or a tax form.