Entity Mapping for OCR Text Correction and Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems inaccurately digitize physical documents, especially for small, rotated, or distorted text, leading to incorrect entity extraction and recognition, which results in redundant entities and missed entities, posing challenges in identifying duplicates.
Innovation Solution
An entity mapping system that utilizes named entity recognition and tokenization to correct errors by comparing extracted entities with tokenized character strings, determining similarity metrics, and generating mappings to resolve entities accurately, even with digitization errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If conventional optical character recognition processes are used to digitize physical documents, then the digitization process can be automated, but the accuracy of text extraction deteriorates for small, rotated, or distorted text
Solution Approach 1:
The system segments the digitization process into multiple stages: initial OCR extraction, entity detection, error identification, and correction. By dividing the process into discrete steps, the system can apply specialized processing at each stage to maintain automation while improving overall accuracy.
Solution Approach 2:
The system implements feedback mechanisms where extracted entities are validated against the original document image, and detected errors are used to refine subsequent extraction attempts. This closed-loop approach allows automated correction of OCR mistakes while maintaining high-speed processing.
2Productivity
If conventional OCR processes are used without correction, then processing speed is maintained, but entity extraction accuracy deteriorates leading to redundant and missed entities
Solution Approach 1:
The system performs preliminary entity detection and error identification before final entity extraction. By anticipating and correcting potential errors in advance, the system avoids redundant processing steps while maintaining high accuracy in the final output.
Solution Approach 2:
The system introduces an intermediary correction layer between OCR and entity extraction. This intermediate processing stage identifies and corrects errors without requiring complete re-processing of the document, thus maintaining processing speed while improving accuracy.
3Measurement precision
If entity correction and resolution processes are added to the digitization system, then entity extraction accuracy is improved, but system complexity increases
Solution Approach 1:
The system extracts only the critical correction and resolution functions from the overall digitization process, separating them into modular components. This allows the core OCR functionality to remain simple while adding sophisticated correction capabilities where needed without overwhelming system complexity.
Solution Approach 2:
The entity correction system uses self-service mechanisms where the extracted text automatically identifies and corrects its own errors through validation against the source document and application of correction rules, reducing the need for complex external processing.
4Measurement precision
If meticulous entity verification is performed on all extracted entities, then accuracy of entity identification is improved, but processing time increases
Solution Approach 1:
The system applies partial verification to high-confidence entity extractions while performing more thorough verification only on low-confidence or ambiguous cases. This selective approach maintains high accuracy for critical entities while avoiding unnecessary processing time for obvious matches.
Data Source
AI summary
Methods, systems, and non-transitory computer readable storage media are disclosed for correcting entity detection errors with entity correction and resolution in optical character recognition for digitization of physical documents. Specifically, the disclosed system utilizes named entity recognition to extract entities from character strings (e.g., words) in a digital text document. The disclosed system also tokenizes the character strings in the digital text document based on attributes of the character strings. Furthermore, the disclosed system compares the extracted entities and tokenized character strings to determine similarity metrics between the extracted entities and tokenized character strings. The disclosed system also compares extracted entities to character strings including special/numerical characters to determine similarity metrics indicating correlation probabilities between entities and character strings. The disclosed systems generate mappings between the tokens and entities based on the similarity metrics to resolve entities to likely corresponding character strings while correcting for errors during entity extraction.