Dynamic OCR Data Extraction with Learning Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document imaging technologies face challenges in accurately extracting data from documents due to noise in OCR-extracted data, limited adaptability of pre-built templates, and reliance on OCR confidence values, leading to errors being propagated into databases without detection.
Innovation Solution
A system and method that uses a data extraction engine accepting extraction criteria and configuration parameters, adapts captured data with domain-specific rules and OCR error patterns, and employs a learning engine to generate data models for dynamic updates of extraction criteria, including the use of regular expressions, geometric markers, and anchor text markers, to improve data accuracy and detect OCR errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If pre-built fixed templates are used for data extraction, then initial data capture can be performed, but the system cannot adapt to document structure changes, font variations, layout modifications, or new characters over time
Solution Approach 1:
The patent implements dynamic template generation that automatically adapts to document variations. The system learns from historical extraction results and ground truth data to continuously update extraction templates, making them flexible rather than fixed. This allows the system to handle document structure changes, font variations, and layout modifications while maintaining high extraction accuracy.
Solution Approach 2:
The system performs self-learning and self-updating of extraction templates without requiring manual intervention. The automated learning engine analyzes extraction results and ground truth data to automatically refine templates, enabling the system to adapt to new document formats and variations independently over time.
2Measurement precision
If OCR confidence values are used to detect errors, then error detection can be performed, but the system cannot reliably identify misrecognized data due to proprietary and varying confidence computation methods across different OCR software
Solution Approach 1:
The patent introduces ground truth data as an intermediary reference to validate OCR extraction results. Instead of relying solely on proprietary confidence values, the system compares extracted data against ground truth and uses learning models trained on this comparison to identify errors. This intermediary mechanism provides a reliable, standardized basis for error detection that is independent of OCR software-specific confidence computations.
Solution Approach 2:
The system implements feedback loops where extraction results are continuously validated against ground truth data. The learning engine uses this feedback to refine extraction templates and error detection models, progressively improving the ability to identify misrecognized data regardless of the OCR software's confidence value methodology.
3Reliability
If manual template updating is performed based on previous errors and corrections, then extraction accuracy can be improved, but the process is tedious and time-consuming
Solution Approach 1:
The system automatically performs template updates by learning from historical extraction results and ground truth data. The learning engine continuously refines extraction templates without requiring manual intervention, eliminating the tedious manual updating process while maintaining and improving extraction accuracy over time.
Solution Approach 2:
The system performs preliminary learning and template refinement in the background using historical data. By pre-processing and learning from past extraction results and ground truth before actual extraction tasks, the system prepares optimized templates in advance, avoiding time-consuming manual updates during operational phases.
4Productivity
If threshold-based error detection is used, then simple error filtering can be performed, but errors are propagated into the enterprise database without explicit detection when the threshold is not met
Solution Approach 1:
The patent replaces simple threshold-based mechanical error filtering with intelligent learning models that analyze extraction results against ground truth data. These models dynamically identify errors based on learned patterns rather than fixed thresholds, ensuring comprehensive error detection while maintaining efficient processing throughput.
Solution Approach 2:
The system dynamically adjusts error detection parameters based on learned patterns from historical data and ground truth comparisons. Instead of using fixed thresholds, the learning engine adapts detection sensitivity and criteria according to the specific extraction context, document type, and observed error patterns, ensuring both efficiency and completeness in error detection.
Data Source
AI summary
A computer implemented a method and system for enrichment of OCR extracted data is disclosed comprising of accepting a set of extraction criteria and a set of configuration parameters by a data extraction engine. The data extraction engine captures data satisfying an extraction criteria using the configuration parameters and adapts the captured data using a set of domain specific rules and a set of OCR error patterns. A learning engine generates learning data models using the adapted data and the configuration parameters and the system dynamically updates the extraction criteria using the generated learning data models. The extraction criteria comprise one or more extraction templates wherein an extraction template includes one of a regular expression, geometric markers, anchor text markers and a combination thereof.