NLP Model Training via OCR and Entity Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current computer vision and process automation technologies are unable to effectively review and extract data from complex, data-rich electronic documents that contain a mixture of text, tables, images, and other content, leading to the need for manual review which is time-consuming, expensive, and prone to human error.
Innovation Solution
A computer-implemented method using a trained natural language processing model that extracts text data, determines candidate entity and context data, accesses n-gram words from a knowledge base, and calculates similarity scores to identify optimum entity data, enabling accurate review of complex data sets, including text, tables, and images, and allowing for iterative improvement through user feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual review is used for complex data-rich documents, then extraction accuracy can be maintained, but processing time and cost increase significantly
Solution Approach 1:
The patent introduces an intermediary system comprising multiple processing layers (OCR layer, NER layer, relation extraction layer, etc.) that mediate between the input document and final extracted entities. This intermediary architecture enables automated processing while maintaining high accuracy through progressive refinement at each layer, resolving the contradiction between automation speed and extraction precision.
Solution Approach 2:
The system implements feedback mechanisms where extraction results from each layer are refined based on results from subsequent layers. The relation extraction layer provides feedback to refine entity identification, and confidence scoring provides feedback for iterative improvement, enabling automated processing to achieve manual-level accuracy without sacrificing processing speed.
2Productivity
If existing computer vision technology is used, then simple document processing is efficient, but complex data-rich documents cannot be reviewed
Solution Approach 1:
The patent creates a universal processing framework that handles multiple document types and formats (text, tables, images, charts) through a single multi-functional system. The architecture integrates OCR, NLP, computer vision, and knowledge graph technologies to process diverse content types uniformly, enabling the system to adapt to complex data-rich documents while maintaining processing efficiency.
Solution Approach 2:
The system combines multiple technological components (OCR engines, NLP models, computer vision algorithms, knowledge graphs) into a composite processing architecture. This composite approach integrates the strengths of different technologies to handle complex documents containing mixed content types, achieving both efficiency and versatility.
3Productivity
If automated processing is implemented, then processing speed increases, but accuracy and reliability decrease
Solution Approach 1:
The patent segments the extraction process into multiple independent layers (OCR, entity recognition, relation extraction, event extraction), each specializing in specific tasks. This segmentation allows each component to be optimized for its specific function while working together to maintain overall reliability, enabling fast automated processing without sacrificing accuracy.
Solution Approach 2:
The system performs preliminary actions by pre-processing documents through OCR and initial entity recognition before main extraction. Confidence scoring and validation are performed preliminarily to filter low-quality results, ensuring that only high-reliability extractions proceed to final output, maintaining reliability while enabling automated speed.
Data Source
AI summary
A data processing system receives a plurality of electronic documents in image format. The system extracts text from the electronic documents using an optical character recognition processor, and determines a plurality of entity data based on the extracted text. The system receives pre-defined question data from a user, and determines pre-annotated answer data based on the entity data and the question data using an open-domain question answering model. The system determines context data based on the entity data and the question data. The system provides the pre-annotated answer data to the user, and receives corrected entity data from the user. The system trains a closed-domain question answering model based on the corrected entity data and re-aligned context data. The system determines a plurality of n-gram words based on the corrected entity data and the context data using a context phrase model. The n-grams words are stored in a knowledge base.