Document Data Extraction Using Segmentation and NER Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for extracting data from template-based documents face challenges due to the inability of natural language processing (NLP) to understand structural layout and computer vision's difficulty in recognizing the true meaning of segmented portions, leading to inaccurate data extraction.
Innovation Solution
A system employing a computer vision-based image segmentation deep learning model and an NLP-based named entity recognition (NER) deep learning model to segment and classify document elements, followed by OCR text extraction and NER processing, combining both techniques for accurate data extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If NLP is used to scan and recognize document content, then text recognition is performed, but the structural layout information cannot be understood or leveraged
Solution Approach 1:
The patent segments the document processing into two distinct stages: first using computer vision to segment the document into discrete portions based on visual appearance, then using NLP to process each segment. This segmentation allows structural layout information to be preserved and utilized while performing text recognition.
Solution Approach 2:
The patent introduces computer vision as an intermediary between the document and NLP processing. The computer vision component extracts visual features and structural information, which then serve as input for NLP processing, enabling both layout understanding and text recognition.
2Measurement precision
If computer vision is used to segment document into discrete portions, then visual appearance is utilized, but the true meaning of each segmented portion cannot be understood
Solution Approach 1:
The patent merges computer vision and NLP techniques into a unified processing pipeline. Computer vision segments and classifies document portions based on visual appearance, while NLP processes the same portions to extract semantic meaning. The combination of both approaches enables accurate classification and understanding of segmented portions.
3Shape
If computer vision segments document based on visual appearance, then layout structure is captured, but similar-looking fields cannot be distinguished
Solution Approach 1:
The patent uses NLP as an intermediary to disambiguate visually similar fields. After computer vision segments and classifies portions based on visual appearance, NLP processes the text content to provide semantic context that distinguishes between similar-looking fields such as invoice number versus invoice date.
Data Source
AI summary
Techniques for automatically extracting data from documents using multiple deep learning models are provided. According to one set of embodiments, a computer system can receive a document in an electronic format and can segment, using an image segmentation deep learning model, the document into a plurality of segments, where each segment corresponds to a visually discrete portion of the document and is classified as being one of a plurality of types. The computer system can then, for each segment in the plurality of segments, retrieve text in the segment using optical character recognition (OCR) and extract data in the segment from the retrieved text using a named entity recognition (NER) deep learning model, where the retrieving and the extracting are performed in a manner that takes into account the segment's type.