Grid-Less Table Processing with Deep Learning Grid Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing table processing tools, particularly for PDF documents, struggle with accurately parsing grid-less tables and extracting data from merged cells, leading to inefficient manual extraction and loss of data hierarchy.
Innovation Solution
A method involving a deep learning model (DLM) for bounding box detection and a conditional generative adversarial network (cGAN) for grid generation, which processes grid-less tables by training on labeled documents to automatically detect and generate grid structures, maintaining data hierarchy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing parsing tools are used for grid-less tables, then manual extraction is required, but this increases time consumption and labor effort
Solution Approach 1:
The system enables automatic detection and extraction of grid-less tables through machine learning models that self-identify table boundaries, grid structures, and cell contents without requiring manual analyst intervention for each table extraction task
Solution Approach 2:
Manual mechanical table extraction processes are replaced with automated computer vision and deep learning systems that use neural networks to detect tables, generate grid structures, and extract data programmatically
2Loss of information
If existing tools process grid-less tables, then data hierarchy is lost, but implementing proper detection increases system complexity
Solution Approach 1:
The table processing system segments the document into distinct table regions, identifies grid structures within those regions, and extracts cells while preserving the hierarchical relationship between tables, rows, and columns through structured data output
Solution Approach 2:
A grid generation model acts as an intermediary between the detected table region and the final extracted data, creating virtual grid structures that organize cell contents while maintaining the underlying data hierarchy without requiring physical grid lines in the source document
3Productivity
If manual selection and extraction is performed, then accuracy can be maintained, but productivity decreases
Solution Approach 1:
The system incorporates feedback mechanisms where the detection model and grid generation model are trained on labeled data and continuously improved through evaluation metrics that measure detection accuracy, grid alignment, and extraction precision to maintain high accuracy while scaling throughput
4Extent of automation
If existing tools are used for semi-structured documents, then automatic detection fails, but manual processing increases time consumption
Solution Approach 1:
The detection system uses parameter changes in the document structure, such as variations in spacing, font patterns, and layout characteristics, to automatically identify table regions in semi-structured documents without requiring manual specification or predefined templates
Data Source
AI summary
In a first aspect, the invention pertains to a computer-implemented method for improved grid-less table processing. In a second and third aspect, the invention pertains to a computer system and a computer program product for improved grid-less table processing. In a fourth aspect, the invention pertains use of any of the method, system or product for document parsing.