Tabular Data Extraction via Deep Learning and OCR

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for extracting tabular data from medical documents are inefficient, requiring human intervention, are laborious, and struggle with unstructured documents and borderless tables, leading to inaccuracies and high computational costs.

Innovation Solution

An end-to-end system utilizing a deep learning-based model combined with statistical methods for table detection, type classification, and optical character recognition (OCR) to identify and extract tabular data from bordered, partially bordered, and borderless tables, preserving the integrity of table contents.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If traditional rule-based extraction approaches are used, then extraction accuracy can be maintained for structured documents, but the system requires constant manual rule updating and deep domain knowledge, making it difficult to adapt to different document formats

Engineering Contradiction:

Improveadaptability to different document formatsVSAvoidcomplexity of rule maintenance and updates

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent replaces manual rule-based extraction with an optical character recognition (OCR) system that automatically converts images of tables into editable text. This substitution eliminates the need for manual rule creation and updates, allowing the system to adapt to various document formats without requiring deep domain knowledge for rule maintenance

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The OCR system enables the extraction process to be self-service oriented, automatically handling table detection and text extraction without requiring human experts to continuously update rules. The system serves itself by adapting to different formats through automated image-to-text conversion rather than requiring manual intervention

Inventive Principle:

Principle #25Self-service

2Productivity

If optical character recognition is applied to tables, then extraction speed and automation improve, but the integrity of tabular content is compromised due to line-by-line reading

Engineering Contradiction:

Improveextraction speed and automation levelVSAvoidintegrity of tabular content

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent segments the table extraction process into distinct phases: first detecting table boundaries and structure in the image, then applying OCR specifically within those detected regions, and finally reconstructing the tabular data with preserved structure. This segmentation allows automated high-speed extraction while maintaining content integrity through structured reconstruction

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary table detection and boundary identification before applying OCR. By pre-processing to identify table regions, row boundaries, and column structures, the system ensures that subsequent OCR operations maintain the integrity of tabular content while benefiting from automated speed

Inventive Principle:

Principle #10Preliminary action

3Adaptability or versatility

If existing data extraction techniques are used, then simple bordered tables can be processed, but borderless and partially bordered tables cannot be identified, limiting the system's capability

Engineering Contradiction:

Improvecapability to handle various table typesVSAvoidaccuracy in identifying table boundaries

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent changes the detection parameters from relying on visible borders to analyzing spatial relationships between text elements. By detecting row and column boundaries through text position analysis rather than border detection, the system accurately identifies all table types including borderless tables, maintaining precision while expanding versatility

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS11887393B2End-to-end system for extracting tabular data present in electronic documents and method thereof

Publication Date: 2024.01.30 CLARITRICS INC D B A BUDDI AI

AI summary

The present disclosure describes a method, system, and a computer readable medium for extracting tabular data present in a document. The method comprises detecting presence of at least one table in the document using a deep learning based model and a statistical method. The method further comprises identifying a type of the table based on determining a count of horizontal and vertical lines, presence of outer borders, and presence of row-column intersections in the table. The type of the table comprises a bordered table, a partially bordered table, or a borderless table. The method further comprises processing the detected table, depending on its type, to identify one or more cells present in the table. The method further comprises generating an output file by extracting the tabular data present in the table, where the extracting comprises performing optical character recognition on the identified one or more cells.