Extracting Attributes from Embedded Tables in Documents

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current natural language processing (NLP) systems fail to extract information from embedded table structures in documents, which are crucial for decision-making in medical literature, clinical studies, and other types of documents, as they ignore these structures when processing unstructured text.

Innovation Solution

A computer-implemented method identifies tables in documents, locates co-occurring attributes to determine headers, and annotates values with corresponding header information, enabling the extraction of attributes from embedded table structures within documents.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If current NLP systems process only unstructured text, then the processing simplicity is maintained, but information extraction completeness deteriorates due to ignoring embedded table structures

Engineering Contradiction:

Improveinformation extraction completenessVSAvoidNLP system complexity

Core Design Contradiction:

Loss of informationVSDevice complexity

Solution Approach 1:

The patent segments the document processing into distinct components: unstructured text processing and structured table processing. Tables are identified as separate entities with their own header detection and value extraction mechanisms, allowing each component to be optimized independently while working together to provide complete information extraction.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary layer that converts table structures into a format compatible with NLP processing. By transforming tables into annotated structures with header-value relationships, the system enables unified processing of both unstructured text and structured data without requiring complete redesign of the NLP pipeline.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Quantity of substance

If tables are added as information source, then data richness improves, but processing difficulty increases due to mixed structured and unstructured content

Engineering Contradiction:

Improvedata richnessVSAvoidprocessing difficulty

Core Design Contradiction:

Quantity of substanceVSDifficulty of detecting and measuring

Solution Approach 1:

The patent applies local quality by detecting and treating table regions differently from text regions. Within table structures, header rows are identified and annotated separately from data rows, allowing the system to extract and process structured information with appropriate methods while maintaining overall document processing capability.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent performs preliminary actions by pre-identifying table structures and annotating headers before the main processing occurs. This preparation step marks table regions and establishes relationships between headers and values, simplifying subsequent processing and enabling the system to handle mixed content more effectively.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11270065B2Extracting attributes from embedded table structures

Publication Date: 2022.03.08 MERATIVE US LP

AI summary

Embodiments include methods, system and computer program products for extracting attributes from embedded table structures in a document. Aspects include identifying a table in the document and identifying one or more headers of the table by locating co-occurring attributes in the table. Aspects also include identifying a plurality of values in the table and creating an annotation for each of the plurality of values value in the table, wherein each annotation includes text extracted from the one or more headers that correspond to the location of the value in the table.