Probabilistic Table Parsing via Separator Segmentation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current methods for extracting information from tables in documents, such as invoices and healthcare forms, are inefficient and require significant manual effort, as simple Optical Character Recognition (OCR) cannot recover the table structure, leading to high operating costs for businesses processing large volumes of documents.

Innovation Solution

A method and system for parsing tables that involves receiving an input image, finding candidate separators, and determining real versus spurious separators by optimizing an objective function that considers quality and coherence of cells, rows, columns, and layout consistency, using perceptually motivated cues and probabilistic models to accurately identify and extract table structure.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of information

If simple Optical Character Recognition (OCR) is used to extract text from tables, then text extraction is achieved, but the table structure is not preserved

Engineering Contradiction:

Improvetable structure informationVSAvoidstructure recognition accuracy

Core Design Contradiction:

Loss of informationVSMeasurement precision

Solution Approach 1:

The patent segments the table image into multiple candidate separators (horizontal and vertical lines) and individually evaluates each separator's likelihood of being real versus spurious. This segmentation allows the system to separately identify and remove spurious separators while preserving real structural separators, thereby maintaining table structure information that simple OCR would lose.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces an intermediary probabilistic model that evaluates candidate separators. This intermediary layer between OCR text extraction and final table structure reconstruction allows the system to filter out spurious separators and preserve genuine structural information, bridging the gap between raw text extraction and structured data recovery.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If manual extraction methods are used to preserve table structure, then structure accuracy is improved, but processing cost and time increase significantly

Engineering Contradiction:

Improvestructure extraction accuracyVSAvoiddocument processing throughput

Core Design Contradiction:

Measurement precisionVSProductivity

Solution Approach 1:

The patent implements a self-service automated system that uses probabilistic models to independently identify and remove spurious separators without human intervention. The system processes tables automatically by evaluating candidate separators and making decisions based on computed probabilities, achieving both high accuracy and high throughput without manual extraction.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical manual extraction process with an automated computational system. Instead of human operators manually identifying table structures, the system uses probabilistic modeling and automated algorithms to detect and remove spurious separators, substituting human labor with computational processing that is both accurate and scalable.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Loss of information

If all candidate separators are retained to preserve potential structure information, then no structure information is lost, but spurious separators reduce parse accuracy

Engineering Contradiction:

Improvepotential structure informationVSAvoidparse accuracy

Core Design Contradiction:

Loss of informationVSReliability

Solution Approach 1:

The patent changes the parameter evaluation by computing a likelihood score for each candidate separator being real versus spurious. By transforming the binary presence/absence of separators into a probabilistic parameter space, the system can selectively retain separators with high likelihood scores while removing those with low scores, balancing information preservation with accuracy.

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent employs feedback through the probabilistic evaluation process, where the likelihood scores of candidate separators inform the decision of which separators to retain. The system uses the computed probabilities as feedback to iteratively refine the separator selection, ensuring that only separators contributing positively to parse accuracy are retained while filtering out spurious ones.

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS9047533B2Parsing tables by probabilistic modeling of perceptual cues

Publication Date: 2015.06.02 XEROX CORP

AI summary

A method is provided for parsing a table. The method includes: receiving an input containing the table; finding candidate separators within the table; and determining which candidate separators are at least one of real and spurious by optimizing an objective function over the set of found candidate separators. Suitably, the function measures numerically whether a parse produced by the set of real separators is accurate. The function suitably includes one or more terms that account for multiple aspects of the table including at least two of: quality of candidate separators; coherence of cells within the parse; quality of cells within the parse; coherence of entire rows within the parse; quality of entire rows within the parse; coherence of entire columns within the parse; quality of entire columns within the parse; layout consistency along an axis of the table; and repeatability along the axis of the table.