Document Image Text Extraction with Overlapping Entity Segmentation

Solution Overview

Problem

Existing techniques for extracting character strings from document images face challenges when multiple item values overlap, leading to increased processing costs and reduced extraction accuracy, particularly in cases where character strings corresponding to different items are intertwined, such as company names and dates within titles.

Innovation Solution

An information processing apparatus and method that employs a trained model to first extract character strings for multiple items from a document image and then performs re-extraction on any missed items using a dedicated re-extraction model or rule-based key-value extraction, allowing for accurate classification of overlapping character string ranges.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If multilabel classification is applied to handle N items, then the extraction capability for overlapping items is improved, but the processing cost increases N times and extraction accuracy decreases

Engineering Contradiction:

Improveextraction capabilityVSAvoidprocessing cost

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent segments the extraction process into two distinct phases: first extracting character strings for all items using a training model, then performing re-extraction only on items that were not successfully extracted in the first phase. This segmentation avoids applying multilabel classification to all items simultaneously, thereby reducing processing cost while maintaining extraction capability for overlapping items.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of performing full multilabel classification on all N items (excessive action), the patent applies re-extraction only to the subset of items that were not extracted in the first phase (partial action). This reduces the processing burden from N times to a fraction of N, while still ensuring accurate extraction of overlapping items.

Inventive Principle:

Principle #16Partial or excessive action

2Adaptability or versatility

If multilabel classification is applied to handle N items, then the extraction capability for overlapping items is improved, but the extraction accuracy is reduced

Engineering Contradiction:

Improveextraction capabilityVSAvoidextraction accuracy

Core Design Contradiction:

Adaptability or versatilityVSMeasurement precision

Solution Approach 1:

The patent segments the extraction process into two distinct phases: first extracting character strings for all items using a training model, then performing re-extraction only on items that were not successfully extracted in the first phase. This segmentation avoids applying multilabel classification to all N items simultaneously, thereby reducing processing cost while maintaining extraction capability for overlapping items.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of performing full multilabel classification on all N items (excessive action), the patent applies re-extraction only to the subset of items that were not extracted in the first phase (partial action). This reduces the processing burden from N times to a fraction of N, while still ensuring accurate extraction of overlapping items.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If single label classification is used, then the processing cost is reduced, but the extraction capability for overlapping items is insufficient

Engineering Contradiction:

Improveprocessing costVSAvoidextraction capability

Core Design Contradiction:

ProductivityVSAdaptability or versatility

Solution Approach 1:

The patent segments the extraction process into two distinct phases: first extracting character strings for all items using a training model, then performing re-extraction only on items that were not successfully extracted in the first phase. This segmentation avoids applying multilabel classification to all N items simultaneously, thereby reducing processing cost while maintaining extraction capability for overlapping items.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of performing full multilabel classification on all N items (excessive action), the patent applies re-extraction only to the subset of items that were not extracted in the first phase (partial action). This reduces the processing burden from N times to a fraction of N, while still ensuring accurate extraction of overlapping items.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS20240193370A1Information processing apparatus, information processing system, information processing method, and storage medium

Publication Date: 2024.06.13 CANON KK

AI summary

To make it possible to extract a character string corresponding to each extraction-target item with accuracy even in a case where the character string ranges of a plurality of extraction-target items overlap one another in the task of named entity recognition. By using a training model trained to extract a character string corresponding to each of a plurality of items within a document, a character string corresponding to each of the plurality of items is extracted and output for an input document image. Then, a character string corresponding to an item among the plurality of items, for which a corresponding character string is not extracted, is re-extracted from the character string output by the first extracting.