Document Hierarchy Generation via OCR and ML Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital documents generated from scanning in-print publications lack metadata indicating document hierarchies, preventing the creation of linked table-of-contents, which is time-consuming and user-input intensive.
Innovation Solution
Perform optical character recognition (OCR) on scanned documents to generate text data, detect bounded regions using predefined boundary rules, match text to regions, extract features, and use a trained machine learning classifier to generate metadata for a hierarchical layout, such as a table-of-contents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual metadata generation is used for scanned documents, then document hierarchy accuracy is improved, but user input intensity and time consumption increase
Solution Approach 1:
The system performs preliminary OCR processing and text extraction from scanned documents before hierarchy generation. By pre-processing the document content and organizing text blocks, the system prepares structured data that facilitates automated hierarchy determination, reducing both user input requirements and processing time while maintaining accuracy
Solution Approach 2:
The patent introduces an intermediary machine learning classifier that acts as a mediator between raw OCR text extraction and final hierarchy generation. This classifier automatically determines document hierarchy levels by analyzing text features, positions, and relationships, eliminating the need for manual metadata generation while preserving hierarchy accuracy
2Ease of operation
If automated OCR-based hierarchy generation is implemented, then user input is reduced, but document structure understanding capability is challenged
Solution Approach 1:
The system replaces manual mechanical analysis of document structure with automated machine learning-based analysis. The ML classifier processes OCR-extracted text blocks, detecting hierarchical relationships through learned patterns in text positioning, formatting, and content, thereby reducing user input while successfully understanding document structure
Solution Approach 2:
The patent transforms the document structure understanding problem into a parameter-based classification task. By extracting features such as text block positions, sizes, fonts, and spatial relationships, the system converts structural understanding into measurable parameters that the ML classifier can process automatically, reducing user input requirements
3Adaptability or versatility
If metadata is extracted from scanned documents, then linked table-of-contents can be created, but processing complexity increases
Solution Approach 1:
The system segments the complex metadata extraction process into distinct modular components: OCR text extraction, text block organization, feature extraction, ML classification, and hierarchy assembly. Each module handles a specific aspect of processing, reducing overall complexity while enabling linked table-of-contents generation and other document enhancements
Data Source
Figure 1
Figure 2
Figure 3
AI summary
In some aspects, a method includes performing optical character recognition (OCR) based on data corresponding to a document to generate text data, detecting one or more bounded regions from the data based on a predetermined boundary rule set, and matching one or more portions of the text data to the one or more bounded regions to generate matched text data. Each bounded region of the one or more bounded regions encloses a corresponding block of text. The method also includes extracting features from the matched text data to generate a plurality of feature vectors and providing the plurality of feature vectors to a trained machine-learning classifier to generate one or more labels associated with the one or more bounded regions. The method further includes outputting metadata indicating a hierarchical layout associated with the document based on the one or more labels and the matched text data.