Neural Network Page Segmentation for Vector Graphics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing page segmentation techniques for unstructured vector graphics documents, such as PDF files, fail to accurately identify and classify objects like tables, lists, and captions due to the lack of structural information and reliance on heuristic algorithms that require manual correction and cannot effectively utilize both visual and textual information.
Innovation Solution
A system and method using neural networks to generate semantic page segmentation by combining visual and textual feature representations, where textual feature representations are mapped to pixels and used in conjunction with visual feature representations to classify document objects, enabling accurate identification of document objects through a combination of appearance-based and text-based information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If heuristic algorithms are used for page segmentation, then the system can operate without structural information, but the segmentation accuracy deteriorates and manual correction is required
Solution Approach 1:
The patent combines visual appearance information from rendered page images with textual information from the document content to create a multi-modal segmentation system. This merging of different information types enables accurate segmentation of unstructured documents without relying solely on heuristic rules, thereby improving segmentation accuracy while maintaining the ability to process documents lacking structural information.
2Ease of manufacture
If region-based classification algorithms are used, then high-level objects can be distinguished, but low-level features cannot be identified
Solution Approach 1:
The patent applies segmentation at multiple levels by dividing the document into different types of regions (text regions, image regions, table regions) and then further segmenting these regions into specific objects (paragraphs, captions, figure references). This multi-level segmentation approach enables the identification of both high-level document structures and fine-grained low-level features, overcoming the limitation of single-level classification algorithms.
3Extent of automation
If existing segmentation solutions are deployed to end users, then automation is achieved, but manual intervention cannot be performed for corrections
Solution Approach 1:
The patent implements feedback mechanisms that allow the segmentation system to learn from corrections and improve over time. The system can process user corrections and use this feedback to refine its segmentation models, thereby maintaining high automation while enabling easy correction of errors through iterative learning and model updates.
4Device complexity
If heuristic rules are used for object identification, then the algorithm can function with simple rules, but corner cases require manual correction
Solution Approach 1:
The patent creates a composite segmentation approach that combines multiple types of information (visual features, textual features, layout analysis) to form a robust segmentation model. This composite approach is analogous to using composite materials in engineering, where combining different materials creates a system that is more reliable and robust than any single component alone, thereby handling corner cases effectively without requiring manual corrections.
Data Source
AI summary
Disclosed systems and methods categorize text regions of an electronic document into document object types based on a combination of semantic information and appearance information from the electronic document. A page segmentation application executing on a computing device provides a textual feature representation and a visual feature representation to a neural network. The application identifies a correspondence between a location of the set of pixels in the electronic document and a location of a particular document object type in an output page segmentation. The application further outputs a classification of the set of pixels as being the particular document object type based on the identified correspondence.