Document Structural Block Classification via ML Feature Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document processing techniques face challenges in accurately extracting structural blocks from documents, particularly in non-standardized formats like PDF, due to lack of standard tags or properties, and fail when dealing with variations in layout and image documents converted from scans, where features like font and spacing are lost.
Innovation Solution
A method and system using a machine learning model to determine structural blocks by extracting text, generating feature vectors, creating input matrices from neighboring text lines, and assigning structural block tags, which allows for context-aware and position-independent classification of document sections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If rule-based approaches are used for document structure identification, then the method works well for documents with consistent layouts, but it fails for documents with large variations in layout and image documents where font and spacing features are lost
Solution Approach 1:
The patent replaces rule-based mechanical processing with a machine learning model that automatically learns document structure patterns. The system extracts features from text lines and their neighbors, then uses a trained neural network to classify structural blocks, eliminating the need for manual rule creation and improving adaptability to various document layouts.
Solution Approach 2:
The patent changes from fixed rule-based parameters to dynamic feature extraction. The system extracts multiple features including text line properties, neighbor relationships, and structural patterns, then uses a machine learning model to determine optimal classification parameters based on the specific document content, enabling handling of layout variations.
2Difficulty of detecting and measuring
If existing pattern recognition techniques are used to segment document images into maximal homogenous regions, then blocks can be segmented as images, texts, and tables, but the actual structure of text content cannot be determined
Solution Approach 1:
The patent applies segmentation by dividing the document into text lines and further into structural blocks. The system processes each text line individually, extracting features and comparing with neighbors to determine its structural block membership, achieving fine-grained text structure identification beyond coarse block segmentation.
Solution Approach 2:
The patent introduces feature vectors as an intermediary between raw text lines and structural classification. The feature extraction process creates a representation that captures essential properties, which then feeds into the machine learning model for accurate structure determination, preserving information that would be lost in direct classification.
3Illumination intensity
If PDF format is used for document presentation, then the format is optimized for presentation, but it lacks structural information and has no standard tags or properties for identifying sections
Solution Approach 1:
The patent replaces reliance on PDF's lack of structural tags with a machine learning-based structural analysis system. Instead of depending on embedded metadata or standard tags that PDF lacks, the system analyzes visual and textual features to infer and determine document structure, effectively creating structural information where none officially exists.
Data Source
AI summary
This disclosure relates generally to document processing, and more particularly to method and system for determining structural blocks of a document. In one embodiment, the method may include extracting text from the document, the text including text lines. The method may further include generating a feature vector for each of the text lines, the feature vector for the text line including a set of feature values for a set of corresponding features in the text line. The method may further include creating an input matrix for each of the text lines, the input matrix for the text line including a set of feature vectors corresponding to a set of neighboring text lines along with the text line. The method may further include determining a structural block tag for each of the text lines based on the corresponding input matrix using a machine learning model.