Document Clustering via Layout Segment Pairs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classifying documents based on their source information is challenging when the source identity is not readily available, especially in semi-structured documents where the layout structure is not formal and keywords are similar across different templates.
Innovation Solution
The method involves identifying pairs of character segments in documents, generating a map of unique segment pairs, creating feature vectors based on these pairs, and clustering documents based on their layout structures to associate each cluster with a specific layout template, enabling information extraction from known locations within the documents.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If source information is extracted directly from documents, then classification accuracy is improved, but source information is often unavailable or lost in semi-structured documents
Solution Approach 1:
The patent uses document layout structure as an intermediary feature to infer source information. Instead of directly extracting source identifiers from documents where they may be lost or unavailable, the system extracts layout characteristics (segment positions, sizes, relationships) that serve as indirect indicators of document source, enabling classification without direct source information.
Solution Approach 2:
The patent creates a copy or representation of the document's layout structure as a feature vector. This layout fingerprint captures the essential structural characteristics of the document without requiring the original source information to be present, allowing the system to work with a structural copy that preserves source-identifying patterns.
2Measurement precision
If layout structure analysis is used to distinguish documents, then classification accuracy is improved, but processing complexity increases
Solution Approach 1:
The patent segments the document into character segments and identifies pairs of segments with specific spatial relationships. By dividing the document analysis into discrete segment pairs rather than analyzing the entire document layout at once, the system reduces processing complexity while maintaining the ability to capture distinguishing layout features for accurate classification.
Data Source
AI summary
A method and system are provided for classifying semi-structured documents. According to one embodiment, the method includes accessing a number of documents, and identifying in each of the documents pairs of character segments. The method also includes generating a map of unique identified segment pairs across all documents in the several documents. The method further includes generating for each of the several documents a respective feature vector based on one or more unique pair of segments in the map that are also identified in the document, and clustering the plurality of documents into a plurality of clusters, using the feature vectors.