Document Subgrouping via OCR Word Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in efficiently grouping and classifying paper documents that have been scanned, as they often contain OCR errors, text differences, graphics, noise, rotations, skewing, and handwriting, making it difficult to transform them into intelligent content for enterprise applications.
Innovation Solution
The solution involves creating subgroups of documents using optical character recognition data by generating a matrix for words included in documents, calculating distances between word pairs, forming word clusters, and creating sets of word clusters to subgroup documents based on common words, thereby distinguishing pages by content rather than overall appearance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional OCR methods are used to process scanned documents, then text extraction is performed, but recognition accuracy deteriorates due to OCR errors, text differences, graphics, noise, rotations, skewing, and handwriting
Solution Approach 1:
The patent divides documents into subgroups based on common words and their spatial relationships. By segmenting the document set into smaller, more homogeneous groups, the system can process each subgroup with tailored parameters, improving recognition accuracy while reducing the impact of variations across the entire document set.
Solution Approach 2:
The patent applies different processing approaches to different parts of documents based on their content characteristics. By analyzing word clusters and their spatial relationships locally, the system adapts to specific document regions, handling variations in handwriting, rotations, and noise more effectively than uniform global processing.
2Adaptability or versatility
If documents are grouped by overall appearance, then visual similarity is achieved, but content-based classification deteriorates due to differences in text and graphics
Solution Approach 1:
The patent transitions from grouping documents by overall visual appearance to grouping by content dimensions - specifically by common words and their spatial relationships. This dimensional shift enables content-based classification while managing complexity through focus on key textual features rather than entire document structures.
Solution Approach 2:
The patent extracts common words from documents and uses their spatial relationships as the basis for grouping. By taking out and focusing on these key textual elements rather than processing entire documents for visual similarity, the system achieves content-based classification with reduced complexity.
3Measurement precision
If all pages are treated uniformly, then processing simplicity is maintained, but recognition accuracy deteriorates due to semi-independent page variations
Solution Approach 1:
The patent segments pages into subgroups based on content similarity measured through common words and spatial relationships. This segmentation allows each subgroup to be processed with optimized parameters, improving recognition accuracy for semi-independent pages while maintaining processing efficiency through automated clustering.
Solution Approach 2:
The patent changes processing parameters dynamically based on document subgroup characteristics. By identifying common words and their spatial patterns, the system adapts processing parameters for each subgroup, improving recognition accuracy for varied page types while maintaining overall processing efficiency.
Data Source
AI summary
Creating subgroups of documents using optical character recognition data is described. A matrix is created for words included in documents. Each column-row combination in the matrix indicates whether a corresponding word that is associated with the column-row combination is included in a corresponding document that is associated with the column-row combination. Distances are identified between pairs of the words. Each distance is based on a number of the documents that differ in including a corresponding pair of the words. Word clusters are created. Each word cluster includes pairs of words associated with a corresponding distance less than a distance threshold. Sets of word clusters are created. A set of word clusters includes word clusters that are not associated with any of the documents associated with other word clusters in the set. Subgroups of the digitized documents are created based on a set of word clusters with a highest word score.