Multi-Layer Clustering for Multi-Format Field Extraction Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data extraction technologies struggle with efficiently processing documents of the same type that vary in format, requiring multiple form extraction models, which is cumbersome and time-inefficient.
Innovation Solution
A multi-layer process involving automated visual and contextual analysis to cluster documents, generate cluster templates, propagate annotations, and create field extraction models, utilizing image hashes and TF-IDF vectors for accurate data extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple form extraction models are generated for different document formats of the same type, then data extraction accuracy is improved, but the complexity and time required for model generation increases significantly
Solution Approach 1:
The patent segments the model generation process into two distinct clustering stages: visual clustering based on document layout and structure, followed by contextual clustering based on semantic content analysis. This segmentation allows the system to handle format variations systematically while maintaining manageable complexity at each stage.
Solution Approach 2:
The patent creates a universal template clustering system that can process multiple document formats through a single multi-stage clustering framework. The system generates cluster templates that are applicable across different document variations, reducing the need for separate models for each format while maintaining extraction accuracy.
2Measurement precision
If multiple form extraction models are generated for different document formats, then extraction accuracy for varied formats is improved, but the time efficiency deteriorates
Solution Approach 1:
The patent performs preliminary visual clustering on document structures before contextual analysis, pre-organizing documents into format-based groups. This preliminary action reduces the computational burden of subsequent contextual analysis and accelerates the overall model generation process while maintaining accuracy.
Solution Approach 2:
The patent extracts and analyzes only the relevant visual and contextual features necessary for clustering, rather than processing entire documents. This selective extraction of key features (layout patterns, semantic content) significantly reduces processing time while preserving extraction accuracy.
3Measurement precision
If manual model generation is performed for each document variation, then model accuracy is improved, but automation level decreases
Solution Approach 1:
The patent implements self-service automation where the system automatically performs visual analysis, contextual analysis, clustering, and template generation without manual intervention. The automated system serves itself by generating accurate extraction models through multi-stage clustering, eliminating the need for manual model creation while maintaining high accuracy.
Solution Approach 2:
The patent incorporates feedback mechanisms where clustering results from visual analysis inform contextual analysis, and subsequent clustering results refine the final templates. This iterative feedback process ensures high accuracy while maintaining full automation, as the system continuously improves its own output without human intervention.
Data Source
AI summary
A computer-implemented process for generating cluster templates used for creating extraction models includes the following operations. A plurality of training files associated with a selected class are received. An automated visual analysis is performed on each of the plurality of training files. An automated contextual analysis is performed on each of the plurality of training files. A first clustering of the plurality of training files into a first plurality of clusters using results from the automated visual analysis is performed. A second clustering of one of the plurality of clusters into a second plurality of clusters is performed using results from the automated contextual analysis. Cluster templates for the first and second plurality of clusters are generated.