Form Structure Extraction Network Tile Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for converting paper forms to adaptive forms using deep learning and deep neural networks face challenges in extracting document structure from high-resolution images due to computational costs, leading to limitations in capturing fine structures.
Innovation Solution
A form extraction network architecture that processes high-resolution images by segmenting them into tiles, using a convolutional network for feature detection and a multidimensional recurrent neural network to maintain spatial state information, combined with a global-lookup module for efficient information propagation across tiles, allowing for accurate classification of form elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are used to extract document structure from high-resolution images, then measurement precision and manufacturing precision are improved, but use of energy and computational costs increase significantly
Solution Approach 1:
The patent segments the high-resolution document image into multiple overlapping patches or tiles. Each patch is processed independently by the deep neural network to extract local structural features. This segmentation approach reduces the computational burden on any single processing unit while collectively capturing the entire document structure with high precision.
Solution Approach 2:
The patent introduces a spatial hierarchy dimension by organizing patches at multiple scales. A pyramid structure is employed where coarse-level patches provide contextual information and fine-level patches capture detailed structural elements. This multi-scale approach enables the system to maintain high measurement precision across different document regions without requiring uniform high-resolution processing throughout the entire image.
2Measurement precision
If high-resolution images are processed to extract fine document structure, then measurement precision is improved, but computational costs and processing time increase
Solution Approach 1:
The patent divides the high-resolution image into multiple patches that can be processed in parallel. This segmentation enables fine structure detection in each local region while allowing simultaneous processing of multiple patches, thereby maintaining measurement precision without proportionally increasing total processing time.
Solution Approach 2:
The patent employs a multi-scale pyramid processing approach where images are processed at different resolution levels in a hierarchical sequence. Coarse-level processing provides initial structural context quickly, followed by progressively finer-level processing that refines structural details. This periodic action at multiple scales achieves fine structure detection accuracy while managing computational throughput efficiently.
3Reliability
If deep learning models are trained to recognize form elements, then reliability of form identification is improved, but loss of time for training and deployment increases
Solution Approach 1:
The patent segments the training process by creating specialized sub-models or feature extractors for different form element types (e.g., headers, fields, tables, signatures). Each sub-model is trained on specific form elements, improving overall reliability through specialized recognition. This segmentation of training reduces the time required to train each individual component compared to training a single monolithic model on all form elements.
Solution Approach 2:
The patent employs pre-trained base models that have been trained on general document image data before being fine-tuned for specific form elements. This preliminary action of pre-training on large-scale general document data provides a strong foundation that accelerates subsequent specialized training, reducing the overall time loss while maintaining high reliability in form element identification.
Data Source
AI summary
A method and system for detecting and extracting accurate and precise structure in documents. A high-resolution image of documents is segmented into a set of tiles. Each tile is processed by a convolutional network and subsequently by a set of recurrent networks for each row and column. A global-lookup process is disclosed that allows “future” information required for accurate assessment by the recurrent neural networks to be considered. Utilization of high-resolution image allows for precise and accurate feature extraction while segmentation into tiles facilitates the tractable processing of the high-resolution image within reasonable computational resource bounds.