Document Text Embeddings for Vertical and Horizontal Layout Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing document understanding technologies suffer from low accuracy, slow training processes, and difficulty in scaling and customizing neural network models for document analysis due to the exclusive focus on horizontally aligned text, ignoring other relevant features.
Innovation Solution
A pre-trained neural network model that incorporates embeddings for horizontal and vertical text, computer vision, character shape, font, table, and artifact features, allowing for fast training and customization without manual tuning, and supports multiple languages.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a traditional image model and text analytic model are trained separately with horizontal alignment assumption, then the training process can be completed, but the accuracy is low and training time is long
Solution Approach 1:
The patent combines the image model and text analytic model into a unified neural network model that processes document images and extracts text simultaneously. The model integrates computer vision capabilities for image understanding with natural language processing for text extraction, eliminating the need for separate training processes and improving both accuracy and training efficiency through joint learning.
Solution Approach 2:
The patent employs pre-training on large-scale document image datasets before fine-tuning on specific tasks. This preliminary action allows the model to learn general document understanding patterns and features beforehand, reducing the time required for task-specific training while improving accuracy through transfer learning.
2Reliability
If manual tuning is performed after training, then model performance can be optimized, but the process becomes complex and time-consuming
Solution Approach 1:
The patent implements self-adjusting mechanisms where the neural network automatically optimizes its parameters and architecture through continuous learning and adaptation. The model performs self-tuning by adjusting weights and biases during training without requiring manual intervention, and can adapt to different document types and languages automatically through few-shot learning capabilities.
3Adaptability or versatility
If the text analytic model is designed for horizontal text detection, then it can process standard documents, but it cannot handle vertical text or other document features
Solution Approach 1:
The patent designs a universal neural network model that can handle multiple document types, orientations, and languages within a single architecture. The model incorporates multi-scale feature extraction to detect text in various orientations (horizontal, vertical, angled) and includes specialized modules for recognizing different document elements such as tables, formulas, and structured layouts, making it adaptable to diverse document formats while maintaining high accuracy.
Data Source
AI summary
A system and method for training a neural network model includes obtaining, by a processing device, a document image containing raw text, tokenizing the raw text in the document image to obtain tokens located in a plurality of rows, identifying a first token in one of the plurality of rows, calculating a horizontal language feature of the first token based on the first token and one or more tokens in the row, and encoding, using a first encoder, the horizontal language feature into a horizontal language embedding, calculating a vertical language feature of the first token based on the token and one or more tokens in rows above or below the row, and encoding, using a second encoder, the vertical language feature into a vertical language embedding, and training a neural network model using the horizontal language embeddings and the vertical language embeddings.