Character-Level LSTM for Form Reading Order
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for determining the reading order of documents, especially those with complex layouts like paper forms, struggle to accurately maintain context due to reliance on visual modalities and word-level models, which are prone to errors from imperfect OCR processes and typos.
Innovation Solution
A character-level model using Long Short-Term Memory (LSTM) recurrent neural networks is employed to determine the reading order by associating text runs with text blocks, processing text runs through stateful and stateless networks to generate internal representations, and a classifier network predicts the next text run direction or end of statement, effectively handling errors and complex layouts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If visual modalities and word-level models are used to determine reading order, then the system can process documents with complex layouts, but accuracy deteriorates due to OCR errors and typos
Solution Approach 1:
The patent replaces the mechanical/visual OCR-based text extraction system with a neural network-based character-level model. Instead of relying on visual modalities to extract text and then process words, the system uses a deep learning model that operates directly on character sequences, substituting the traditional mechanical OCR pipeline with an intelligent neural network approach that is more robust to errors
Solution Approach 2:
The patent changes the fundamental parameter of text processing from word-level to character-level. By processing documents at the character level rather than the word level, the system can handle OCR errors and typos more effectively, as character-level models can infer correct character sequences even when individual characters are misrecognized
2Device complexity
If word-level models are used for reading order determination, then processing is simpler, but robustness deteriorates when text contains typos or OCR errors
Solution Approach 1:
The patent substitutes the simple word-level modeling approach with a neural network-based character-level model. This replacement increases processing capability while providing robustness to errors, as the neural network can learn patterns and contexts that make it resilient to individual character recognition errors
3Adaptability or versatility
If conventional heuristic approaches are used to determine reading order, then the system can handle multi-column layouts, but context maintenance deteriorates
Solution Approach 1:
The patent replaces conventional heuristic approaches with a neural network-based system that can learn and infer reading order patterns. The character-level model, combined with the bidirectional LSTM architecture, can maintain context information by processing text in both forward and backward directions, allowing it to understand the semantic flow even in complex multi-column layouts
Data Source
AI summary
Techniques for determining reading order in a document. A current labeled text run (R1), RIGHT text run (R1) and DOWN text run (R3) are generated. The R1 labeled text run is processed by a first LSTM, the R2 labeled text run is processed by a second LSTM, and the R3 labeled text run is processed by a third LSTM, wherein each of the LSTMs generates a respective internal representation (R1′, R2′ and R3′). Deep learning tools other than LSTMs can be used, as will be appreciated. The respective internal representations R1′, R2′ and R3′ are concatenated or otherwise combined into a vector or tensor representation and provided to a classifier network that generates a predicted label for a next text run as RIGHT, DOWN or EOS in the reading order of the document.