Text Localization and Recognition Neural Network for Complex Documents
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text extraction technologies face challenges in accurately localizing and recognizing text from complex documents and natural scenes, particularly due to issues like background separation, font variation, orientation, and occlusions, which are not effectively addressed by traditional OCR methods.
Innovation Solution
A text localization and recognition neural network (TLaRNN) is developed, incorporating a convolutional backbone, feature pyramid network, and dynamic pooling mechanisms to improve text localization and recognition, with an attentional mechanism for enhanced accuracy, and synthetic training data to generalize across diverse document types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional OCR methods are used, then text extraction from clearly laid out documents with global structure performs well, but text extraction from complex documents with scattered text elements and natural scenes fails
Solution Approach 1:
The patent divides the text extraction task into two independent sub-tasks: text localization (detecting text regions) and text recognition (recognizing character sequences). This segmentation allows each sub-task to be optimized separately, with the localization network identifying text regions regardless of document structure, and the recognition network processing each region independently, thereby achieving versatility across different document types while maintaining reliability.
Solution Approach 2:
The patent creates a universal text extraction system that handles multiple document types (complex documents, natural scenes, receipts, forms) through a unified architecture. The localization network is designed to detect text regions in any context, and the recognition network processes all detected text uniformly, enabling the system to adapt to diverse scenarios while maintaining consistent extraction accuracy.
2Ease of operation
If existing object detection algorithms like Mask R-CNN are used, then multiple outputs (bounding boxes, classification, segmentation) are obtained, but high spatial resolution information is lost during pooling operations
Solution Approach 1:
The patent applies local quality by using different pooling strategies for different tasks. The localization network uses standard pooling operations sufficient for bounding box detection, while the recognition network receives high-resolution feature maps through selective pooling that preserves spatial details. This allows each task to receive appropriately processed features, maintaining multi-task capability while preserving spatial resolution where needed.
3Ease of manufacture
If text localization and recognition are performed separately in two steps, then each step can be optimized independently, but the system cannot effectively handle complex documents with scattered text elements and distractions
Solution Approach 1:
The patent implements feedback by using the localization results to guide the recognition process. The localization network identifies text regions and provides bounding boxes and feature maps to the recognition network, which then focuses computational resources on these specific regions. This feedback mechanism allows the system to handle complex documents effectively by concentrating recognition effort on actual text elements while ignoring distractions, while maintaining independent optimizability of each module.
Data Source
AI summary
Disclosed are methods, systems, and non-transitory computer-readable medium for localization and recognition of text from images. For instance, a first method may include: receiving an image; processing the image through a convolutional backbone to obtain feature maps(s); processing the feature maps through a region of interest (RoI) network to obtain RoIs; filtering the RoIs through a filtering block to obtain final RoIs; and processing the final RoIs through a text recognition stack to obtain predicted character sequences for the final RoIs. A second method may include: constructing a text localization and recognition neural network (TLaRNN); obtaining training data; training the TLaRNN on the training data; and storing trained weights of the TLaRNN. The constructing the TLaRNN may include: connecting a convolutional backbone to a region of interest (RoI) network; connecting the RoI network to a filtering block; and connecting the filtering block to a text recognition network.