Graph-Based Text Reconstruction for Mobile Document Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Character recognition on mobile devices is noisy, especially with small font sizes, and is exacerbated by limited screen area and camera field of view, leading to misread words and loss of context when trying to display and recognize text from documents.
Innovation Solution
A graph-based approach is used to reconstruct text in reading order from multiple images by generating content graphs for each image, stitching them together using graph isomorphism techniques, and selecting cluster labels based on similarity, allowing for improved recognition and noise reduction across images.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If character recognition is performed on mobile device screens, then text identification is enabled, but recognition accuracy deteriorates due to small font sizes and limited screen area
Solution Approach 1:
The patent divides the document into multiple sections by capturing several images of different portions. Each image is processed independently to identify text, and then the results are combined using graph isomorphism techniques to reconstruct the complete document in reading order, thereby overcoming the limitation of small screen area.
Solution Approach 2:
The patent introduces content graphs as an intermediary data structure to represent text elements and their spatial relationships. By using graph isomorphism as a mediator, the system can accurately match and combine text from multiple images, improving recognition accuracy despite small font sizes.
2Loss of information
If multiple images are captured to cover the entire document, then complete text coverage is achieved, but context is lost when displaying sections separately
Solution Approach 1:
The patent transitions from two-dimensional image data to a graph-based representation that captures spatial relationships and reading order. This dimensional transformation allows the system to maintain context information while efficiently processing multiple images by representing text elements as nodes with positional attributes.
3Measurement precision
If graph isomorphism techniques are used to stitch content graphs, then text reconstruction accuracy is improved, but computational complexity increases
Solution Approach 1:
The patent applies graph isomorphism techniques locally to match specific content nodes between images based on their spatial relationships and text content. Rather than performing global graph matching, the system compares local regions and their corresponding graphs, reducing computational complexity while maintaining reconstruction accuracy.
Data Source
AI summary
Systems and methods are provided for sharing a screen from a mobile device. For example, a method includes receiving an image from a mobile device, performing recognition on the image to identify space-delimited strings, and generating a content graph for the image, the content graph having content nodes that represent at least some of the strings and the content graph having edges that represent a relative position of strings associated with the content nodes connected by the edges. The method may also include repeating the receiving, performing recognition, and generating for a plurality of images, the plurality of images belonging to a session, and generating a combined graph from the plurality of content graphs based on similarity of content nodes between content graphs, the combined graph representing text from the plurality of images in reading order.