AR Zonal OCR with Homography-Based Object Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems struggle to accurately and efficiently decode information for non-expert users, particularly in detecting object states through text recognition, often leading to misdetection and high computational costs.
Innovation Solution
A system utilizing a mobile device with a camera and a computer connected to a database, employing deep learning and homography-based image transformation to perform zonal OCR, minimizing misdetection and computational overhead by aligning feature points and executing OCR on transformed images based on reference zones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional OCR is applied to decode text in AR workflows, then information can be extracted for non-expert users, but misdetection occurs and computational costs increase
Solution Approach 1:
The patent divides the image processing task into two distinct stages: (1) detecting object states using deep learning networks to identify key regions, and (2) applying OCR only within those identified zones. This segmentation allows the system to process only relevant portions of the image, reducing computational cost while maintaining or improving text detection accuracy by focusing resources on areas where text is likely to appear.
Solution Approach 2:
The system performs preliminary object state detection before executing OCR. By first using deep learning to identify the state of objects (e.g., determining if a door is open or closed, or if a toner drum is removed), the system prepares the context needed to accurately locate and extract relevant text information, thereby improving reliability while optimizing computational resource usage.
2Measurement precision
If object state detection is used to confirm task completion, then task validation is achieved, but the system lacks context understanding for global object state
Solution Approach 1:
The patent merges object state detection with text extraction by integrating both functions within a unified processing framework. The deep learning network not only detects object states but also identifies regions containing text, and the OCR component extracts text from those same regions. This merging ensures that both task validation and global context understanding are achieved simultaneously, preventing information loss while maintaining precise measurement of task completion.
3Ease of manufacture
If standard OCR is applied without zonal segmentation, then text extraction is simpler, but different known regions where text is located become confused
Solution Approach 1:
The patent applies local quality by treating different regions of the image differently. Instead of applying uniform OCR processing across the entire image, the system identifies specific zones where text is likely to appear based on object state detection, and applies OCR only to those localized regions. This approach maintains implementation simplicity while significantly improving text region discrimination by focusing processing on relevant areas only.
Data Source
AI summary
A system for zonal OCR in an AR context is provided having a monitored device having text in at least one zone for indicating device states; a computer receives a picture of said monitored device and said text and processes said picture to determine a plurality of feature points; a database is in data communication with said computer and stores a plurality of reference images with reference feature points and reference zones; the computer matches at least some of the plurality of feature points with at least some of the plurality of reference feature points to compute a homography matrix; the computer transforms the image into a transformed image using the homography matrix; the computer executes optical character recognition on a zone of the transformed image based on the reference zones to generate extracted text.