Video OCR Tracking via Common Coordinate System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing optical character recognition (OCR) technologies face challenges in processing video frames in real-time, leading to latency issues and suboptimal performance when recognizing and rendering text, especially when dealing with multiple frames or documents that exceed a single frame's capacity.
Innovation Solution
The architecture employs an OCR text recognition engine and a tracker system that establishes a common coordinate system, allowing for the comparison and combination of OCR results from multiple frames using a tracking reference coordinate system, enabling asynchronous processing and real-time registration of recognized text results by applying transformations from a keyframe to subsequent frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If frame-by-frame OCR processing is performed to enable video registration, then real-time display capability is improved, but processor loading increases and performance suffers due to latency constraints
Solution Approach 1:
The system performs OCR processing in advance on a buffer of frames before display is needed. By pre-processing frames and maintaining a buffer of processed results, the system can fulfill real-time display requirements without subjecting the processor to the full computational burden during actual display moments, thus reducing peak processor loading while maintaining real-time capability.
Solution Approach 2:
The system continuously processes frames through a buffer mechanism, maintaining ongoing OCR operations on multiple frames simultaneously. This continuous processing approach ensures that when display is needed, results are already prepared, eliminating idle processor time while distributing the computational load over time rather than concentrating it at single display moments.
2Speed
If single-frame OCR without tracking is used, then processing speed is improved, but real-time video registration of results with subject matter is not enabled
Solution Approach 1:
The system introduces a buffer as an intermediary between the fast single-frame OCR processor and the real-time display system. The buffer receives results from the high-speed OCR processor and manages their timing and registration, enabling real-time video registration capability without requiring the OCR processing itself to be slowed down or synchronized with display refresh rates.
Solution Approach 2:
The system dynamically adjusts the buffer size and processing rate based on actual display timing requirements. By making the buffer management adaptive rather than fixed, the system can maintain high processing speeds while ensuring results are available for registration at the appropriate moments, resolving the conflict between speed and registration capability.
3Measurement precision
If all frames are processed to achieve optimal OCR accuracy, then recognition quality is improved, but processing time increases and real-time performance is compromised
Solution Approach 1:
The system segments the stream of video frames into manageable batches that are processed through the buffer. Rather than processing all frames sequentially to completion, the buffer divides the work into smaller units that can be processed in parallel or with overlapping operations, achieving high accuracy for each segment while maintaining overall real-time performance through efficient segment management.
Solution Approach 2:
The system changes the processing parameters dynamically based on buffer contents and timing requirements. By adjusting processing depth, buffer size, and processing rate as parameters rather than using fixed values, the system can optimize accuracy for individual frames when time permits while maintaining faster processing when real-time performance is critical, thus resolving the accuracy-time tradeoff.
Data Source
AI summary
Architecture that enables optical character recognition (OCR) of text in video frames at the rate at which the frames are received. Additionally, conflation is performed on multiple text recognition results in the frame sequence. The architecture comprises an OCR text recognition engine and a tracker system; the tracker system establishes a common coordinate system in which OCR results from different frames may be compared and/or combined. From a set of sequential video frames, a keyframe is chosen from which the reference coordinate system is established. An estimated transformation from keyframe coordinates to subsequent video frames is computed using the tracker system. When text recognition is completed for any subsequent frame, the result coordinates can be related to the keyframe using the inverse transformation from the processed frame to the reference keyframe. The results can be rendered for viewing as the results are obtained.