Spatial-Textual Clustering for Video Text Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional Optical Character Recognition (OCR) systems struggle with accurately recognizing text in videos due to factors like blurriness, slant, and partial occlusion, leading to inaccurate and partial predictions.
Innovation Solution
The implementation of spatial-textual clustering-based recognition systems, which use a greedy clustering method with time constraints and combine spatial and textual distances in a scoring system to cluster OCR predictions, allowing for the selection of a representative prediction for each cluster.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional OCR is used to recognize text in videos, then text detection can be performed, but recognition accuracy deteriorates due to blurriness, slant, and partial occlusion
Solution Approach 1:
The system performs preliminary actions by collecting multiple OCR predictions across different video frames before final recognition. By gathering predictions from multiple time instances and spatial locations, the system prepares a pool of candidate recognitions that can be later aggregated to overcome individual frame limitations caused by blurriness, slant, or occlusion.
Solution Approach 2:
The system merges multiple OCR predictions from different frames and spatial locations into a unified recognition result. By combining predictions that appear across multiple frames or clustered spatially, the system achieves more reliable and accurate text recognition, overcoming the limitations of individual frame quality issues.
2Quantity of substance
If multiple video frames are processed individually by OCR, then more text instances are captured, but the number of unique predictions increases and precision decreases
Solution Approach 1:
The system segments the video into multiple frames and processes each frame individually through OCR, capturing text instances that may appear at different times. This segmentation allows the system to collect a comprehensive quantity of text predictions while maintaining the ability to later aggregate them for improved precision.
Solution Approach 2:
The system uses feedback mechanisms by comparing OCR predictions across multiple frames and using spatial-temporal clustering to identify consistent text instances. Predictions that appear multiple times or cluster together spatially receive positive feedback, reinforcing their validity and improving overall prediction precision while managing the quantity of unique predictions.
3Measurement precision
If a strict confidence score threshold is applied to OCR predictions, then high precision is achieved, but many valid predictions are filtered out
Solution Approach 1:
The system applies partial filtering by using a moderate confidence score threshold that accepts more predictions than a strict threshold would. By combining this partial filtering with spatial-temporal clustering, the system maintains high precision while preserving a larger quantity of valid predictions that might have been rejected by stricter thresholds.
4Productivity
If OCR is applied to each frame independently, then processing speed is maintained, but recognition accuracy deteriorates due to frame-to-frame variations
Solution Approach 1:
The system performs preliminary independent OCR processing on each frame to maintain processing speed, then subsequently aggregates results through spatial-temporal clustering. This two-stage approach preserves the speed advantage of independent frame processing while adding a refinement step that improves recognition accuracy by accounting for frame-to-frame variations.
Solution Approach 2:
The system merges OCR results from multiple independently processed frames through spatial-temporal clustering. By combining predictions that appear across multiple frames or cluster together in space, the system improves recognition accuracy while maintaining the processing efficiency of independent frame analysis.
Data Source
AI summary
Systems and methods for spatial-textual clustering-based recognition of text in videos are disclosed. A method includes performing textual clustering on a first subset of a set of predictions that correspond to numeric characters only and performing spatial-textual clustering on a second subset of the set of predictions that correspond to alphabetical characters only. The method includes, for each cluster of predictions associated with the first subset of the set of predictions, choosing a first cluster representative to correct any errors in each cluster of predictions associated with the first subset of the set of predictions and outputting any recognized numeric characters. The method includes, for each cluster of predictions associated with the second subset of the set of predictions, choosing a second cluster representative to correct any errors in each cluster of predictions associated with the second subset of the set of predictions and outputting any recognized alphabetical characters.