AR Video Overlay Placement With Saliency and Jitter Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems for real-time overlay placement in augmented reality applications are inefficient on mobile devices due to high computational demands and fail to optimally position labels without occluding the object/scene of interest, especially in dynamic camera movements and low-light scenarios.
Innovation Solution
A method and system that computes saliency maps and Euclidean distances in real-time to determine updated overlay positions, minimizing occlusion and temporal jitter by formulating an objective function that balances image saliency and temporal coherence, suitable for low-end devices like smartphones and tablets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If sophisticated state-of-the-art techniques for optimal positioning of textual labels are used, then label placement quality is improved, but computational efficiency and real-time performance deteriorate
Solution Approach 1:
The label placement problem is segmented into two distinct phases: an offline pre-computation phase that calculates initial overlay positions using sophisticated algorithms, and an online real-time phase that only computes incremental adjustments based on camera motion. This segmentation allows high-quality placement algorithms to run without real-time constraints, while real-time performance is maintained through simplified incremental updates.
Solution Approach 2:
Initial overlay positions are pre-computed offline before real-time video processing begins. This preliminary action establishes a foundation of optimal label positions that can be quickly adjusted in real-time without re-running complex optimization algorithms, thereby maintaining both placement quality and real-time performance.
2Manufacturing precision
If label positions are frequently updated to avoid occlusion, then scene interpretation quality is improved, but temporal jitter increases
Solution Approach 1:
The system implements feedback mechanisms that monitor camera motion and scene changes to dynamically adjust label positions. By using feedback from motion detection and saliency map changes, the system updates labels only when necessary, maintaining scene interpretation quality while avoiding unnecessary updates that would cause temporal jitter.
Solution Approach 2:
The label positioning system transitions from static pre-computed positions to dynamic adjusted positions based on real-time camera motion and scene changes. This dynamic adjustment allows labels to adapt to occlusion scenarios while maintaining temporal stability through controlled update frequencies and threshold-based triggering.
3Measurement precision
If complex saliency map computations are performed for every frame, then label placement accuracy is improved, but computational load increases
Solution Approach 1:
Full saliency map computations are performed periodically or at key frames rather than for every single frame. Between these periodic computations, the system uses incremental updates based on camera motion estimation and previous saliency information, reducing computational load while maintaining placement accuracy through periodic recalibration.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Textual overlays/labels add contextual information in Augmented Reality (AR) applications. The spatial placement of labels is a challenging task particularly for real time videos. Embodiments of the present disclosure provide systems and methods for optimal placement of contextual information for Augmented Reality (AR) applications to overcome the limitations of occlusion with object/scene of interest through optimally placing labels aiding better interpretation of scene. This is achieved by combining saliency maps computed for each frame of an input video with Euclidean distance between current and previous overall positions for each frame based on an initial overlay position of the label to calculate an updated overlay position for label placement in the video. The placement of overlays is formulated as an objective function that minimizes visual saliency around the object of interest and minimizes the temporal jitter facilitating coherence in real-time AR applications.