AR Video Clip Tracking with 6DoF-3DoF Fallback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual objects in augmented reality experiences often disappear or behave erratically due to environmental conditions and unanticipated visual interruptions, disrupting the illusion of their presence in real-world environments.
Innovation Solution
The augmented reality system stores tracking indicia with the video clip during capture, enabling seamless transitions between 6DoF and 3DoF tracking using multiple redundant systems, and combines sensor information from various approaches to maintain object positioning, even after capture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single tracking system is used for virtual object positioning, then the system complexity is low, but the tracking reliability deteriorates due to environmental conditions and visual interruptions
Solution Approach 1:
The tracking system is segmented into multiple independent tracking subsystems, each capable of operating autonomously. The system divides the tracking function across different sensor types (e.g., visual tracking, inertial tracking, magnetic tracking) so that if one subsystem fails due to environmental conditions or visual interruptions, other subsystems can maintain tracking reliability.
Solution Approach 2:
The system performs preliminary action by pre-configuring multiple redundant tracking subsystems and establishing fallback protocols before tracking begins. When visual interruptions occur, the system has already prepared alternative tracking methods (such as switching to inertial or magnetic tracking) to maintain continuous object positioning without interruption.
2Reliability
If multiple redundant tracking systems are used to maintain continuous tracking, then the tracking reliability improves, but the energy consumption increases
Solution Approach 1:
The system dynamically adjusts the operation of multiple tracking subsystems based on real-time conditions. Instead of running all subsystems continuously, the system activates additional tracking methods only when needed (e.g., switching to alternative tracking when visual interruptions are detected), thereby maintaining tracking continuity while minimizing unnecessary energy consumption from continuously operating redundant systems.
Solution Approach 2:
The system changes operational parameters of tracking subsystems based on environmental conditions. When visual tracking is interrupted, the system changes parameters by increasing the weight or priority of alternative tracking methods (inertial, magnetic) in the fusion algorithm, allowing continuous tracking without requiring all subsystems to operate at full capacity simultaneously, thus reducing overall energy consumption.
3Measurement precision
If real-time sensor processing is performed during video capture, then the tracking precision improves, but the processing time increases
Solution Approach 1:
The system performs preliminary action by pre-processing and storing sensor data during video capture in an optimized format. Tracking indicia and sensor information are pre-computed and stored alongside the video clip, allowing rapid retrieval and processing during playback without requiring intensive real-time computation during the original capture phase, thus reducing processing delays while maintaining positioning precision.
4Adaptability or versatility
If tracking indicia are stored with the video clip for post-processing, then the adaptability improves for adding virtual objects to captured videos, but the data storage requirements increase
Solution Approach 1:
The system extracts and stores only the essential tracking indicia and sensor information needed for virtual object tracking, rather than storing complete raw sensor datasets. By extracting key positioning data, orientation information, and timestamp correlations, the system enables post-processing adaptability for adding virtual objects to captured videos while minimizing the quantity of stored data through selective extraction of critical tracking parameters.
Data Source
AI summary
Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing at least one program, and a method for rendering a three-dimensional virtual object in a video clip. The method and system include capturing, using a camera-enabled device, video content of a real-world scene and movement information collected by the camera-enabled device during capture of the video content. The captured video and movement information are stored. The stored captured video content is processed to identify a real-world object in the scene. An interactive augmented reality display is generated that: adds a virtual object to the stored video content to create augmented video content comprising the real-world scene and the virtual object; and adjusts, during playback of the augmented video content, an on-screen position of the virtual object within the augmented video content based at least in part on the stored movement information.