Third-Person AR Spectator Views With Synchronized Object Compositing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Spectators in augmented reality (AR) sessions are limited to viewing only the user's physical interactions and are unable to see the virtual environment, hindering their engagement and understanding of the user's AR experience.
Innovation Solution
Systems and methods that utilize one or more cameras to capture the user's interactions and virtual elements, synchronizing and compositing them into a third-person view for spectators, allowing seamless integration of virtual and physical objects in the AR session.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If spectators view only the user's physical interactions in AR sessions, then the system complexity remains low, but the spectator engagement and understanding of the virtual environment deteriorates
Solution Approach 1:
The patent merges the first-person view from the head-mounted display camera and the third-person view from the spectator camera into a single composite video feed. Virtual elements are rendered and composited onto both video streams simultaneously, allowing spectators to see both the user's physical actions and the virtual environment in an integrated view, thereby resolving the contradiction between maintaining simple system architecture and providing comprehensive virtual environment information.
Solution Approach 2:
The patent introduces a remote system as an intermediary that receives video feeds from both the head-mounted display and spectator cameras, renders virtual elements, and composites them into a unified spectator view. This intermediary system handles the complex task of synchronizing and combining multiple video streams and virtual elements, preventing the complexity from directly affecting the user's AR experience while still providing comprehensive spectator information.
2Reliability
If virtual elements are composit ed onto spectator camera feed in real-time, then spectator engagement improves, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary rendering of virtual elements based on the head-mounted display's view of the physical environment. By pre-rendering virtual elements before compositing them onto the spectator camera feed, the system reduces real-time processing requirements during the compositing stage, thereby maintaining view accuracy while minimizing additional processing time.
Solution Approach 2:
The patent creates a copy of the virtual elements rendered for the head-mounted display and applies them to the spectator camera feed. Instead of independently rendering and compositing virtual elements for the spectator view, the system reuses the already-rendered virtual elements from the user's perspective, significantly reducing computational overhead and processing time while maintaining visual accuracy.
Data Source
AI summary
A user may interact and view virtual elements such as avatars and objects and/or real world elements in three-dimensional space in an augmented reality (AR) session. The system may allow one or more spectators to view from a stationary or dynamic camera a third person view of the users AR session. The third person view may be synchronized with the user view and the virtual elements of the user view may be composited onto the third person view.