Shared AR Scene Alignment for Cross-Space Participant Overlays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR devices struggle to synchronize the virtual overlays of participants in different physical spaces, leading to inconsistent orientations and arrangements among users, while also causing high network and server computational overheads.
Innovation Solution
A method and apparatus that generate a common scene description for AR devices worn by users in different physical spaces, determining user orientations relative to objects in their environments, and controlling the display to overlay participants virtually on these objects in a consistent manner, reducing network and server overheads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If AR devices stream images of other participants to conferencers in different physical spaces, then participants can see each other remotely, but the virtual overlays cannot be synchronized to match object and depth perspectives across different users
Solution Approach 1:
A cloud-based server acts as an intermediary that receives 3D avatars and spatial data from multiple AR devices, processes them into a unified virtual representation, and distributes synchronized overlay information back to each user's device. This mediator enables precise synchronization across different physical spaces without requiring direct peer-to-peer coordination between users.
Solution Approach 2:
The system transforms 2D camera images into 3D avatars with depth information, adding a third dimension to the communication. By representing participants as three-dimensional models with spatial awareness, the system enables accurate overlay synchronization that respects the depth and orientation of objects in each user's physical environment.
2Reliability
If the system processes and synchronizes virtual overlays for multiple participants in real-time, then consistent spatial arrangement is achieved, but network overhead and server computational requirements increase significantly
Solution Approach 1:
The processing workload is segmented between client devices and the server. Each AR device independently captures its local environment and participant information, while the server only processes and synchronizes the virtual overlay compositions. This division reduces server computational burden while maintaining spatial consistency through coordinated data exchange.
Solution Approach 2:
The system performs preliminary processing of 3D avatar generation and spatial mapping on the client devices before transmission to the server. By preparing these elements in advance, the server only needs to perform synchronization and composition operations, significantly reducing real-time computational requirements and enabling reliable spatial arrangement consistency.
Data Source
AI summary
There is included a method and apparatus comprising computer code configured to cause a processor or processors to obtain video from a AR devices, determine, based on the video data, orientations of users of the AR devices relative to object in respective physical spaces, generate generating a common scene description based on the orientation, and control, based on the common scene description, at least one of the AR devices to display a user of another of the AR devices such that the user is virtually overlaid on an object in the first physical space in which the one of the AR devise is located.