Depth Map Alignment for MR Content Transfer to Mobile Camera Feeds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mixed reality (MR) headsets limit seamless interaction and sharing of overlay content with external devices like smartphones due to reliance on the headset itself, hindering tasks such as QR code scanning and content sharing.
Innovation Solution
A system that aligns and synchronizes the spatial computing device's depth map with the mobile device's camera view, allowing the mobile device to interact with and display MR overlay content in real-time, maintaining spatial coherence regardless of camera movement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If content is displayed only on MR headset display, then immersive experience is improved, but device versatility and interaction capability deteriorate
Solution Approach 1:
The patent introduces a mobile device as an intermediary between the MR headset and the user's external devices. The mobile device receives depth map data from the MR headset, processes it to align with the camera view, and enables interaction with overlay content through the mobile device's interface, thus bridging the isolated MR experience with external device capabilities
Solution Approach 2:
The system segments the content display and interaction functions: the MR headset handles immersive visual display, the mobile device handles depth map processing and alignment, and external devices handle interaction. This segmentation allows each component to optimize its function while working together as an integrated system
2Manufacturing precision
If depth maps are aligned between spatial computing device and mobile device, then content synchronization is improved, but processing complexity deteriorates
Solution Approach 1:
The mobile device creates a simplified copy of the depth map data from the spatial computing device, processing it to match the mobile device's camera view perspective. This copied and transformed depth map enables synchronization without requiring the mobile device to process the full original depth map in detail
Solution Approach 2:
The system transforms depth map parameters to accommodate different coordinate systems and perspectives between the spatial computing device and mobile device camera. By adjusting parameters such as coordinate origin, orientation, and scale, the system achieves alignment without complex point-by-point processing
3Ease of operation
If overlay content is transferred to mobile device for display, then interaction capability is improved, but data transmission requirements deteriorate
Solution Approach 1:
The system extracts only the essential spatial alignment information and overlay content data needed for mobile device display, separating this from the full depth map data. This extraction minimizes transmitted data while preserving the critical information needed for accurate content placement and interaction
Data Source
AI summary
A computer implemented method includes receiving a first depth map of a spatial computing device field of view and receiving a second depth map of a mobile device camera field of view. The first and second depth maps are aligned. A spatial computing device field of view location of spatial computing device generated content displayed by the spatial computing device is then determined. The spatial computing device generated content and spatial computing device field of view location are sent to the mobile device for use in displaying the content by the mobile device in a manner that is aligned with the content displayed int eh spatial computing device.