AR-VR Spatial Alignment via Coordinate Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) and virtual reality (VR) devices face challenges in interacting and sharing data, particularly in mixed reality (MR) video capture, where AR devices struggle to track the actual camera and align virtual objects with the real-world environment, requiring additional controllers and green screens, which complicates the synchronization of coordinate systems.
Innovation Solution
A method and system for synchronizing AR devices with VR devices using a networked MR window system that aligns tracking spaces, allowing AR devices to capture real-world data and share it with VR devices, enabling correct spatial alignment of virtual objects within the real-world environment through 3D point determination and coordinate transformation matrices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If an external camera with controller or tracker is used to capture MR video, then the system can track both the camera and MR device in the same tracking space, but the MR device cannot track the actual camera directly and must track the extra controller instead, increasing device complexity
Solution Approach 1:
The patent introduces a coordinate transformation matrix as an intermediary that translates positions between the camera coordinate system and the MR device coordinate system. This allows the MR device to accurately track the camera's position without requiring direct tracking hardware on the camera, resolving the contradiction by using a software-based mediation layer.
Solution Approach 2:
The patent creates a virtual representation of the camera's position and orientation in the MR device's coordinate system through coordinate transformation. This virtual copy allows the MR device to track the camera accurately without needing physical tracking hardware attached to the camera, reducing system complexity while maintaining tracking precision.
2Manufacturing precision
If green screen and depth camera are used to film VR user, then virtual objects can be aligned with tracking information, but the process requires additional equipment and complicates the synchronization of coordinate systems
Solution Approach 1:
The patent makes the coordinate transformation matrix universal, applicable to both AR devices and VR devices. This single mechanism handles coordinate system synchronization for different device types, eliminating the need for separate green screen and depth camera setups while maintaining alignment precision across multiple device types.
Solution Approach 2:
The patent changes the parameter representation from physical tracking markers (green screens and controllers) to mathematical coordinate transformations. By transforming the problem from physical alignment to parameter-based coordinate system translation, the system achieves the same alignment precision with reduced equipment complexity.
3Adaptability or versatility
If separate coordinate systems are used for AR and VR devices, then each device can operate independently, but virtual objects cannot appear in the correct position for both device types simultaneously
Solution Approach 1:
The coordinate transformation matrix serves as an intermediary that bridges the AR device coordinate system and the VR device coordinate system. This allows both devices to maintain their independent coordinate systems while achieving accurate spatial alignment of virtual objects across both device types through mathematical translation.
Data Source
AI summary
A method of synchronizing digital content between a first mobile device and a second mobile device is disclosed. Device position and orientation data is received at a first application executing in an operating system of the first mobile device. Sensor data is used to determine at least two 3D points associated with a physical location of the second mobile device. At least two 3D points are used to determine an offset for the position, orientation and scale of the first mobile device relative to the second mobile device. The offset and digital content data captured by the first mobile device are shared over a network with a second application executing in an operating system of the second mobile device. The second mobile device uses the offset to display the digital content captured by the first mobile device together with digital content data captured by the second mobile device.