AR Content Mirroring With Hand-Tracked Phone Navigation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems face challenges in efficiently capturing, mirroring, and navigating augmented reality content, particularly on wearable devices like glasses, due to computational intensity and power constraints, which affect latency and user interaction.
Innovation Solution
A wearable device with a waveguide-based display and hand-tracking capabilities enables manipulation of a mirrored 2D phone display in a 3D augmented reality space, using a gradient-coated diffractive structure for consistent light propagation and low-power circuitry to enhance user interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If augmented reality content is processed on wearable devices, then user interaction and immersion are improved, but computational intensity and power consumption increase
Solution Approach 1:
The system segments processing tasks by mirroring the phone display to the wearable device rather than processing all AR content locally on the glasses. The wearable device handles only the mirroring and basic hand tracking, while the phone handles heavier computational tasks, reducing power consumption on the wearable device.
Solution Approach 2:
The patent introduces a mirroring layer as an intermediary between the phone display and the AR environment. This mirroring display acts as a mediator that allows interaction with phone content through hand gestures in AR space, reducing the computational burden on the wearable device while maintaining user interaction capability.
2Ease of operation
If hand tracking is implemented for manipulating mirrored display, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The wearable device utilizes existing hand tracking capabilities that can be applied across multiple functions - navigating the mirrored display, interacting with AR content, and controlling phone interface elements. This multi-functional approach reduces overall system complexity by reusing the same tracking infrastructure for different purposes.
3Ease of operation
If mirrored display is rendered in waveguide-based display, then user experience is improved, but light propagation consistency becomes challenging
Solution Approach 1:
The patent applies gradient coating to specific regions of the waveguide to optimize light propagation characteristics. By varying the coating properties across different zones of the waveguide, the system achieves consistent light distribution and image quality across the displayed content, addressing the light propagation challenge while maintaining enhanced user experience.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution reduces latency and power consumption while improving the efficiency of processing augmented reality content, allowing seamless navigation and manipulation through hand gestures, enhancing the augmented reality experience on wearable devices.
Implementation Method 1
using a gradient-coated diffractive structure for consistent light propagation
Implementation Method 2
waveguide-based display
Data Source
AI summary
The subject technology receives, by one or more hardware processors implementing a local wireless network, a request from a client device to mirror media content displayed on a screen of the client device on a wearable device. In response to the request, the subject technology causes a display of the media content in a mirroring lens of the wearable device. While the media content is being displayed in the mirroring lens of the wearable device, the subject technology tracks hand gestures of a user wearing the wearable device and viewing the media content displayed in the mirroring lens of the wearable device. The subject technology processes navigational or manipulation data based on the tracked hand gestures and sends a navigation or manipulation instruction to the client device or a mirroring lens processor of the wearable device based on the tracked hand gestures.