Local-Network Media Mirroring for Gesture-Controlled AR Glasses

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing technologies face challenges in efficiently processing and enhancing augmented reality experiences on wearable devices, particularly in mirroring and navigating augmented reality content using hand tracking in a 3D space, while managing power and resource constraints.

Innovation Solution

A wearable device with a waveguide-based display incorporating a gradient-coated diffractive structure in the pupil replication element, along with low-power and high-speed circuitry, enables efficient mirroring and navigation of augmented reality content, allowing users to manipulate a mirrored 2D phone display in a 3D space using hand gestures.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If hand tracking is implemented for manipulating augmented reality content in 3D space, then ease of operation is improved, but device complexity increases

Engineering Contradiction:

Improveease of operationVSAvoiddevice complexity

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary processing layer between the camera and the display. A camera captures hand gestures, an image processing system analyzes the gestures, and a graphics engine renders the corresponding 3D interactions. This intermediary chain enables complex hand tracking functionality while distributing computational complexity across multiple specialized components rather than requiring a single complex system.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent replaces traditional mechanical interaction methods (physical buttons, knobs, or touchscreens) with optical field-based interaction. Instead of mechanical contact, users employ hand gestures captured by cameras and processed through image recognition algorithms to control augmented reality content, substituting mechanical actuation with optical sensing and computational processing.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Productivity

If augmented reality content is mirrored in real-time, then productivity is improved, but loss of time increases due to processing requirements

Engineering Contradiction:

ImproveproductivityVSAvoidlatency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent segments the real-time processing pipeline into distinct functional modules: camera capture, image processing for hand tracking, 3D graphics rendering, and waveguide display. Each segment operates independently and can be optimized separately, enabling efficient real-time processing. The segmentation allows parallel execution of these tasks and facilitates targeted performance optimization without affecting the entire system.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements preliminary processing actions by pre-computing and preparing graphical assets, models, and rendering parameters before user interaction. The system anticipates processing needs and pre-loads necessary resources into memory and buffers, reducing processing delays during actual hand gesture interaction and enabling smoother real-time rendering.

Inventive Principle:

Principle #10Preliminary action

3Manufacturing precision

If waveguide-based display with gradient coating is used, then manufacturing precision is improved, but device complexity increases

Engineering Contradiction:

Improvemanufacturing precisionVSAvoiddevice complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent employs parameter changes through gradient coating on the waveguide display. Instead of uniform coating, the coating density or refractive index varies gradually across the waveguide structure, creating a gradient that optimizes light coupling and display quality. This parameter variation enables precise control over optical properties while using a single coating process, avoiding the need for complex multi-step manufacturing.

Inventive Principle:

Principle #35Parameter changes

Applied Scientific Principles

This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.

Function Achieved in This Case

The solution provides a consistent and efficient visual experience with reduced latency and power consumption, enhancing the interaction and processing of augmented reality content on wearable devices.

Implementation Method 1

gradient-coated diffractive structure in the pupil replication element

Methodology Applied

Scientific EffectDiffraction: Diffraction

Implementation Method 2

waveguide-based display

Methodology Applied

Scientific EffectWaveguide optics: Waveguide (optics)

Data Source

PatentUS20250272047A1Media streaming to augmented reality glasses over a local network

Publication Date: 2025.08.28 SNAP INC

AI summary

The subject technology receives, by one or more hardware processors implementing a local wireless network, a request from a client device to mirror media content displayed on a screen of the client device on a wearable device. In response to the request, the subject technology causes a display of the media content in a mirroring lens of the wearable device. While the media content is being displayed in the mirroring lens of the wearable device, the subject technology tracks hand gestures of a user wearing the wearable device and viewing the media content displayed in the mirroring lens of the wearable device. The subject technology processes navigational or manipulation data based on the tracked hand gestures and sends a navigation or manipulation instruction to the client device or a mirroring lens processor of the wearable device based on the tracked hand gestures.