AR Eyewear Gesture Control for Virtual Object Manipulation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing augmented reality technologies lack intuitive and immersive methods for users to interact with virtual objects in a physical environment, limiting the user experience.

Innovation Solution

The use of wearable eyewear devices equipped with dual visible-light cameras and an image processor to capture stereo images, allowing for three-dimensional projections and enabling interaction with virtual objects through hand gestures or movements, which can change attributes or settings of the virtual objects.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If traditional AR display methods are used, then virtual objects can be displayed in the physical environment, but intuitive and immersive interaction methods are lacking

Engineering Contradiction:

Improveuser interaction with virtual objectsVSAvoidimmersion quality of AR experience

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent replaces traditional mechanical interaction interfaces (buttons, screens, controllers) with gesture-based control. The system detects hand gestures in the physical environment and translates them into commands for manipulating virtual objects, eliminating the need for physical contact with control interfaces while maintaining precise control capability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces a gesture recognition system as an intermediary between the user and virtual objects. This intermediary layer captures hand movements in the physical space, interprets them as intended actions, and translates them into appropriate manipulations of virtual objects, creating a natural bridge between physical and digital domains.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Ease of operation

If hand gesture recognition is implemented, then intuitive interaction is achieved, but system complexity increases

Engineering Contradiction:

Improvegesture-based controlVSAvoidprocessing and detection systems

Core Design Contradiction:

Ease of operationVSDevice complexity

Solution Approach 1:

The patent makes the eyewear device's camera system multi-functional by using it for both capturing the physical environment for AR display and detecting hand gestures for control. This universal use of existing hardware components avoids adding dedicated gesture sensing equipment, thereby limiting the increase in device complexity.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent combines the gesture recognition functionality with the existing AR processing pipeline. The same image processing units and algorithms used for rendering virtual objects are also employed for detecting and interpreting hand gestures, merging two functions into a unified processing framework and reducing overall system complexity.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12585337B2Augmented reality experiences with object manipulation

Publication Date: 2026.03.24 SNAP INC

AI summary

Augmented reality experiences with an eyewear device including a position detection system and a display system are provided. The eyewear device detects at least one of a hand gesture or movement of the user in the physical environment, and associates the hand gesture with a setting for a virtual object held within the memory of the eyewear device. The eyewear device may then change an attribute of the virtual object based on the detected hand gesture or movement. The eyewear device then provides an output corresponding to one or more attributes of the virtual object. The virtual object may be, for example, a music player or a virtual game piece.