Wrist-Worn AR Gesture Detection Near Virtual and Physical Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial-reality environments require users to make large, attention-demanding gestures and use multiple devices, isolating them from physical interactions and requiring significant energy and space, leading to inconvenient and socially awkward experiences.
Innovation Solution
Wearable devices, such as wrist-wearable devices equipped with time-of-flight and EMG sensors, detect in-air and surface gestures to interact with artificial-reality environments, allowing users to perform operations without direct physical contact or fixed locations, using virtual affordances on curved surfaces and location-agnostic gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If hand-held devices are used to detect user gestures in artificial-reality environments, then gesture detection capability is improved, but device complexity and user burden increase due to requiring multiple devices
Solution Approach 1:
The wrist-wearable device performs multiple functions including gesture detection, spatial mapping, and interaction control, replacing the need for separate hand-held controllers and eliminating device complexity while maintaining comprehensive gesture detection capability
Solution Approach 2:
The patent combines gesture sensing, spatial awareness, and interaction control into a single wrist-wearable device, merging previously separate functions into one integrated system that reduces the number of devices users must carry and manage
2Measurement precision
If large gestures are required for interaction, then gesture detection accuracy is improved, but ease of operation deteriorates due to requiring significant space and energy
Solution Approach 1:
The system performs spatial mapping to identify local physical surfaces and creates virtual affordances anchored to specific locations, allowing users to perform small, localized gestures at convenient positions rather than requiring large movements across significant space
Solution Approach 2:
Virtual affordances act as intermediaries between the user and the artificial-reality environment, providing visual targets that guide small, precise gestures and enabling accurate interaction without requiring large physical movements
3Reliability
If users are isolated from physical surfaces through guardian boundaries, then safety is improved, but ease of operation deteriorates by preventing interaction with physical surfaces
Solution Approach 1:
Virtual affordances serve as intermediaries that bridge the guardian boundary, allowing users to interact with virtual representations of physical surfaces and objects without physically crossing safety boundaries, maintaining both safety and interaction capability
Solution Approach 2:
The system creates virtual copies of physical surfaces and objects within the guardian boundary, allowing users to interact with these copies through gestures without needing to physically contact the actual surfaces, preserving safety while enabling interaction
4Measurement precision
If multiple electronic devices are worn on contact points, then gesture detection precision is improved, but ease of operation deteriorates due to being tedious and inconvenient
Solution Approach 1:
The wrist-wearable device is designed to perform multiple gesture detection functions and interaction control tasks alone, eliminating the need to wear multiple devices on different body parts while maintaining comprehensive detection precision through its integrated sensor suite
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient, ergonomic, and socially acceptable interaction with AR environments, allowing users to navigate and control devices seamlessly through gestures, reducing visual clutter and ergonomic strain, and enhancing user interface efficiency.
Implementation Method 1
The sensors at the wearable devices can include time-of-flight sensors (e.g., to detect spatial distances)
Implementation Method 2
EMG sensors (e.g., to detect muscular responses)
Data Source
AI summary
A method includes, while presenting an artificial-reality environment that includes a virtual object and a representation of a physical object, detecting, based on data from a first group of sensors of a wrist-wearable device, that the wrist-wearable device is within a first distance of an electronic device responsive to user gestures and within another threshold distance of the virtual object. The electronic device is represented within the artificial-reality environment. The method includes, while the wrist-wearable device is within the first activation-threshold distance of the electronic device and is also within a second activation-threshold distance of the virtual object, determining, based on data from a second set of sensors of the wrist-wearable device, that a gesture corresponds to an operation at the electronic device. And the method includes, based on determining that the user gesture corresponds to the operation, causing the physical electronic device to perform the operation.