Wrist-Worn TOF and EMG Gesture Detection Near Physical Surfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial-reality environments require users to engage in inconvenient, awkward, and socially unacceptable interactions using hand-held devices or multiple wearable devices, often isolating users from physical surfaces and requiring significant space and energy expenditure.
Innovation Solution
Wearable devices, such as wrist-wearable devices equipped with time-of-flight and EMG sensors, detect in-air and surface gestures to interact with artificial-reality environments, allowing users to perform operations without direct physical contact or fixed locations, using a combination of in-air, surface, and location-agnostic gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If hand-held devices or multiple wearable devices are used to detect user gestures, then gesture detection capability is provided, but device complexity and ease of operation deteriorate due to requiring users to carry and manipulate multiple devices
Solution Approach 1:
The patent merges gesture detection functionality into a single wrist-wearable device that integrates multiple sensor types (IMU, EMG, TOF) to detect both in-air and surface gestures. This consolidates what would traditionally require multiple separate devices or hand-held controllers into one wearable unit, reducing device complexity while maintaining comprehensive gesture detection capabilities.
Solution Approach 2:
The wrist-wearable device is designed with universal functionality to detect multiple gesture types (in-air gestures, surface gestures, contactless gestures) and interface with various electronic devices. The device serves multiple purposes: gesture recognition, spatial mapping, and device control, eliminating the need for specialized hand-held controllers for each function.
2Adaptability or versatility
If hand-held devices or multiple wearable devices are used to detect user gestures, then gesture detection capability is provided, but ease of operation deteriorates due to requiring users to carry and manipulate devices
Solution Approach 1:
The wrist-wearable device continuously monitors for gestures using its sensor array without requiring user initiation or manipulation of the device itself. The system automatically detects in-air gestures, surface gestures, and contactless interactions, providing self-service gesture recognition that eliminates the need for users to carry and manipulate separate controllers.
3Measurement precision
If users perform gestures requiring significant space and energy expenditure, then gesture detection accuracy is improved, but energy consumption and user convenience deteriorate
Solution Approach 1:
The patent employs local quality by using EMG sensors to detect localized muscle activity in the wrist and hand regions, enabling accurate gesture recognition without requiring large-scale body movements. This localized detection approach maintains gesture detection accuracy while significantly reducing the energy expenditure required from the user compared to traditional full-body gesture systems.
4Reliability
If VR devices create stationary guardian boundaries to isolate users from physical surfaces, then user safety is improved, but adaptability and ease of operation deteriorate due to isolation from physical surroundings
Solution Approach 1:
The wrist-wearable device acts as an intermediary that enables users to interact with both virtual and physical environments simultaneously. By detecting surface gestures and contactless gestures, the system allows users to navigate around physical objects without requiring virtual boundaries, maintaining safety while preserving adaptability to the physical surroundings through gesture-based interaction.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient, ergonomic, and socially acceptable interaction with artificial-reality systems, allowing users to navigate and control electronic devices seamlessly through gestures, reducing the need for physical contact and visual distractions, and enhancing user interface simplicity and flexibility.
Implementation Method 1
The sensors at the wearable devices can include time-of-flight sensors (e.g., to detect spatial distances)
Implementation Method 2
EMG sensors (e.g., to detect muscular responses)
Data Source
AI summary
A artificial-reality system, including an optional head-wearable device to display an artificial-reality environment and a wrist-wearable device, identifies and detects shapes of surfaces, and, optionally, the distances of surfaces from the wrist-wearable device, and makes operations, including haptic events, available to users by detecting gestures performed by the user, causing respective operations to be performed based on the type of gestures, and optionally, the distance of the wrist-wearable device from surfaces during performance of the gestures. For example, the wrist-wearable device can be configured to cause performance of a first set of operations corresponding to in-air gestures, which are performed within an in-air threshold distance of a surface, a second set of operations corresponding to near-surface gestures performed within a surface threshold distance of a surface. The operations can, for example, update the display at the optional head-wearable device, control device properties at other electronic devices, or respond to messages.