Wearable-Assisted Gesture Detection for Single-Camera Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computing devices with limited sensors, such as laptops with single front-facing cameras, struggle to accurately detect complex and fast user gestures due to insufficient image data capture and lack of depth information.
Innovation Solution
A system that combines image data from a front-facing camera with motion data from external wearable devices, such as smartwatches, using inertial measurements, signal strength, audio data, and radar measurements to enhance gesture detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single front-facing camera is used in a computing device, then the device complexity is reduced, but the measurement precision of gesture detection deteriorates
Solution Approach 1:
The patent combines data from multiple sources including the front-facing camera with inertial measurement data from a wearable device to detect gestures. By merging visual data with motion sensor data, the system achieves accurate gesture detection without requiring multiple complex sensors in the computing device itself.
Solution Approach 2:
The wearable device acts as an intermediary that provides additional motion data to supplement the limited camera data. The camera captures visual information while the wearable device's sensors capture motion information, and both are integrated to achieve accurate gesture detection.
2Use of energy by moving object
If a low-frame-rate camera is used, then the device complexity and power consumption are reduced, but the reliability of detecting fast gestures deteriorates
Solution Approach 1:
The system merges low-frame-rate camera data with high-frequency inertial measurement data from the wearable device. This combination allows the system to detect fast gestures reliably without requiring the camera to operate at high frame rates, thus maintaining low power consumption.
Solution Approach 2:
The wearable device continuously captures motion data in advance, providing high-frequency motion information that supplements the camera's lower-frame-rate visual data. This preliminary capture of motion data ensures that fast gestures are not missed even when the camera frame rate is low.
3Device complexity
If only image data from visual sensors is used, then the device complexity is reduced, but the measurement precision of depth information deteriorates
Solution Approach 1:
The patent merges image data from the visual sensor with motion data from the wearable device to obtain depth information. By combining these data sources and performing coordinate transformations, the system achieves accurate depth measurement without requiring additional depth sensors in the computing device.
Data Source
AI summary
The technology provides for a system for determining a gesture provided by a user. In this regard, one or more processors of the system may receive image data from one or more visual sensors of the system capturing a motion of the user, and may receive motion data from one or more wearable computing devices worn by the user. The one or more processors may recognize, based on the image data, a portion of the user's body that corresponds to a gesture to perform a command. The one or more processors may also determine one or more correlations between the image data and the received motion data. Based on the recognized portion of the user's body and the one or more correlations between the image data and the received motion data, the one or more processors may detect the gesture.