Wrist-Worn Grasp Detection With Low-Power Cameras and Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current wearable devices, such as smartwatches, lack the capability to detect and identify user actions and interactions with objects, limiting their functionality and context awareness.
Innovation Solution
A wrist-wearable device equipped with non-image sensors (e.g., neuromuscular sensors) and image sensors, along with processors, captures sensor data and image data to identify grasp actions by combining both types of data for context-aware interaction detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multiple sensors (image sensors and non-image sensors) are combined for grasp detection, then measurement precision and context awareness are improved, but device complexity increases
Solution Approach 1:
The patent combines image sensors and non-image sensors (such as IMU, accelerometer, gyroscope) into a unified sensor fusion system that processes multiple data streams simultaneously. This merging of different sensor types enables comprehensive grasp detection by correlating visual data with motion and orientation data, thereby improving measurement precision while managing device complexity through integrated processing.
Solution Approach 2:
The wearable device is designed with multi-functional capabilities, where the same sensor array serves multiple purposes: image capture for visual recognition, inertial measurement for motion tracking, and orientation sensing for context awareness. This universal sensor system handles various detection tasks (grasp detection, gesture recognition, activity monitoring) without requiring separate dedicated sensors for each function, thus improving precision across multiple metrics while controlling overall device complexity.
2Adaptability or versatility
If sensor fusion is implemented for context-aware detection, then functionality and context awareness are improved, but processing requirements and energy consumption increase
Solution Approach 1:
The system implements periodic sensing and processing cycles where sensors are activated at specific intervals rather than continuously. The sensor fusion algorithm processes data in discrete time windows, correlating image data with sensor data at periodic checkpoints. This periodic operation maintains context-aware functionality by detecting grasp events and user interactions while significantly reducing overall power consumption compared to continuous processing.
Solution Approach 2:
The system performs preliminary processing of sensor data to identify potential events of interest before initiating full sensor fusion analysis. Pre-processing steps filter out non-event periods and prepare data structures in advance, so that when a grasp or interaction event occurs, the system can quickly correlate image and sensor data without requiring sustained high-power processing. This preliminary action enables context-aware detection while managing energy consumption through selective intensive processing.
Data Source
AI summary
A method of grasp detection is described. The method includes, capturing, via one or more image sensors of a wearable device, image data including a plurality of frames. The plurality of frames includes an object within a field of view of the one or more image sensors. The method further includes capturing, via one or more non-image sensors of the wearable device, sensor data including a sensed interaction with the object and a user of the wearable device and identifying a grasp action performed by the user based on a combination of the sensor data and the image data.