3D Gesture Recognition via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current user interface technologies struggle to effectively recognize and respond to three-dimensional gestures, limiting the intuitive and interactive capabilities of touch-based input devices in mobile devices and computing systems.
Innovation Solution
Implementing a system that combines touch input devices, proximity sensors, and cameras to detect and interpret three-dimensional gestures by determining the location and movement of objects both on and away from the device's surface, using geometric features and triangulation techniques, and executing corresponding actions through dedicated libraries and software modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional touch-based input devices are used, then the device structure remains simple, but the capability to recognize three-dimensional gestures is limited
Solution Approach 1:
The patent combines multiple sensing technologies (touch sensors, proximity sensors, and cameras) into an integrated gesture recognition system. The touch sensor detects contact points on the surface, the proximity sensor detects objects near the surface, and the camera captures spatial positions, with all data processed together by a gesture recognition module to enable comprehensive three-dimensional gesture recognition.
Solution Approach 2:
The gesture recognition system serves multiple functions: it recognizes touch gestures on the surface, detects proximity gestures near the surface, and identifies three-dimensional gestures in space. This multi-functional approach allows a single system to handle various gesture types without requiring separate dedicated devices for each gesture category.
2Measurement precision
If multiple sensors are integrated for three-dimensional gesture detection, then gesture detection accuracy improves, but the device complexity increases
Solution Approach 1:
The gesture recognition system is divided into distinct functional modules: a touch sensor module for detecting surface contact, a proximity sensor module for detecting near-surface objects, a camera module for capturing spatial positions, and a gesture recognition module for processing data. This segmentation allows each component to be optimized independently while maintaining overall system accuracy.
Solution Approach 2:
The gesture recognition module acts as an intermediary that receives and integrates data from multiple sensor types (touch sensor, proximity sensor, camera). It processes this multi-source information to determine gesture characteristics, serving as a mediator that harmonizes the inputs from different sensing technologies into unified gesture recognition results.
3Ease of operation
If three-dimensional gestures are recognized, then user interaction intuitiveness improves, but the processing requirements increase
Solution Approach 1:
The system processes gesture data in stages, first identifying basic gesture characteristics from sensor inputs, then determining specific gesture meanings based on context. This partial processing approach allows the system to handle complex three-dimensional gestures without requiring full processing of all possible gesture variations, reducing overall computational requirements while maintaining recognition accuracy.
Data Source
AI summary
In one embodiment, a method includes identifying a gesture with respect to one or more surfaces of the computing device, the gesture comprising an earlier portion on at least one of the surfaces that includes one or more first points touching the at least one of the surfaces; and immediately following the earlier portion of the gesture, a later portion of the gesture that includes a series of second points distant from the surfaces, wherein the series of second points forms a trajectory vector having: an initial point defined by the first points, and a terminal point distant from the surfaces; determining a user input based on the identified gesture and a corresponding first location on the user interface; and executing one or more actions based on the determined user input.