Finger Tracking via Gaze-Aware 3D Vector Intersection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods face challenges in accurately determining user input on portable devices without physical contact, particularly due to limitations in camera elements that struggle with relative position and motion analysis, especially when the user is at a distance or the device is tilted, leading to errors in interpreting intended input positions.
Innovation Solution
The use of multiple digital cameras and infrared sensors to determine the relative position and motion of a user's features, such as a fingertip, in three dimensions, combined with gaze position determination, enables non-contact position, motion, and gesture-based input by analyzing image information and calculating vectors to accurately intersect with the device's virtual plane.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If simple camera elements are used in portable devices, then device complexity and power consumption are reduced, but measurement precision and reliability of position determination deteriorate
Solution Approach 1:
The system divides the measurement task into multiple segments by using separate cameras for different functions: one camera captures the user's finger position while another camera captures the user's eye position. This segmentation allows each camera to specialize in specific measurements, improving overall precision without requiring each individual camera to be complex
Solution Approach 2:
The patent introduces an intermediary computational process that uses perspective projection geometry to transform two-dimensional camera images into three-dimensional position information. By calculating the intersection of projection rays from multiple camera views and incorporating eye position data, the system accurately determines finger position relative to the display surface without requiring complex camera hardware
2Ease of operation
If user distance from device varies, then ease of operation is improved, but measurement precision of finger position deteriorates
Solution Approach 1:
The system transitions from two-dimensional image plane coordinates to three-dimensional spatial coordinates by incorporating depth information through perspective projection. By calculating the intersection of projection rays in 3D space and using eye position to establish the viewing perspective, the system accurately determines finger position regardless of distance from the device
Solution Approach 2:
The system uses eye position detection as feedback to dynamically adjust the perspective calculation for finger position determination. By continuously monitoring where the user is looking and using this information to calculate the appropriate projection geometry, the system maintains accuracy across varying distances and angles of interaction
3Adaptability or versatility
If device tilt angle varies, then adaptability is improved, but measurement precision of intended input position deteriorates
Solution Approach 1:
The system dynamically adapts its measurement geometry to match the device's current orientation and the user's viewing angle. By calculating projection rays and intersection points based on the actual device tilt and user position rather than assuming a fixed orientation, the system maintains accuracy across various device orientations and usage scenarios
Data Source
AI summary
A user can provide input to a computing device by moving a feature or object, such as a user's finger, within a field of view of at least one imaging element of the computing device. In order to ensure an accuracy of the determined input, the computing device can also attempt to determine a point of view of the user, such as by determining a relative position of the user's face or eyes. By determining a three-dimensional position of a feature and the user's point of view, a three-dimensional vector or other directional information can be determined whereby the intersection of that vector with the computing device indicates an intended location of input corresponding to the feature from the user's point of view.