Relative Head Motion UI Selection Under Low-Light Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a need for more efficient user/device interaction methods that leverage multiple input modalities, including video image streams and inertial measurement unit data, to perform tasks such as UI element selection on portable electronic devices, especially in low lighting conditions or when the user's eyes are occluded.
Innovation Solution
The use of relative user head motion, detected by integrated cameras, to determine UI element selection on a device's display, involving face detection, gaze direction estimation, and dynamic hysteresis to prevent accidental actions, with confirmation gestures for final action execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional touch-screen or touchless controls are used for UI element selection, then the user can interact with the device, but the interaction becomes inefficient or impossible in low lighting conditions or when the user's eyes are occluded
Solution Approach 1:
The patent replaces traditional mechanical interaction methods (touch-screen, gesture recognition requiring visual feedback) with a vision-based head pointing direction detection system. The camera captures facial images and determines head orientation to control UI elements, eliminating the need for visual feedback during interaction and enabling operation in low lighting or when eyes are occluded.
2Ease of operation
If head pointing direction is used for UI element selection, then hands-free navigation is enabled, but accidental selections may occur due to minor head movements
Solution Approach 1:
The patent implements dynamic threshold adjustment based on detected head movement characteristics. The system adapts the selection threshold according to the magnitude and velocity of head movements, allowing minor movements to be filtered out while responding to intentional directional changes. This dynamic adaptation prevents accidental selections while maintaining responsive hands-free navigation.
Solution Approach 2:
The system provides visual feedback by highlighting the currently selected UI element based on head pointing direction. This feedback mechanism allows users to verify the intended selection before confirmation, enabling correction of unintended selections and improving overall selection reliability in hands-free mode.
3Measurement precision
If continuous camera monitoring is used for head pointing detection, then accurate UI element selection is achieved, but energy consumption increases
Solution Approach 1:
The patent implements periodic camera activation triggered by detected head movements rather than continuous monitoring. The system monitors for head movement events and activates the camera only when such events occur, capturing facial images at these discrete moments to determine head pointing direction. This periodic operation significantly reduces energy consumption while maintaining accurate head pointing detection capability.
Data Source
AI summary
Techniques are disclosed herein to perform user/device interaction tasks for electronic devices (e.g., wearable devices) with greater efficiency—and while leveraging information streams gathered by multiple types of input modalities (e.g., captured video images, as well as inertial measurement unit (IMU) data, or the like). Performance of such user/device interaction tasks are desirably able to leverage a user's natural head pointing direction, thereby enabling touchless input. According to some embodiments, the user's “relative” head motion (which has been determined to be more robust than a user's eye gaze or “absolute” head pointing direction—especially when the user is in low lighting conditions, being captured by a low-resolution camera, and/or when the area around the user's eyes is occluded), e.g., as determined from images captured by one or more integrated device cameras, may be used to determine the selection of particular UI elements on a display screen of the electronic device.