Camera Attention Detection for Low-Power User Interface Response
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
There is a need for more efficient processing of user/device interaction tasks in electronic devices, leveraging multiple input modalities such as camera and IMU data, to determine user attention and reduce power and computing resource consumption.
Innovation Solution
An attention awareness algorithm that uses integrated cameras and IMUs to detect user head/gaze pointing direction relative to the device's display, initiating user interface actions based on attention detection, including dimming or brightening the display and performing actions like auto-scrolling when attention is detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If continuous user interface monitoring is performed to ensure responsiveness, then user experience quality is improved, but power consumption and processing resource usage increase
Solution Approach 1:
The system performs attention detection operations periodically rather than continuously, triggering detection only at specific moments (e.g., when notifications appear, device is awakened, or user interface events occur). This periodic monitoring approach maintains interface responsiveness while significantly reducing power consumption and processing resource usage compared to continuous monitoring.
Solution Approach 2:
The system extracts and processes only the essential information needed for attention detection from camera images, using selective face detection and pose estimation rather than analyzing all visual data. This extraction approach focuses computational resources on critical detection tasks while minimizing overall processing requirements.
2Measurement precision
If comprehensive camera image processing is performed to accurately detect user attention, then detection precision is improved, but processing time and computational resources increase
Solution Approach 1:
The attention detection process is segmented into distinct operational stages: triggering events initiate detection, face detection identifies user presence, pose estimation determines head orientation, and attention determination makes the final assessment. This segmentation allows the system to perform only necessary processing steps based on current context, improving both accuracy and efficiency.
Solution Approach 2:
The system performs preliminary checks before full attention detection, such as verifying device pose is within threshold ranges using IMU data before initiating camera-based face detection. This preliminary filtering prevents unnecessary complex image processing when the device orientation already indicates the user cannot be attending to the display.
Data Source
AI summary
This disclosure relates generally to the field of user/device interactions. More particularly, it relates to techniques for detecting when a user's attention is directed at an electronic device, e.g., as determined based, at least in part, on analysis of images captured by one or more cameras integrated in the electronic device. Attention awareness can help to reduce the power and/or computing resources consumed by the electronic device, e.g., by only providing certain user experiences at the electronic device when they are actually likely to be desired by the user. In some embodiments, an attention awareness algorithm may be initiated by some triggering event or action. Once initiated, images captured by a camera of the electronic device may be fed to an attention detection algorithm to determine whether the user's head is in a pose where the algorithm believes that the user likely desires to interact with the device's user interface.