Mixed-Reality Camera Focus Control Using Hand Motion Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Auto-focus hunting in mixed-reality head-mounted display (HMD) devices caused by user hand movements reduces the quality of user experience and consumes excessive power due to frequent lens adjustments.
Innovation Solution
An adjustable-focus PV camera in the HMD device uses a depth sensor to track user hand location and movements, assessing hand characteristics to control auto-focus operations, thereby reducing auto-focus hunting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If auto-focus subsystem continuously adjusts focus based on hand movements, then the camera can track user interactions, but auto-focus hunting occurs causing frequent lens motion and degraded image quality
Solution Approach 1:
The system performs preliminary hand detection and characteristic assessment before triggering auto-focus operations. By analyzing hand characteristics (speed, motion type, location) in advance, the system determines whether auto-focus should be triggered or suppressed, preventing premature or inappropriate focus adjustments that cause hunting behavior.
Solution Approach 2:
The auto-focus subsystem receives feedback from hand tracking data and adjusts its operation accordingly. When hand characteristics indicate interaction with holograms (high speed, specific motion patterns), the system suppresses auto-focus triggering. When hands are stationary or moving slowly, auto-focus is allowed to operate normally, creating a feedback loop that prevents unnecessary focus hunting.
2Ease of operation
If auto-focus subsystem frequently adjusts focus to track hand movements, then user interaction responsiveness improves, but power consumption increases
Solution Approach 1:
The system performs preliminary analysis of hand characteristics before activating auto-focus motors. By assessing hand speed, motion type, and location in advance, the system can predict whether auto-focus adjustment is necessary, avoiding unnecessary motor activation and associated power consumption while maintaining responsive user interaction when needed.
Solution Approach 2:
Instead of continuous auto-focus adjustment, the system employs periodic sampling of hand characteristics at predetermined intervals. The auto-focus subsystem is triggered only when hand characteristics exceed threshold criteria, creating a periodic rather than continuous operation mode that reduces power consumption while maintaining responsiveness to significant user interactions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An adjustable-focus PV (picture/video) camera in a mixed-reality head-mounted display (HMD) device operates with an auto-focus subsystem that is configured to be triggered based on location and motion of a user's hands to reduce the occurrence of auto-focus hunting during camera operations. The HMD device is equipped with a depth sensor that is configured to capture depth data from the surrounding physical environment to detect and track the user's hand location, movements, and gestures in three-dimensions. The hand tracking data from the depth sensor may be assessed to determine hand characteristics - such as which of the user's hands or part of a hand is detected, its size, motion, speed, etc. - within a particular region of interest (ROI) in the field of view of the PV camera. The auto-focus subsystem uses the assessed hand characteristics as an input to control auto-focus of the PV camera to reduce auto-focus hunting occurrences.