Gaze-Adaptive Depth Sensor Control for Extended Reality Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality devices lack real-time adaptability and flexibility in controlling sensing apparatuses such as eye-tracking and depth sensors due to independent and inflexible control strategies, limiting their applicability in diverse application scenarios.
Innovation Solution
A method and apparatus for controlling sensing apparatuses by capturing image data, displaying extended reality images, determining a user's gaze area, and adjusting parameters like modulation frequency, exposure time, and frame rate based on the gaze area to enhance adaptability and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If independent control strategies are used for eye-tracking apparatus and depth sensor, then device complexity is reduced, but adaptability and flexibility are worsened
Solution Approach 1:
The patent combines the control of the depth sensor and eye-tracking apparatus into a unified control strategy. The processor integrates gaze information from the eye-tracking apparatus with depth sensing control, merging previously independent control systems into a coordinated system that shares control parameters and decision-making logic.
Solution Approach 2:
The control strategy is designed to serve multiple functions simultaneously: it controls both the depth sensor and eye-tracking apparatus, adapts to different viewing conditions, and optimizes for both power consumption and image quality. This universal control mechanism replaces multiple specialized control strategies.
2Device complexity
If fixed control parameters are used for depth sensor, then device complexity is reduced, but adaptability to different scenes is worsened
Solution Approach 1:
The control parameters for the depth sensor are made dynamic rather than fixed. The processor continuously adjusts parameters such as exposure time, gain, and frame rate based on real-time gaze information and viewing conditions. This dynamic adaptation allows the system to optimize performance for different scenes and user interactions.
Solution Approach 2:
The system implements a feedback loop where gaze information from the eye-tracking apparatus feeds into the depth sensor control. The processor uses this feedback to continuously adjust control parameters, creating a closed-loop system that adapts to changing conditions in real-time.
3Measurement precision
If high frame rate and long exposure time are used for depth sensor, then measurement precision is improved, but power consumption is worsened
Solution Approach 1:
Instead of continuously operating at maximum performance, the system applies partial action by adjusting control parameters based on actual needs. When gaze stability indicates lower precision requirements, the system reduces frame rate or exposure time, applying only the necessary amount of sensing effort rather than excessive continuous high-performance operation.
Solution Approach 2:
The system dynamically changes control parameters (frame rate, exposure time, gain) based on gaze stability and scene requirements. This parameter adaptation allows the system to achieve high measurement precision only when necessary, while reducing power consumption during periods when lower precision suffices or when the user is not actively interacting with the content.
Data Source
AI summary
A method for controlling a sensing apparatus, an electronic device, and a storage medium are provided. The method includes: capturing image data of a real environment; displaying an extended reality image generated based on the image data; determining a gaze area of a user on the extended reality image; and adjusting a control parameter of a depth sensing apparatus based on the gaze area, where the control parameter includes at least one of a modulation frequency, exposure time, a frame rate, or a frame-rate proportion.