XR Depth Map Re-projection for Power-Constrained Headsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Lightweight extended reality (XR) systems, such as head-mounted displays with reduced processing power and limited power management, struggle to maximize performance while minimizing power consumption and thermal output, leading to suboptimal user experiences due to limited image and depth data refresh updates.
Innovation Solution
The XR display device selectively re-projects depth maps based on image and pose data updates, calculating a suitable pose range to determine if the current image frame can be re-rendered locally, and if not, accesses stored anchors for image and depth data to adjust the frame, thereby reducing reliance on external data refreshes and conserving power.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If lightweight XR systems continuously refresh image and depth data from external devices, then image quality and user experience are improved, but power consumption and network traffic increase
Solution Approach 1:
The system pre-calculates pose ranges and stores anchor poses in advance, allowing the XR device to autonomously determine whether to request updated depth maps without continuous external communication. This preliminary preparation reduces real-time power consumption while maintaining image quality.
Solution Approach 2:
The XR display device independently evaluates whether its current pose falls within acceptable ranges using stored anchor poses and pose ranges. By self-determining when depth map updates are necessary, the device reduces reliance on continuous external data refreshes, thereby conserving power while maintaining acceptable image quality.
2Use of energy by stationary object
If lightweight XR systems reduce power consumption by reducing data refresh updates, then power consumption and thermal output are minimized, but image quality and user experience deteriorate
Solution Approach 1:
The system dynamically adjusts the frequency of depth map updates based on pose changes. When pose remains within acceptable ranges, updates are reduced or eliminated; when pose exceeds thresholds, updates are triggered. This parameter-based adaptation maintains image quality while optimizing power consumption.
Solution Approach 2:
Instead of continuously refreshing all image and depth data, the system performs partial updates only when necessary (when pose exits acceptable ranges). This selective updating approach reduces power consumption while maintaining sufficient image quality for user experience.
3Reliability
If lightweight XR systems process all image and depth data locally, then image quality is maintained, but processing power requirements and device complexity increase
Solution Approach 1:
The system segments the processing tasks between the XR display device and external rendering devices. The XR device handles lightweight pose evaluation and decision-making using stored anchor poses, while complex depth map generation and image rendering are performed externally. This segmentation reduces local processing requirements while maintaining image quality.
4Measurement precision
If lightweight XR systems continuously communicate with external devices for data updates, then image content accuracy is improved, but network traffic and power consumption increase
Solution Approach 1:
Instead of continuous communication, the system uses periodic action triggered by pose threshold exceedances. The XR device monitors pose changes and only initiates network communication when pose exits acceptable ranges, reducing network traffic while maintaining sufficient image content accuracy through timely updates.
Data Source
AI summary
A method implemented by an extended reality (XR) display device includes rendering a current image frame received from an external electronic device associated with the XR display device. The current image frame is associated with a current pose of the XR display device. The method further includes receiving an updated image frame from the external electronic device, calculating an updated pose based on one or more characteristics of the updated image frame, and determining whether the updated pose is within a pose range with respect to the current pose. The method thus further includes re-rendering, on one or more displays of the XR display device, a previous image frame based on whether the current pose is determined to be within the pose range.