Foveated XR Sensing for High-Resolution 3D Scanning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
XR systems face challenges in balancing the trade-offs between frame rate, spatial resolution, and data bandwidth due to the high visual acuity of users only in a small field of view, leading to inefficient power consumption and rendering inaccuracies in virtual content occlusions.
Innovation Solution
An XR system optimizes sensor performance by using eye gaze to dynamically control scanning sensors, restricting their field of view and scanning velocity to the user's focal point, and using high-resolution data only where visual acuity is high, reducing data collection and processing where acuity is low.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If scanning sensors use high spatial resolution across the entire field of view, then measurement precision is improved, but use of energy increases significantly
Solution Approach 1:
The patent applies local quality by making the scanning sensor's field of view dynamic and user-specific. Each user's foveal region is identified through eye tracking, and high-resolution scanning is applied only to that specific local region rather than uniformly across the entire field of view. This resolves the contradiction by concentrating measurement precision where needed while reducing energy consumption in peripheral regions.
Solution Approach 2:
The patent implements dynamics by making the scanning field of view adaptive and user-dependent. The system dynamically adjusts the scanning region based on real-time eye tracking data, allowing the high-resolution scanning area to move and change size according to each user's foveal position. This dynamic adaptation enables the system to maintain high measurement precision in the foveal region while minimizing energy consumption by reducing or eliminating scanning in peripheral regions.
2Productivity
If scanning velocity is increased to capture more scene data, then productivity is improved, but measurement precision deteriorates due to motion blur and sampling errors
Solution Approach 1:
The patent applies local quality by concentrating scanning resources on the foveal region where high measurement precision is critical. By restricting high-precision scanning to this small local area rather than distributing it across the entire field of view, the system can maintain high spatial resolution while increasing overall data collection efficiency through faster scanning velocity in the critical region.
3Measurement precision
If the field of view of scanning sensors is expanded to capture more of the real-world scene, then measurement precision is improved, but use of energy increases due to processing larger data volumes
Solution Approach 1:
The patent applies local quality by making the scanning field of view user-specific and focused on the foveal region. Instead of uniformly expanding the field of view across the entire scene, the system expands high-resolution scanning only to the local region where the user is looking, as determined by eye tracking. This resolves the contradiction by maintaining measurement precision in the critical foveal region while minimizing energy consumption by limiting scanning to a small portion of the overall scene.
Solution Approach 2:
The patent implements dynamics by making the scanning field of view adaptive and user-dependent. The system dynamically adjusts the scanning region based on real-time eye tracking data, allowing the field of view to expand and contract according to each user's foveal position and attention focus. This dynamic adaptation enables the system to maintain high measurement precision where needed while minimizing energy consumption by reducing scanning coverage in peripheral regions.
Data Source
AI summary
An extended Reality (XR) system that provides services for determining three-dimensional (3D) data of physical objects in a real-world scene is provided. The XR system receives a request from an application to initiate a spatial scan of a real-world scene. In response, the XR system captures video frame data of the real-world scene and captures a pose of the XR system. The XR system determines a physical object in the real-world scene and determines a two-dimensional (2D) position of the physical object, using the video frame data. The XR system determines a depth of the physical object using the 2D position and determines a 3D position of the physical object in the real-world scene using the 2D position of the physical object, the depth of the physical object, and the pose of the XR system. The XR system communicates the 3D position data to the application.