Perspective Correction with Gravitational Smoothing for XR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted devices (HMDs) in extended reality environments often impair distance perception and cause disorientation due to the offset positions of the eyes, display, and camera, leading to poor hand-eye coordination, as the image of the physical environment presented to the user does not accurately reflect what they would see without the HMD.
Innovation Solution
The method involves capturing an image of the physical environment, generating a depth map to represent the distance of objects from the camera, smoothing the depth map to reduce artifacts, and transforming the image to appear as if captured from the user's eye perspective, using a world-fixed vector for maximal smoothing in a direction corresponding to the gravity vector.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the image is captured from the camera position and displayed directly, then the system is simple and fast, but the image does not align with the user's eye perspective causing disorientation and poor distance perception
Solution Approach 1:
The patent applies preliminary action by pre-computing transformation parameters based on the geometric relationship between camera and eye positions. These parameters are calculated in advance and stored, allowing the image transformation to be applied efficiently during operation without real-time complex calculations, thus resolving the contradiction between perspective accuracy and processing complexity
Solution Approach 2:
The patent introduces an intermediary transformation process that maps the camera-captured image to the eye-position perspective. This intermediary step uses pre-calculated geometric transformation parameters to bridge the gap between camera view and natural eye view, achieving accurate perspective alignment without requiring complex real-time processing
2Reliability
If the depth map is smoothed aggressively to reduce artifacts, then the image quality improves, but distance precision is lost
Solution Approach 1:
The patent applies local quality by implementing adaptive smoothing that varies by spatial location and depth range. Different smoothing strengths are applied to different regions of the depth map based on local characteristics, preserving distance precision in critical areas while reducing artifacts in others, thus resolving the contradiction between image stability and distance accuracy
Solution Approach 2:
The patent applies partial smoothing action selectively to specific depth ranges and spatial regions rather than uniformly across the entire depth map. This partial action approach reduces artifacts where needed while maintaining distance precision in critical regions, resolving the contradiction between image stability and distance accuracy
Data Source
AI summary
In one implementation, a method of performing perspective correction is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, an image of a physical environment. The method includes obtaining a depth map including a plurality of depths respectively associated with a plurality of pixels of the image of the physical environment. The method includes smoothing the depth map based on a world-fixed vector. The method includes transforming, using the one or more processors, the image of the physical environment based on the smoothed depth map. The method includes displaying, on the display, the transformed image.