HMD Scene Camera Retargeting for Accurate Depth Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
HMDs often impair distance perception, cause disorientation, and result in poor hand-eye coordination due to the mismatch between the positions of the eyes, display, and camera, leading to distorted real-world images.
Innovation Solution
Implementing a planar world assumption to transform captured images using homography, adjusting for the difference in viewpoints between the scene camera and the user's eyes, and compensating for depth discrepancies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the scene camera captures images from its position and displays them directly to the user, then the system structure is simple, but the user experiences impaired distance perception, disorientation, and poor hand-eye coordination due to the mismatch between camera position and eye position
Solution Approach 1:
The patent introduces an intermediary computational process (homography transformation) that acts as a mediator between the camera capture and display stages. This transformation uses the calibrated relationship between camera and display positions to warp the captured image, creating a virtual view that compensates for the spatial mismatch. The intermediary process resolves the contradiction by adding computational complexity to eliminate the harmful effects of simple direct display.
Solution Approach 2:
The patent applies parameter changes by transforming the image coordinates through homography matrices that encode the spatial relationship between camera and display. By changing the coordinate parameters of the displayed image based on calibrated transformation matrices, the system adjusts the visual output to match what the user would see from their actual eye position, thereby improving spatial perception without requiring physical repositioning of components.
2Ease of operation
If the display position is moved closer to the user's eyes to improve viewing alignment, then hand-eye coordination improves, but the field of view and comfort are reduced
Solution Approach 1:
The patent resolves this contradiction by operating in the dimension of image transformation rather than physical repositioning. Through homography warping, the system creates a virtual display position that can be arbitrarily positioned in 3D space relative to the user's eyes. This allows the effective display position to be optimized for hand-eye coordination while the physical display maintains its position for field of view and comfort, thus solving the problem by transitioning from physical to virtual positioning.
3Measurement precision
If the camera position is adjusted to match the user's eye position, then distance perception improves, but the device structure becomes more complex and the camera cannot remain fixed
Solution Approach 1:
The patent creates a virtual copy of the ideal camera position (at the user's eye position) through computational image transformation. Instead of physically moving the camera to match the eye position, the system captures images from the fixed camera position and then applies homography transformations to create a virtual view that appears as if captured from the eye position. This copying approach achieves accurate distance perception without the complexity of movable camera mechanisms.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the accuracy of distance perception and hand-eye coordination by aligning the displayed image with the user's natural view, reducing disorientation and improving interaction with the real world.
Implementation Method 1
a scene camera that captures an image of the environment of the user
Implementation Method 2
a display that displays the image to the user
Data Source
AI summary
In one implementation, a method of scene camera retargeting is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes determining, using the one or more processors, a depth of the scene. The method further includes transforming, using the one or more processors, the image of the scene based on the depth and a difference between a point-of-view of the scene camera and a point-of-view of a user. The method further includes displaying, on the display, the transformed image.