VST XR Camera Offset Mapping for Wider Passthrough Views
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video see-through (VST) extended reality (XR) systems are limited by the assumption that see-through cameras are positioned directly in front of the user's eyes and pointing straight ahead, restricting device configuration and field of view.
Innovation Solution
Implementing see-through cameras that are offset or angled relative to the forward axes of the user's eyes, using passthrough transformations based on first and second transformations to align camera and display panel principal points, and applying rectification to map virtual camera image frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If see-through cameras are positioned directly in front of user's eyes and pointing straight ahead, then device configuration is simplified, but field of view is limited
Solution Approach 1:
The patent changes the spatial parameters of camera placement by positioning cameras at offset locations and/or rotating them to angled orientations relative to the forward axes extending from expected eye locations. This allows the system to achieve wider fields of view while maintaining proper image alignment through computational transformations.
Solution Approach 2:
The patent introduces passthrough transformations as an intermediary computational step that maps images from the offset/angled camera viewpoints to the expected eye viewpoints. This mediator transformation resolves the contradiction by enabling diverse camera configurations while maintaining simplified user experience through automated image warping and alignment.
2Adaptability or versatility
If see-through cameras are offset from or angled relative to forward axes, then device configuration diversity is enabled, but image alignment becomes more complex
Solution Approach 1:
The patent employs passthrough transformations as an intermediary computational layer that handles the complexity of mapping images from offset/angled camera positions to the expected eye positions. This mediator transformation encapsulates the geometric complexity while allowing diverse device configurations to be implemented.
Solution Approach 2:
The patent replaces mechanical alignment solutions with computational image processing. Instead of physically positioning cameras perfectly aligned with eye axes, the system uses software-based passthrough transformations to correct misalignment, substituting mechanical precision requirements with computational methods.
3Ease of manufacture
If principal points of cameras and display panels are not aligned, then device manufacturing is easier, but image quality and registration accuracy deteriorate
Solution Approach 1:
The patent introduces passthrough transformations as an intermediary that aligns principal points of cameras with display panel principal points. This transformation acts as a mediator that compensates for manufacturing tolerances and misalignments, enabling easier manufacturing while maintaining precise image registration through computational correction.
Data Source
AI summary
A method includes identifying a passthrough transformation associated with a VST XR device. The VST XR device includes see-through cameras that are (i) offset from forward axes extending from expected locations of a user's eyes and/or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes. The method also includes obtaining images captured using the see-through cameras, applying the passthrough transformation to the images to generate transformed images, and displaying the transformed images on one or more display panels of the VST XR device. The passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.