HMD Camera Calibration Using Displayed Markers for Spatial Awareness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional VR systems lack effective methods to prevent users from becoming disoriented and colliding with real-world objects while immersed in virtual environments, as they block the view of the real world and fail to provide awareness of the user's surroundings.
Innovation Solution
The method involves calibrating a head-mounted device's camera system by capturing images of known markers on a display screen, allowing the device to determine its positional and angular alignment relative to the screen, and rendering a virtual anchor object within the VR environment that corresponds to real-world objects, keeping users aware of their physical environment and obstacles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If HMD completely blocks real world view to create immersive VR experience, then user immersion in virtual environment is improved, but user awareness of real-world surroundings deteriorates
Solution Approach 1:
The patent introduces a calibration system with markers and image processing as an intermediary between the virtual and real worlds. The HMD camera captures images of known markers in the real world, and through bundle adjustment algorithms, the system computes the HMD's pose and renders virtual anchor objects that correspond to real-world positions. This intermediary calibration process enables the system to provide spatial awareness information without completely blocking the real world view, thus maintaining immersion while preserving real-world awareness.
2Ease of operation
If HMD provides immersive VR experience without real-world awareness, then virtual environment engagement is improved, but user safety deteriorates
Solution Approach 1:
The patent performs calibration actions in advance before the user begins the VR experience. The system captures multiple images of calibration markers at different positions and orientations, computes the HMD's intrinsic and extrinsic parameters through bundle adjustment, and establishes the mapping between real-world and virtual coordinate systems. This preliminary calibration enables the system to subsequently render virtual anchor objects that accurately represent real-world obstacles, providing users with spatial awareness information that helps prevent collisions while maintaining VR engagement.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A head-mounted device ("HMD") is configured to perform intrinsic and/or extrinsic calibration of the HMD's camera system by exploiting a displayed electronic image rendered on a separate display screen. A series of images are captured using one or more of the HMD's cameras. The displayed image is a known image that includes markers with known characteristics to the HMD. The known characteristics include known marker shapes and a number of coded or un-coded markers. Each image in the series captures the displayed image at a different angle or distance relative to another image in the series. The HMD then identifies, from within the series of images, two-dimensional image positions of the markers. The HMD uses the two-dimensional image positions and a determined three-dimensional position of the markers to perform a bundle adjustment used to subsequently determine a position and angular alignment of the separate display screen relative to the HMD.