Opaque HMD AR Conversion via Dynamic Transparency and Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mounted displays (HMDs) designed for virtual reality (VR) often fail to seamlessly transition into augmented reality (AR) applications, lacking transparent or translucent displays that allow users to see the real world, and existing AR solutions rely on cumbersome markers for accurate virtual object placement.
Innovation Solution
The implementation of systems and methods that enable AR functionality in opaque HMDs using RGB or RGBD cameras, inertial measurement units, and advanced pose estimation techniques for accurate camera localization, allowing for real-time display of virtual objects without jitter or environmental modifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If opaque screens are used in HMDs for VR applications, then user immersion in virtual reality is improved, but the ability to see the real world and use the device for AR applications is lost
Solution Approach 1:
The patent applies dynamics by making the display characteristics changeable through software control. The HMD can dynamically switch between opaque mode (for VR) and transparent/translucent mode (for AR) using electronically controllable displays such as LCD panels that can adjust their light transmission properties, allowing the same device to adapt to different application requirements
2Adaptability or versatility
If transparent or translucent displays are used in HMDs for AR applications, then real-world visibility is improved, but user immersion and isolation from the real world deteriorates
Solution Approach 1:
The system dynamically adjusts display transparency based on the active application mode. When VR mode is activated, the display becomes opaque to provide full immersion. When AR mode is activated, the display becomes transparent or translucent to allow real-world visibility. This dynamic adaptation resolves the contradiction between immersion and real-world visibility
3Measurement precision
If markers are used for virtual object placement in AR, then placement accuracy is improved, but system complexity and user burden increase
Solution Approach 1:
The patent extracts and removes the marker dependency from the AR system. Instead of requiring external markers for virtual object placement, the system uses the HMD's camera to capture and process visual features directly from the environment, eliminating the need for separate marker components and simplifying the overall system
Solution Approach 2:
The HMD device performs self-localization and virtual object placement using its own onboard camera and processing capabilities. The system captures images, identifies environmental features, determines device pose, and places virtual objects without requiring external marker systems or additional辅助设备, making the device self-sufficient
Data Source
AI summary
While many augmented reality systems provide “see-through” transparent or translucent displays upon which to project virtual objects, many virtual reality systems instead employ opaque, enclosed screens. Indeed, eliminating the user's perception of the real-world may be integral to some successful virtual reality experiences. Thus, head mounted displays designed exclusively for virtual reality experiences may not be easily repurposed to capture significant portions of the augmented reality market. Various of the disclosed embodiments facilitate the repurposing of a virtual reality device for augmented reality use. Particularly, by anticipating user head motion, embodiments may facilitate scene renderings better aligned with user expectations than naïve renderings generated within the enclosed field of view. In some embodiments, the system may use procedural mapping methods to generate a virtual model of the environment. The system may then use this model to supplement the anticipatory rendering.