HMD List Image Layout for Mixed Reality Depth Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In mixed reality systems, displaying information about parts of a virtual object superimposed on a physical space can cause discomfort due to occlusion and limited display area, especially when using stereoscopic vision with Head-Mounted Displays (HMDs).
Innovation Solution
An image processing method and apparatus that acquires the position and orientation of a viewpoint and a pointing device, generates a virtual image of a virtual object, and composites it with the physical image, allowing for the display of part information without occluding the sense of depth, by laying out a list image near the pointing device and determining its layout position based on the observer's viewpoint and pointing device position.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If information about parts is displayed in the mixed reality system, then the observer can access part information, but the display area is limited and causes discomfort in stereoscopic vision
Solution Approach 1:
The patent transitions from 2D display constraints to 3D spatial display by positioning the list image at a specific location in the 3D virtual space near the pointed object. The list image is rendered with depth information and can be viewed from different angles, effectively utilizing the third dimension to resolve the display area limitation while maintaining stereoscopic comfort.
Solution Approach 2:
The patent introduces a list image as an intermediary element between the pointed object and the observer. This list image serves as a mediator that presents part information in a structured format near the object of interest, reducing the cognitive load and visual discomfort by organizing information spatially rather than forcing the observer to switch between 3D viewing and 2D information display.
2Loss of information
If a list image is displayed near the pointed position, then part information is accessible, but the list image may occlude other images and damage the sense of depth
Solution Approach 1:
The patent applies transparency and color adjustments to the list image to reduce its visual impact. The list image is rendered with transparent or semi-transparent background and adjusted brightness/color properties, allowing it to blend with the surrounding 3D environment. This reduces occlusion effects and maintains the sense of depth by preventing the list image from completely blocking underlying visual information.
3Loss of information
If the display area of HMD is used to show part information, then information is presented, but the limited display area causes discomfort
Solution Approach 1:
The patent exploits the 3D display capability of the HMD to position the list image in virtual space near the pointed object rather than confining it to the limited 2D screen area. This spatial positioning allows the list image to be viewed in context with the 3D environment, effectively expanding the information display area without increasing physical display dimensions.
Data Source
AI summary
The position and orientation of the viewpoint of an observer (100) are acquired. The position and orientation of a stylus (120) are acquired. A list image is laid out near the position of the stylus (120). An image of a virtual space after laying out the list image, which is seen in accordance with the position and orientation of the viewpoint, generated. The generated image is output to the display screen of an HMD (110).