Head Mounted Display Viewpoint Alignment via Predictive Shifting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compositing physical and virtual space images in Mixed Reality (MR) techniques suffer from delays in displaying composite images due to processing time and errors in viewpoint alignment, leading to an unnatural observer experience, especially when the observer's viewpoint moves quickly.
Innovation Solution
A head-mounted display system that includes an image sensing device, a display device, and a position and orientation sensor, which acquires and transmits position and orientation information to an external computer for generating a virtual space image. The system calculates differences in orientation and position between sensed and displayed images, shifting either the virtual or physical space image based on resolution and angle-of-view information to align them accurately and reduce composition delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a CG image is created in advance based on predicted viewpoint movement and composited on a physical space image, then the composition time is shortened, but an error occurs between the viewpoint in the CG image and that in the physical space image, causing an unnatural impression
Solution Approach 1:
The patent applies preliminary action by creating the CG image in advance based on predicted viewpoint movement before the actual composition occurs. The system predicts the viewpoint movement amount and generates the CG image corresponding to the future viewpoint position, allowing the composition to be performed quickly without real-time processing delays while maintaining viewpoint alignment through the prediction mechanism
Solution Approach 2:
The patent uses feedback by comparing the actual viewpoint information at composition time with the predicted viewpoint information used for CG image creation. The system calculates the difference between these viewpoints and adjusts the composition accordingly, using the actual viewpoint as feedback to correct any prediction errors and maintain accurate alignment
2Speed
If the viewpoint moves quickly, then the delay in displaying the composite image becomes conspicuous, but creating a CG image in advance based on predicted movement causes viewpoint errors
Solution Approach 1:
The patent applies dynamics by making the CG image creation process adaptive to the actual viewpoint movement. Instead of using a fixed prediction model, the system dynamically adjusts the composition based on the actual viewpoint information, allowing it to handle rapid viewpoint changes effectively while maintaining viewpoint consistency through real-time feedback
Data Source
AI summary
A viewpoint information calculation unit (710) calculates a difference value (α) between reference position and orientation information and position and orientation information held by a camera viewpoint storage unit (207), and a difference value (β) between the reference position and orientation information and position and orientation information held by a CG viewpoint storage unit (208). If |α|≧|β|, a physical space image is shifted by a shift amount calculated in accordance with the resolution of a display unit (213), a difference amount, and the angle of view of a camera (203), and is composited with a virtual space image. If |α|<|β|, the virtual space image is shifted by the shift amount, and is composited with the physical space image.