Mixed Reality Viewpoint Prediction for Registration Error Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Mixed Reality (MR) systems face dynamic registration errors when composing three-dimensional virtual space images with physical space images, leading to inconsistencies in stereoscopic angles and impaired sense of reality due to time delays in detecting and generating images.
Innovation Solution
A mixed reality presentation system that includes a head-mounted display and an image processing apparatus, which calculates and transmits difference information between imaging and generation times, predicts the viewpoint's position and orientation, and synchronizes the generation of virtual space images with physical space images to minimize registration errors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a CG image is composited on a physical space image requiring little processing time, then processing speed is improved, but dynamic registration error occurs between images
Solution Approach 1:
The system performs preliminary actions by detecting the viewpoint position and orientation in advance, calculating the required CG image generation time, and predicting the viewpoint state at the completion time. This allows the CG image to be generated with proper timing and perspective compensation before composition, resolving the contradiction between fast processing and accurate registration.
2Reliability
If the viewpoint position and orientation are detected and CG image is generated in real time, then the sense of reality is enhanced, but processing time increases causing dynamic registration error
Solution Approach 1:
The system detects viewpoint information in advance and uses it to predict the viewpoint state at CG generation completion. This preliminary detection and prediction approach allows real-time generation with proper timing compensation, maintaining the sense of reality while managing processing time effectively.
Solution Approach 2:
The system dynamically adjusts the composition timing by calculating the actual generation completion time of the CG image and compensating for viewpoint changes that occur during processing. This dynamic compensation maintains registration accuracy despite variable processing times, preserving the sense of reality.
3Device complexity
If a CG image is merely shifted in horizontal and vertical directions to match head movement, then processing is simplified, but inconsistency occurs between stereoscopic angle and composition position for three-dimensional objects
Solution Approach 1:
The system dynamically determines the composition position of the CG image based on the viewpoint position and orientation at the generation completion time, rather than using simple horizontal and vertical shifts. This dynamic positioning approach maintains proper stereoscopic angles and spatial relationships for three-dimensional objects while adapting to viewpoint changes.
Data Source
AI summary
An image composition unit outputs a composition image of a physical space and virtual space to a display unit. The image composition unit calculates, as difference information, a half of the difference between an imaging time of the physical space and a generation completion predicted time of the virtual space. The difference information and acquired position and orientation information are transmitted to an image processing apparatus. A line-of-sight position prediction unit updates previous difference information using the received difference information, calculates, as the generation completion predicted time, a time ahead of a receiving time by the updated difference information, and predicts the position and orientation of a viewpoint at the calculated generation completion predicted time using the received position and orientation information. The virtual space based on the predicted position and orientation, and the generation completion predicted time are transmitted to a VHMD.