Mixed Reality Display Synchronization to Reduce Motion Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality systems experience delays in displaying virtual space images due to calculations of head-mounted display position and orientation, causing user discomfort and motion sickness.
Innovation Solution
A display system that synchronizes the display of real and virtual space images by acquiring the position and orientation of an image pickup apparatus, generating virtual space images based on this data, and combining them with real space images to minimize delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the real space image is displayed with minimized delay, then user comfort is improved, but the delay between real and virtual space images becomes different causing visual inconsistency
Solution Approach 1:
The patent divides the mixed reality space image into two separate components: a first mixed reality space image combining real and virtual images with different delays optimized for user comfort, and a second mixed reality space image combining them with matched delays for consistency. This segmentation allows each component to serve its specific purpose without compromising the other.
Solution Approach 2:
The patent merges two different image combination approaches into a single system: one combination prioritizes minimizing real space image delay for user comfort, while the other prioritizes matching delays between real and virtual images. Both combined images are generated and distributed simultaneously to different destinations.
2Measurement precision
If virtual space image generation uses current HMD position and orientation, then image accuracy is improved, but calculation and rendering time increases causing delay
Solution Approach 1:
The patent performs preliminary action by generating the virtual space image based on the HMD position and orientation at the time of capture, rather than waiting for the latest position and orientation data. This allows the virtual image generation to proceed without delay while maintaining sufficient accuracy for the application.
3Adaptability or versatility
If the mixed reality space image is distributed to multiple devices, then collaboration capability is improved, but the movement of shared image does not reflect viewer movement causing disorientation
Solution Approach 1:
The patent applies different quality characteristics to different distributed images: the first mixed reality space image is optimized for the HMD wearer with minimized real space image delay, while the second mixed reality space image is optimized for other devices with matched delays. Each image has local quality tailored to its specific use case and viewer context.
Data Source
AI summary
A display system includes a processor and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the processor to function as: a position-orientation acquisition unit that acquires a position and orientation of the image pickup apparatus based on a first frame of the real space image, a generation unit that generates a first frame of the virtual space image on the basis of the position and orientation of the image pickup apparatus based on the first frame of the real space image, a first image combining unit that combines a frame different from the first frame of the real space image with the first frame of the virtual space image, and a second image combining unit that combines the first frame of the real space image with the first frame of the virtual space image.