Mixed Reality Frame Synchronization for Reduced Display Discomfort
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality systems experience delays in displaying virtual space images, causing discomfort due to discrepancies between user movement and displayed images, especially when shared among multiple users.
Innovation Solution
A display system that synchronizes the delay of real and virtual space images by combining real and virtual space images at specific frames, ensuring both types of images have the same delay, thereby reducing user discomfort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If the real space image is displayed with minimized delay, then the user comfort is improved, but the synchronization between real and virtual space images deteriorates
Solution Approach 1:
The system dynamically adjusts the delay of the virtual space image based on the delay of the real space image. The image generation unit generates the virtual space image with a delay matching the real space image delay, ensuring synchronization while allowing the real space image to have minimized delay for user comfort.
2Measurement precision
If the virtual space image is generated with current HMD position and orientation, then the image accuracy is improved, but the display delay increases causing motion sickness
Solution Approach 1:
The system performs preliminary action by generating the virtual space image based on the HMD position and orientation at a previous time point rather than waiting for current data. This reduces display delay and prevents motion sickness, while still providing accurate spatial representation for the user's current viewpoint.
3Adaptability or versatility
If the mixed reality space image is distributed to multiple devices, then the collaboration capability is improved, but the delay synchronization between devices deteriorates
Solution Approach 1:
The system creates copies of the mixed reality space image with consistent delay characteristics and distributes them to multiple devices. Each device receives and displays the image with the same delay as the HMD wearer, ensuring delay synchronization across all collaborative participants while maintaining full collaboration capability.
Data Source
AI summary
A display system includes a processor and a memory coupled to the processor and storing instructions that, when executed by the processor, cause the processor to function as: a position-orientation acquisition unit that acquires a position and orientation of the image pickup apparatus based on a first frame of the real space image, a generation unit that generates a first frame of the virtual space image on the basis of the position and orientation of the image pickup apparatus based on the first frame of the real space image, a first image combining unit that combines a frame different from the first frame of the real space image with the first frame of the virtual space image, and a second image combining unit that combines the first frame of the real space image with the first frame of the virtual space image.