Foveated Rendering Image Display Apparatus for Mixed Reality Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality (MR) systems face challenges in accurately aligning physical and virtual spaces in real-time, particularly when using video see-through methods, leading to reduced precision and stability in overlapping these spaces due to high transmission loads and resolution reduction in non-gaze areas.
Innovation Solution
An image display apparatus and method that acquires position and orientation information from captured images, generates reduced-information-amount images based on the user's gaze distance, and outputs this information to an external device for composite image generation, reducing transmission load while maintaining alignment precision.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-resolution captured images are transmitted to the external device for composite image generation, then the alignment precision between physical and virtual spaces is improved, but the transmission load increases
Solution Approach 1:
The patent applies local quality by transmitting captured images at high resolution only in the gaze region where the user is looking, while transmitting at lower resolution in non-gaze regions. This selective resolution approach maintains alignment precision in the critical foveal area while reducing overall transmission load, directly resolving the contradiction between precision and data quantity.
Solution Approach 2:
The patent segments the captured image into gaze region and non-gaze region, applying different resolution treatments to each segment. The gaze region is processed at high resolution to maintain alignment precision, while non-gaze regions are downsampled to reduce transmission load, thereby resolving the technical contradiction through spatial segmentation.
2Productivity
If the captured image resolution is reduced to shorten rendering time, then the processing speed is improved, but the alignment precision between physical and virtual spaces deteriorates
Solution Approach 1:
The patent applies local quality by maintaining high resolution only in the gaze region where alignment precision is critical, while reducing resolution in non-gaze regions where precision requirements are lower. This selective approach preserves necessary alignment precision while improving overall processing speed through reduced data volume.
3Manufacturing precision
If high-quality captured images are transmitted, then the image quality in the composite image is improved, but the transmission capacity is exceeded
Solution Approach 1:
The patent applies local quality by transmitting high-quality images only in the gaze region where the user is looking, while transmitting lower-quality images in non-gaze regions. This selective quality approach maintains excellent image quality in the critical viewing area while staying within transmission capacity limits by reducing data in less critical areas.
Data Source
AI summary
Information representing a position and orientation of a captured image or information for deriving the position and orientation is acquired from the captured image as extraction information, and a reduced-information-amount image is generated by reducing an amount of information in the captured image in accordance with a distance from a position that a user is gazing at in the captured image. The reduced-information-amount image and the extraction information are outputted to an external device, and a composite image that has been generated based on the reduced-information-amount image and an image of a virtual space generated by the external device based on the extraction information is received.