Mixed Reality Line-of-Sight Image Composition for Clear Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR techniques degrade image quality when reproducing three-dimensional models, leading to communication discrepancies between users sharing a mixed reality space, affecting the immersive experience.
Innovation Solution
An information processing apparatus that acquires and processes line-of-sight images and positional data to generate a combined image, ensuring that the line-of-sight images of users overlap according to predetermined criteria, allowing for accurate representation of real objects as virtual objects in mixed reality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If three-dimensional models are reproduced in real time using a large amount of three-dimensional information, then the reproduction speed is improved, but the image quality degrades
Solution Approach 1:
The system segments the viewing experience into two parts: the local user views the high-quality original three-dimensional model, while remote users receive two-dimensional images captured from the local user's viewpoint. This segmentation allows each user to receive appropriately optimized content without compromising overall system performance.
Solution Approach 2:
Instead of transmitting and rendering full three-dimensional models to all users, the system creates two-dimensional image copies from the local user's camera viewpoint and transmits these to remote users. This copying approach maintains visual fidelity while dramatically reducing computational requirements for real-time reproduction.
2Loss of information
If the screen is changed to be displayed on the HMD to show line-of-sight images, then communication clarity is improved, but the feeling of immersion degrades
Solution Approach 1:
The system applies different quality treatments to different parts of the display: the line-of-sight image is displayed with high quality and appropriate size only when it overlaps with the user's viewpoint, while other areas maintain the original three-dimensional model rendering. This local quality adjustment ensures communication clarity is improved only where necessary without compromising overall immersion.
3Productivity
If degraded image quality is used in three-dimensional models, then processing speed is improved, but communication effectiveness deteriorates
Solution Approach 1:
The local user's camera acts as an intermediary, capturing real-world views that are then transmitted to remote users. This intermediary approach allows remote users to see high-quality images of real objects without requiring the system to transmit and render complex three-dimensional models, thus maintaining both processing speed and communication effectiveness.
Data Source
AI summary
An information processing apparatus for controlling display of a display apparatus worn by a user includes an acquisition unit configured to acquire a line-of-sight image of another user wearing another display apparatus and information about a virtual object corresponding to a real object disposed in a space where the other user exists, and a generation unit configured to generate a combined image including at least the line-of-sight image of the other user when a line-of-sight of the user to the virtual object displayed on the display apparatus and a line-of-sight of the other user to the real object satisfy a predetermined criterion.