Mixed Reality Headset Field-of-View Image Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing mixed reality technologies, such as video see-through head-mounted displays, often cause a sense of incongruity during communication between users wearing HMDs and remote participants, as the remote participants cannot accurately perceive the orientation and position of the HMDs.
Innovation Solution
An information processing apparatus and method that acquire field-of-view images and person images from respective locations, calculate the difference in positions between the imaging device and the display position of the person image, and adjust the display position of the field-of-view image to align with the person image, thereby creating a combined image that simulates a shared space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the remote participant's image is displayed at a fixed position in the real space, then the display system is simple, but the remote participant cannot accurately perceive the user's orientation and position, causing a sense of incongruity
Solution Approach 1:
The patent applies the dynamics principle by making the display position of the user's image dynamic rather than fixed. The system calculates the user's field of view based on head-mounted display orientation data and dynamically adjusts the position of the user's image in the composite video to match the actual direction the user is facing. This allows the remote participant to accurately perceive the user's orientation and position, eliminating the sense of incongruity while maintaining system simplicity through software-based adjustment.
2Measurement precision
If the field of view image is not adjusted, then the processing is simple, but the user's orientation and position cannot be accurately reflected in the composite image
Solution Approach 1:
The patent applies the preliminary action principle by pre-calculating the user's field of view direction based on head-mounted display orientation data before generating the composite image. The system obtains orientation information from sensors, calculates the corresponding field of view direction, and uses this information to position the user's image correctly in advance. This preliminary calculation ensures accurate orientation perception in the final composite image while keeping the overall processing efficient through automated calculations.
3Ease of operation
If chroma-key compositing is used without position adjustment, then the compositing process is straightforward, but the remote participant experiences unnatural communication
Solution Approach 1:
The patent applies the intermediary principle by introducing field of view calculation as a mediating element between chroma-key compositing and the final image output. The system performs chroma-key compositing to combine the user's image with the remote video background, then uses the calculated field of view direction as an intermediary to adjust the position of the user's image within the composite. This intermediary adjustment ensures the user's image is positioned naturally relative to the remote participant's perspective, making communication feel more natural while building upon the straightforward chroma-key approach.
Data Source
AI summary
An information processing performs first acquisition processing of acquiring a field-of-view image representing a range corresponding to a field of view of a user in a real space in which the user is present, performs second acquisition processing of acquiring a person image that is an image obtained by capturing a person at a remote location, performs third acquisition processing of acquiring information on a difference between a position of an imaging device that transmits an image obtained by capturing the user to the remote location and a position at which the person image is displayed in the real space, and performs control processing of causing a display to display a combined image obtained by combining the person image with the field-of-view image, wherein in the control processing, a position at which the field-of-view image is displayed is shifted according to the difference.