MR Display Image Correlation for Accurate Gaze Object Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR technologies struggle to accurately identify whether a user is observing the real space or the CG region, leading to difficulties in managing data processing based on line-of-sight detection results.
Innovation Solution
An information processing system that generates a virtual image, combines it with a real image, and correlates input information with time-stamped data to accurately track user interactions, using a head-mounted display and generation device to manage data processing effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If line-of-sight detection is performed in MR images combining real and CG regions, then user observation tracking is enabled, but accurate identification of observed objects becomes difficult due to time lags between real image and CG image generation
Solution Approach 1:
The patent segments the display image into multiple regions: a first region corresponding to the real space captured image and a second region corresponding to the CG virtual image. This segmentation allows independent tracking and identification of objects in each region, resolving the confusion caused by time lags between real and CG image generation. The line-of-sight detection results are then correlated with the specific region being observed.
Solution Approach 2:
The patent introduces an intermediary mechanism - a database that stores correspondence relationships between display images and their constituent regions (real image regions and CG image regions). This intermediary allows the system to trace line-of-sight detection results back to the specific source image and region, enabling accurate object identification even when time lags exist between image generations.
2Productivity
If MR images fuse real space and CG in real time, then seamless mixed reality experience is achieved, but data management for processing based on user input becomes complex
Solution Approach 1:
The patent performs preliminary actions by pre-establishing and storing the correspondence relationships between display images and their constituent regions in a database before line-of-sight detection occurs. This preliminary structuring of data allows for rapid, straightforward processing during actual use, as the system only needs to query the pre-established correspondence rather than performing complex analysis in real-time.
Data Source
AI summary
An information processing system includes one or more processors and/or circuitry configured to: execute generation processing for generating a virtual image by rendering a virtual object based on a reference position determined at a first time; execute combining processing for generating a display image by combining the virtual image and a first image; execute display control processing for displaying the display image on a display at a second time; execute input processing for acquiring input information from a user for the display at the second time; and execute control processing in which the input information, the display image, information about the reference position, first piece of information which is time information related to the reference position, and second piece of information which is time information related to the first image, are correlated with each other and stored in a storage device.