Mixed Reality Virtual Object Display Control for Unpredictable Movement

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In mixed reality (MR) experiences, users cannot predict the movement of virtual objects superimposed on real objects held by others, leading to unexpected and surprising displays that can inhibit the MR experience.

Innovation Solution

An information processing apparatus that acquires the position and orientation of a display apparatus and a target virtual object, and controls the display method based on this information to prevent inhibition by changing the transparency or appearance of the virtual object when it approaches the user's field of view unexpectedly.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If virtual objects are superimposed on real objects held by other users in MR, then the immersion and interactivity of MR are enhanced, but the virtual objects may move unpredictably and appear suddenly in the user's field of view, causing surprise and inhibiting the MR experience

Engineering Contradiction:

ImproveMR immersion and interactivityVSAvoidUnexpected virtual object appearance causing user surprise

Core Design Contradiction:

Adaptability or versatilityVSObject-affected harmful factors

Solution Approach 1:

The system performs preliminary actions by detecting the position and orientation of the display apparatus and predicting where virtual objects will appear before they actually appear. Warning information is displayed in advance at the predicted location, allowing users to prepare for the upcoming virtual object appearance rather than being surprised by sudden appearances.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback by continuously monitoring the display apparatus position, calculating the predicted virtual object appearance location, and providing warning information to the user. This feedback loop enables the system to adapt to changing user positions and orientations, maintaining effective warning coverage as users move within the MR environment.

Inventive Principle:

Principle #23Feedback

2Reliability

If the position and orientation of the display apparatus are tracked to predict virtual object movement, then unexpected appearances can be anticipated, but the system complexity increases

Engineering Contradiction:

ImprovePrediction accuracy of virtual object appearanceVSAvoidSystem complexity for tracking and calculation

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The system introduces an intermediary processing layer that receives display apparatus position and orientation data, calculates the predicted virtual object appearance location based on the relationship between these parameters, and generates warning information. This intermediary layer simplifies the overall system architecture by centralizing the prediction logic and decoupling the tracking functionality from the warning display functionality.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11527020B2Information processing apparatus, information processing method, and storage medium

Publication Date: 2022.12.13 CANON KK

AI summary

An information processing apparatus includes a first acquisition unit configured to acquire a position and orientation of a display apparatus capable of displaying one or more virtual objects, a second acquisition unit configured to acquire a position and orientation of a target virtual object out of the one or more virtual objects, and a control unit configured to determine a method of displaying the target virtual object on the display apparatus based on the position and orientation of the display apparatus and the position and orientation of the target virtual object.