MR Composite Image Processing for Hidden-Object Contact Alerts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In MR systems, users cannot accurately determine the distance between their real hand and an obscured real object due to hand masking techniques, leading to a diminished sense of immersion and potential operational interruptions.
Innovation Solution
An information processing apparatus and method that acquires a captured image, generates a virtual object, combines it with the real image, and controls display to highlight the hand or specific object based on distance, using transmittance changes, vibration, or sound notifications to indicate potential contact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hand masking technique is used to occlude real hand behind virtual CG object, then the user can recognize front/back relationship and feel immersion, but the user cannot see the real object and cannot recognize distance or possibility of contact
Solution Approach 1:
The patent changes the color or luminance of the virtual CG object to notify the user of the possibility of contact with real objects. When the hand approaches a real object that is occluded by the CG object, the system detects the proximity and changes the visual properties (color, brightness) of the CG object to alert the user, thus preserving immersion while providing distance information.
2Reliability
If hand masking technique is used to mask real hand by virtual CG object, then the user can feel as if virtual CG actually exists, but the user cannot sense the possibility of contact with real objects
Solution Approach 1:
The patent implements a feedback mechanism where the system continuously monitors the position of the user's hand relative to real objects and provides visual feedback by changing the appearance of the CG object. When contact is possible, the CG object's color or luminance changes to alert the user, enabling contact possibility detection while maintaining the realism of the virtual object.
Data Source
AI summary
An information processing apparatus acquires a captured image capturing a real space including a first object and a second object. The information processing apparatus generates a virtual object. The information processing apparatus generates a composite image by combining the captured image and the virtual object. The information processing apparatus controls a display such that the composite image is displayed. In a first case where the virtual object is superimposed on a front side of the second object in a space expressed by the composite image, the information processing apparatus performs notification to a user in accordance with the distance between the first object and the second object.