Mixed Reality Virtual Object Display Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality systems fail to allow users to easily hold and recognize real objects while superimposed with virtual objects, as existing techniques either obscure the real object or provide poor visibility of the virtual object, making it difficult for users to perform tasks like construction and maintenance evaluation effectively.
Innovation Solution
An image processing apparatus that detects the position and orientation of real objects and determines whether the user is holding them, switching the virtual object's display form between translucent and opaque modes to allow users to recognize the real object for grasping and maintain high visibility of the virtual object once held.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Illumination intensity
If the virtual object is displayed in opaque mode to ensure high visibility, then the visibility of the virtual object is improved, but the user cannot visually recognize the shape of the real object for grasping
Solution Approach 1:
The patent applies the dynamics principle by making the display mode of the virtual object changeable based on user interaction state. The system dynamically switches between opaque display mode (when user is not holding) and translucent display mode (when user is holding), allowing the display characteristics to adapt to different operational phases. This resolves the contradiction by providing high visibility during approach and recognition, then enabling shape recognition during grasping through translucent mode.
Solution Approach 2:
The patent applies parameter changes by modifying the transparency parameter of the virtual object display based on detection results. When the user is detected to be holding the real object, the transparency parameter is increased (translucent mode) to allow shape recognition. When not holding, the transparency parameter is decreased (opaque mode) to maximize visibility. This dynamic parameter adjustment resolves the contradiction between visibility and grasping ease.
2Ease of operation
If the virtual object is displayed in translucent mode to allow recognition of the real object shape, then the ease of grasping the real object is improved, but the visibility of the virtual object is reduced
Solution Approach 1:
The system dynamically adjusts the display mode based on the user's interaction state with the real object. By detecting whether the user is holding the real object, the system switches between translucent mode (when holding) for shape recognition and opaque mode (when not holding) for maximum visibility. This dynamic adaptation resolves the contradiction by applying translucent mode only when necessary for grasping.
Solution Approach 2:
The transparency parameter of the virtual object is changed based on detection results. When the user is detected to be holding the real object, the transparency parameter is increased to allow shape recognition. This parameter change is temporary and conditional, resolving the contradiction between visibility and grasping ease by applying translucent mode only during the grasping phase.
3Illumination intensity
If the virtual object remains opaque for a predetermined time after approach to switch to translucent mode, then the visibility of the virtual object is maintained, but it becomes difficult for the user to hold the real object
Solution Approach 1:
The patent applies preliminary action by detecting whether the user is already holding the real object before switching to translucent mode. The system does not wait for a predetermined time period to elapse; instead, it proactively switches to translucent mode as soon as holding is detected. This eliminates the delay that would otherwise make it difficult to hold the real object.
Solution Approach 2:
The system uses feedback from the detection unit that detects whether the user is holding the real object to control the display mode. This real-time feedback mechanism allows the system to switch to translucent mode immediately when holding is detected, rather than waiting for a time delay. The feedback loop resolves the contradiction by making the display mode responsive to actual user interaction.
4Adaptability or versatility
If the real object is completely concealed by the virtual object image, then the feeling of holding the virtual object is enhanced, but the user cannot successfully hold the real object
Solution Approach 1:
The patent applies dynamics by making the display mode adaptive to the user's interaction state. The system transitions from opaque mode (when approaching) to translucent mode (when holding), allowing the degree of concealment to vary dynamically. This ensures the real object shape is visible when needed for successful grasping, while still providing the feeling of holding the virtual object through appropriate visual feedback.
Solution Approach 2:
The transparency parameter is changed based on detection results to balance concealment and visibility. When the user is detected to be holding the real object, the transparency parameter is increased to allow shape recognition, preventing complete concealment. This parameter adjustment ensures the user can successfully hold the real object while still experiencing the feeling of virtual object interaction.
Data Source
AI summary
An apparatus includes: a detection unit that detects a position and an orientation of a real object with respect to a viewpoint of a user; a determination unit that determines whether the user is holding the real object; and a display control unit that displays a virtual object corresponding to the real object based on the position and the orientation of the real object. When the user is not holding the real object, the display control unit displays the virtual object in a first form in which a shape of the real object can be visually recognized. When the user is holding the real object, the display control unit displays the virtual object in a second form in which the shape of the real object on which the virtual object is superimposed cannot be visually recognized.