Virtual Button Placement in Front of Physical Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) and mixed reality (MR) technologies face challenges in effectively integrating virtual images with physical display devices, particularly in ensuring user interaction and awareness of virtual buttons or prompts without obstructing the physical screen or causing user distraction.
Innovation Solution
An information processing apparatus that includes a processor configured to control the placement of virtual images in real space and request operations from external terminals, allowing virtual buttons to be displayed in front of physical display devices, enabling user interaction through gaze tracking and gesture recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If virtual images are disposed in front of physical display devices to enable user interaction, then user interaction capability is improved, but the physical screen may be obstructed causing user distraction
Solution Approach 1:
The patent positions virtual images in the third dimension (depth) in front of the physical display device rather than overlaying them on the two-dimensional screen. This spatial separation allows users to interact with virtual buttons through gaze tracking and gestures while the physical screen remains visible and unobstructed in the background, resolving the contradiction between interaction capability and screen obstruction.
2Ease of operation
If virtual buttons are displayed to enable interaction, then interaction accessibility is improved, but user attention may be divided causing distraction
Solution Approach 1:
The patent introduces gaze tracking as an intermediary mechanism between the user and virtual buttons. The system detects where the user is looking and presents contextual information or activates buttons based on gaze position, allowing users to interact with virtual elements without needing to consciously shift attention between multiple interfaces, thereby maintaining focus while improving accessibility.
3Adaptability or versatility
If virtual images are placed in real space for AR/MR experience, then immersive experience is improved, but integration with physical displays becomes complex
Solution Approach 1:
The patent creates a universal interaction framework that works across multiple device types (XR headsets, smartphones, tablets, PCs) by defining virtual buttons as spatial entities that can be interacted with through gaze tracking and gestures. This multi-functional approach allows the same virtual button interface to be deployed on different platforms without requiring platform-specific integration complexity, as the core interaction mechanism remains consistent across devices.
Data Source
AI summary
An information processing apparatus includes a processor configured to: control a device that disposes a virtual image in real space; and dispose, in a case where the device disposes the virtual image in front of a physical display device with an external terminal including the physical display device connected to the information processing apparatus, a button for requesting an operation of the external terminal as the virtual image.