Virtual Object UI Surface Attachment for Haptic Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality display devices face challenges in providing intuitive and accurate touch-based interaction experiences for virtual user interface elements, as they often float in mid-air without haptic feedback, lacking the familiarity and intuitiveness of physical surface interactions.
Innovation Solution
A method that determines when virtual objects and user interface elements are within a predetermined distance of a physical surface, programmatically transitioning the UI elements to be displayed on the surface, allowing users to experience haptic feedback and enhancing interaction by aligning UI elements with the surface's geometry and characteristics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If virtual user interface elements are displayed floating in mid-air, then they provide freedom of movement and spatial flexibility, but they lack haptic feedback and intuitive touch-based interaction
Solution Approach 1:
The patent introduces a physical surface as an intermediary between the virtual UI elements and the user's touch. When a virtual UI element is positioned near a physical surface, the system renders it on that surface, allowing the user to interact with it through natural touch while maintaining the virtual nature of the element. This mediator (physical surface) provides haptic feedback without constraining the virtual element's fundamental freedom of movement.
Solution Approach 2:
The system dynamically transitions UI elements between floating and surface-attached states based on proximity to physical surfaces. The UI element can float freely in space when远离 surfaces, but automatically attaches to surfaces when brought close, providing haptic feedback during interaction. This dynamic state change resolves the contradiction by adapting the interaction mode to the user's needs.
2Adaptability or versatility
If virtual objects are moved freely in three-dimensional space, then they offer immersive augmented reality experience, but user interaction becomes less familiar and intuitive
Solution Approach 1:
Physical surfaces serve as intermediaries that bridge the gap between virtual three-dimensional objects and user interaction. When users bring virtual objects close to physical surfaces, the system renders the objects on those surfaces, allowing familiar touch-based interaction while preserving the objects' three-dimensional nature and freedom of movement when远离 surfaces.
Solution Approach 2:
The interaction space is segmented into two zones: a floating zone where virtual objects move freely in three-dimensional space, and a surface-attached zone where objects attach to physical surfaces for familiar interaction. This segmentation allows the system to provide both immersive three-dimensional experience and familiar touch interaction depending on the user's actions.
3Ease of operation
If user interface elements are displayed on physical surfaces, then haptic feedback and touch-based interaction are provided, but the elements lose spatial freedom and become constrained
Solution Approach 1:
The UI elements dynamically transition between floating and surface-attached states. When users move virtual elements away from physical surfaces, they regain spatial freedom and can be repositioned anywhere in the three-dimensional space. When brought close to surfaces, they automatically attach to provide haptic feedback. This dynamic behavior ensures elements maintain adaptability while providing familiar interaction when needed.
Solution Approach 2:
The system periodically evaluates the distance between virtual UI elements and physical surfaces, automatically transitioning the elements between floating and attached states based on proximity thresholds. This periodic assessment ensures elements maintain optimal interaction mode while preserving spatial freedom when远离 surfaces.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Methods, computing devices and head-mounted display devices for displaying user interface elements with virtual objects are disclosed. In one example, a virtual object and one or more user interface elements are displayed within a physical environment. User input is received that moves one or more of the virtual object and the one or more user interface elements. One or more of the virtual object and the one or more user interface elements are determined to be within a predetermined distance of a physical surface. Based at least on this determination, the one or more user interface elements are displayed on the surface.