Adaptive Tangible Interfaces for XR Object-Based Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality systems lack intuitive and adaptable user interfaces that allow users to interact with virtual content using tangible objects, as mid-air hand gestures provide no tactile feedback and purpose-built input devices are cumbersome.
Innovation Solution
An extended reality system that detects and identifies existing physical objects in the user's environment, maps UIs to these objects based on their affordances, and dynamically adjusts the interfaces to suit changing user actions and locations, enabling control through logical manipulation of mapped objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If mid-air hand gestures are used for interaction, then immersion in virtual environment is improved, but tactile feedback is lost
Solution Approach 1:
The patent introduces physical objects from the real world as intermediaries between the user and virtual content. These objects serve as mediators that provide tactile feedback while enabling interaction with extended reality content, thus resolving the contradiction between immersion and tactile feedback.
Solution Approach 2:
The patent replaces the mechanical hand gesture system with a system that incorporates physical objects with tangible properties. This substitution allows users to interact with virtual content through physical manipulation of real-world objects, restoring tactile feedback while maintaining immersion.
2Ease of operation
If purpose-built input devices are used, then interaction control is improved, but portability and convenience are reduced
Solution Approach 1:
The patent makes physical objects universal by enabling them to serve multiple functions - their original real-world function plus user interface function for interacting with extended reality content. This eliminates the need for dedicated input devices while maintaining interaction control.
Solution Approach 2:
The patent enables existing physical objects to serve themselves as user interfaces without requiring separate dedicated input devices. The objects in the environment automatically become interactive elements, providing both their original function and UI functionality, thus improving portability while maintaining interaction control.
3Device complexity
If fixed user interfaces are implemented, then system complexity is reduced, but adaptability to changing contexts is lost
Solution Approach 1:
The patent implements dynamic user interfaces that automatically adapt to changing contexts by detecting physical objects in the environment and composing appropriate interfaces based on object affordances and user tasks. This dynamic composition maintains low system complexity while achieving high context adaptability.
Solution Approach 2:
The patent changes interface parameters dynamically based on the detected physical context. The system adjusts interface composition, interaction methods, and object mappings according to the current environment and user needs, achieving adaptability without significantly increasing system complexity.
4Adaptability or versatility
If environmental scanning and dynamic composition are implemented, then context adaptability is improved, but processing requirements and system complexity increase
Solution Approach 1:
The patent implements partial environmental scanning by focusing on detecting only the physical objects relevant to user tasks rather than comprehensively analyzing the entire environment. This selective approach maintains context adaptability while reducing processing requirements.
Solution Approach 2:
The system performs preliminary identification of object affordances and potential interactions before full interface composition. This preliminary action allows the system to prepare interface options in advance, reducing real-time processing requirements while maintaining high context adaptability.
Data Source
AI summary
An Adaptive Tangible User Interface (ATUI) for use in an extended reality environment in which tangible interfaces are composed in real time based on identified affordances of existing objects in the physical environment and the input tasks of a user. An extended reality system can be provided with instructions executable by one or more processors to perform processing including: generating a representation of the real-world environment within a field of view of the user; identifying physical objects within the real-world environment; generating a set of possible performable gestures afforded by available object affordance factors; determining potential input tasks; composing performable gestures for the potential input tasks; and selecting a physical object for use as an adaptive tangible user interface. Techniques for designing a virtual user interface, overlaying the virtual user interface on a selected ATUI physical object, and maintaining alignment of the virtual user interface, are also provided.