Mapped Direct-Touch Virtual Trackpad for Distant XR Objects
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Interacting with virtual objects located at a distance in an extended reality (XR) environment is challenging, leading to user fatigue and increased system complexity due to the need for controllers.
Innovation Solution
Generating a virtual trackpad that responds to direct touch from the user's virtual hands, mapped to the surface of distant virtual objects, allowing precise interaction through thumb or index finger movements without the need for controllers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If controllers are integrated into the system to allow user access to all portions of the XR environment, then interaction capability with distant virtual objects is improved, but device complexity and cost increase
Solution Approach 1:
The patent extracts the interaction function from physical controllers and relocates it to the user's hands through computer vision. The system captures hand gestures and finger movements using cameras, processes these inputs to control the cursor and interact with virtual objects, thereby eliminating the need for separate controller devices while maintaining full interaction capability.
Solution Approach 2:
The patent replaces the mechanical controller system with a computer vision-based system. Instead of using physical buttons, joysticks, or touchpads on controllers, the system uses cameras to detect hand positions, gestures, and finger movements, converting these biological signals into digital control inputs for interacting with virtual objects in the XR environment.
2Device complexity
If hand gestures are used to interact with distant virtual objects, then device portability is improved, but user fatigue increases due to ergonomic inefficiency
Solution Approach 1:
The patent introduces a virtual trackpad as an intermediary between the user's hands and the distant virtual objects. The virtual trackpad appears at a convenient location within the user's natural reach zone, allowing finger movements on this intermediate surface to control the cursor and interact with objects at various distances, thereby reducing the need for uncomfortable hand positions and gestures.
Solution Approach 2:
The patent adds the dimension of a virtual trackpad surface that can be positioned optimally for user interaction. By creating this intermediate interaction plane within the user's natural reach zone, the system transforms the interaction from direct distant manipulation (requiring extended arm movements) to localized finger movements on the virtual trackpad, significantly improving ergonomics.
3Ease of operation
If direct touch interaction is used for virtual objects within reach, then ease of operation is improved, but interaction capability with distant objects remains limited
Solution Approach 1:
The patent creates a universal interaction mechanism that serves multiple functions: the virtual trackpad enables direct touch-like interaction for objects within reach while simultaneously providing cursor control capability for distant objects. The same finger-based interaction model works for both close-range and distant-object manipulation, making the system versatile across different interaction distances without requiring different input methods.
Data Source
AI summary
A virtual trackpad engine is configured to perform a technique for accessing and manipulating distant virtual objects in an XR environment. The technique includes generating, in an extended reality (XR) environment, a virtual model corresponding to a hand of a user. The technique also includes generating a virtual trackpad in the XR environment, where a surface of the virtual trackpad is mapped to a surface of a distant virtual object in the XR environment. Further, the technique includes determining if the virtual trackpad is activated and responsive to a determination that the virtual trackpad is activated, tracking a movement of the virtual hand. The technique further includes determining an action to be performed in relation to the distant virtual object that corresponds to the movement and, subsequently, performing the action.