Mixed Reality Control Points for Gesture Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current mixed reality technologies require multiple hand-to-object gestures for interacting with computer-generated objects, leading to confusion and inefficient use of computational resources.
Innovation Solution
The use of control points associated with rendered objects and mixed reality environments, where a single gesture, such as a pinch gesture, identifies a target control point to alter properties, streamlining interactions and reducing the need for complex gesture recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple hand-to-object gestures are used for different virtual interactions, then functionality is improved, but device complexity and ease of operation deteriorate
Solution Approach 1:
A single hand-to-object gesture is assigned multiple functions based on the virtual object type. The system determines the gesture type once and reuses it across different virtual objects (e.g., a grab gesture works for both virtual cups and virtual balls), eliminating the need for multiple specialized gestures while maintaining full functionality.
Solution Approach 2:
The system changes the interpretation parameters of a single gesture based on the target virtual object properties. Instead of changing the gesture itself, the system adapts the gesture's effect parameters according to what is being interacted with, allowing one gesture to produce different valid outcomes depending on context.
2Adaptability or versatility
If multiple hand-to-object gestures are implemented, then interaction capability is improved, but computational resource consumption increases
Solution Approach 1:
The system implements a universal gesture recognition module that handles all virtual objects with a single gesture vocabulary. This eliminates the need for multiple specialized gesture recognition algorithms, reducing computational overhead while maintaining full interaction capability across different object types.
3Measurement precision
If multiple hand-to-object gestures are required, then precise control is improved, but ease of operation deteriorates
Solution Approach 1:
Users learn and perform only one set of fundamental hand-to-object gestures that work universally across all virtual objects. This reduces the cognitive load and motor memory requirements compared to learning multiple object-specific gestures, significantly improving ease of operation while maintaining precise control through consistent gesture-object mapping.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Altering properties of rendered objects and/or mixed reality environments utilizing control points associated with the rendered objects and/or mixed reality environments is described. Techniques described can include detecting a gesture performed by or in association with a control object. Based at least in part on detecting the gesture, techniques described can identify a target control point that is associated with a rendered object and/or a mixed reality environment. As the control object moves within the mixed reality environment, the target control point can track the movement of the control object. Based at least in part on the movement of the control object, a property of the rendered object and/or the mixed reality environment can be altered. A rendering of the rendered object and/or the mixed reality environment can be modified to reflect any alterations to the property.