Orientation-Aware Mixed-Reality Buttons and Objects

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Mixed-reality systems face challenges in accurately detecting and preventing unintentional user interactions with virtual objects due to the lack of physical constraints, leading to accidental or unintended user interactions, particularly when multiple virtual objects are in close proximity.

Innovation Solution

A mixed-reality system detects controller gestures and orientations, determines interaction regions, and evaluates orientation similarity scores and directional preferences to selectively execute interactions with virtual objects, ensuring alignment with predefined thresholds and preferences.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If virtual objects are placed in close proximity to increase interaction density, then the quantity of interactable objects increases, but unintentional interactions increase

Engineering Contradiction:

Improvequantity of virtual objectsVSAvoidinteraction accuracy

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The interaction detection is segmented into multiple independent checks: directional component analysis, orientation similarity scoring, and threshold-based validation. Each virtual object within interaction region has its orientation affinity evaluated separately, allowing precise discrimination between intended and unintended interactions even when objects are densely packed.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

An orientation similarity score acts as an intermediary metric between the controller orientation and virtual object orientation affinity. This intermediate evaluation layer filters out accidental interactions by requiring the directional component to match the preferred direction within a threshold before executing interaction, preventing direct false positives from close proximity objects.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If physical constraints are applied to virtual objects to prevent accidental interactions, then interaction accuracy improves, but ease of operation decreases

Engineering Contradiction:

Improveinteraction accuracyVSAvoidgesture freedom

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The interaction constraints are dynamic rather than static. The threshold parameter allows the system to adjust sensitivity based on context, and the orientation similarity requirement only activates when objects are within the interaction region. This dynamic approach maintains gesture freedom for clear intentions while preventing accidents during casual hand movements.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system uses parameter-based control through orientation similarity scores and thresholds. By changing the threshold parameter, the system can flexibly adjust between being more permissive (higher threshold) or more restrictive (lower threshold) without changing the fundamental interaction model, balancing accuracy and ease of operation based on specific use cases.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If orientation similarity scoring is implemented to distinguish intended interactions, then interaction accuracy improves, but device complexity increases

Engineering Contradiction:

Improveinteraction accuracyVSAvoidsystem complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces complex mechanical interaction detection with computational orientation analysis. Instead of using physical sensors or complex gesture recognition algorithms, the system uses straightforward vector mathematics to compare controller orientation with virtual object orientation affinity, calculating similarity scores through simple directional component comparison.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentEP3908905B1Hand motion and orientation-aware buttons and grabbable objects in mixed reality

Publication Date: 2025.07.16 MICROSOFT TECHNOLOGY LICENSING LLC

Figure 1
Figure 2A~2B
Figure 3

AI summary

Systems and methods are provided for detecting user-object interaction in mixed-reality environments. A mixed-reality system detects a controller gesture with an associated controller orientation in the mixed-reality environment. The mixed-reality system then determines an interaction region for the controller gesture and identifies one or more virtual objects within the interaction region. The virtual objects each have an associated orientation affinity. Subsequently, the mixed-reality system determines an orientation similarity score between the controller orientation and the orientation affinity for each virtual object within the interaction region. In response to determining that at least one orientation similarity score exceeds a predetermined threshold, the mixed-reality system executes an interaction between the controller and the virtual object that has the greatest orientation similarity score.