AR VR Gesture Interface with Spatial Filtering

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

In virtual reality and augmented reality environments, unintended gestures made by players while moving can be misinterpreted as command inputs, leading to unintended actions within the game or application.

Innovation Solution

A system that renders an emulated appendage on a display, allowing players to make gestures within a three-dimensional object to input commands, while ignoring gestures made outside of the object to prevent accidental command execution.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If gestures are detected without spatial context, then gesture input is simple and intuitive, but unintended gestures are misinterpreted as commands

Engineering Contradiction:
Improvegesture input simplicityVSAvoidcommand input accuracy
Core Design Contradiction:
Ease of operationVSReliability

Solution Approach 1:

The patent introduces a 3D object as an intermediary spatial container that mediates between the player's gestures and the command execution system. Only gestures performed within the spatial boundaries of the 3D object are interpreted as commands, while gestures outside this boundary are ignored. This intermediary spatial context resolves the contradiction by providing a clear delimiter between intentional and unintentional gestures.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent applies local quality by making the gesture recognition system sensitive to spatial location. The system differentiates between gestures in different spatial regions: gestures within the 3D object's boundary are processed as commands, while gestures outside the boundary are discarded. This spatially-dependent quality control allows the system to maintain both simplicity and accuracy.

Inventive Principle:
Principle #3Local quality

2Speed

If gesture recognition is activated continuously, then command input responsiveness is high, but accidental commands from player movement increase

Engineering Contradiction:
Improvecommand input responsivenessVSAvoidaccidental command execution
Core Design Contradiction:
SpeedVSObject-generated harmful factors

Solution Approach 1:

The 3D object serves as a mediator that filters gestures before they reach the command execution system. Continuous gesture recognition remains active, but the spatial filter of the 3D object prevents accidental commands from being executed. This maintains high responsiveness for intentional commands while blocking harmful accidental inputs.

Inventive Principle:
Principle #24Intermediary (Mediator)

Solution Approach 2:

The system performs preliminary spatial verification before executing commands. Every detected gesture is first checked to see if it occurs within the 3D object's spatial boundaries. Only if this preliminary spatial check passes does the system proceed to interpret and execute the gesture as a command, preventing accidental execution while maintaining responsiveness.

Inventive Principle:
Principle #10Preliminary action

3Adaptability or versatility

If the 3D object is always present for gesture input, then command execution is always available, but the player cannot distinguish between interaction mode and movement mode

Engineering Contradiction:
Improvecommand input availabilityVSAvoidinteraction mode clarity
Core Design Contradiction:
Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent makes the 3D object dynamic by allowing it to appear and disappear based on player interaction. The object is instantiated when the player needs to input commands and disappears when the player is merely moving or exploring. This dynamic presence provides clear contextual cues about whether the system is in interaction mode or movement mode, while maintaining versatility when needed.

Inventive Principle:
Principle #15Dynamics

Data Source

PatentUS11402917B2Gesture-based user interface for AR and VR with gaze trigger
Publication Date: 2022.08.02 SONY INTERACTIVE ENTERTAINMENT LLC
  • US11402917B2 patent drawing
  • US11402917B2 patent drawing
  • US11402917B2 patent drawing

AI summary

A 3D object is opened for presentation on a display in response to a trigger, such as a gaze direction detected by eye tracking in conjunction with a trigger hand gesture. A player's emulated hand in emulated space is configured to have the same gesture as the player's real hand as imaged by a camera, and only when the emulated hand is within the 3D object in emulated space are gestures of the hand correlated to input commands. Otherwise, hand gestures are not considered for correlation to commands.