HCI Gesture Placement for 3D Virtual Objects

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current human-computer interfaces for interacting with virtual objects in mixed and virtual reality environments are inefficient, leading to suboptimal user experiences and wastage of computing resources due to cumbersome interaction methods.

Innovation Solution

An improved human-computer interface that allows users to place, size, and orient virtual objects in a 3D representation of a real-world environment using simplified gestures, with automated and manual modes for efficient placement and manipulation, leveraging 3D spatial mapping and mesh data to enhance interaction efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If traditional input devices are used for placement and modification of virtual objects, then basic functionality is maintained, but user experience deteriorates and computing resources are wasted due to inefficient interactions

Engineering Contradiction:

Improveuser interaction efficiencyVSAvoidcomputing resource consumption

Core Design Contradiction:

Ease of operationVSLoss of energy

Solution Approach 1:

The system automatically determines the size of virtual objects based on the distance between real-world objects and sensors, eliminating the need for manual size adjustment by the user. This self-service approach reduces interaction steps and computing resource consumption while maintaining functionality.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system pre-establishes the relationship between physical distance and virtual object size before user interaction occurs. By using sensor data to automatically calculate appropriate object dimensions, the system prepares the optimal configuration in advance, reducing the computational burden during actual interaction.

Inventive Principle:

Principle #10Preliminary action

2Productivity

If automated mode is used for virtual object placement, then placement speed increases, but manual control over position, orientation, and size is reduced

Engineering Contradiction:

Improveobject placement speedVSAvoidmanual control capability

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system dynamically switches between automated and manual modes based on user needs. In automated mode, objects are placed quickly using sensor data. In manual mode, users gain full control over position, orientation, and size. This dynamic adaptability allows the system to optimize for either speed or control depending on the specific task requirements.

Inventive Principle:

Principle #15Dynamics

3Manufacturing precision

If manual mode is used for virtual object placement, then precise control over position, orientation, and size is achieved, but interaction time and complexity increase

Engineering Contradiction:

Improveobject placement precisionVSAvoidinteraction time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The placement process is segmented into distinct modes: automated placement for quick positioning and manual adjustment for precise control. Users can first place objects automatically to achieve rough positioning, then switch to manual mode only when precise adjustment is needed, thereby reducing overall interaction time while maintaining precision when required.

Inventive Principle:

Principle #1Segmentation

4Measurement precision

If virtual objects are manually sized and positioned, then placement accuracy improves, but computing resources such as processor cycles and memory are consumed more heavily

Engineering Contradiction:

Improveobject positioning accuracyVSAvoidprocessor cycle consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system automatically determines virtual object size based on sensor measurements of real-world object distances, eliminating the need for manual size specification. This self-service mechanism reduces the computational complexity of object placement while maintaining accuracy, as the system uses pre-captured sensor data rather than requiring iterative manual adjustments.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS11023093B2Human-computer interface for computationally efficient placement and sizing of virtual objects in a three-dimensional representation of a real-world environment

Publication Date: 2021.06.01 MICROSOFT TECHNOLOGY LICENSING LLC

AI summary

An improved human-computer interface for placing and sizing virtual objects in a three-dimensional (“3D”) representation of a real-world environment is disclosed herein. The disclosed HCI can determine a location, size, and orientation of virtual objects in a 3D representation of a real-world environment based on simplified user gestures. In some embodiments, based on the timing and direction of an input gesture, a computing device can utilize (1) an automated mode that determines a position and orientation of a virtual object or utilize (2) a manual mode that determines position, orientation, and size of the virtual object. A computing device capable of 3D spatial mapping generates mesh data defining a 3D representation of a real-world environment and images of the real-world environment. The size of the virtual object can be based on a computed depth of real-world objects related to the virtual object.