3D Content Sharing and Physical Device Control with Haptic Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing methods for interacting with augmented and virtual reality environments are cumbersome, inefficient, and create a significant cognitive burden on users, often requiring multiple inputs and providing insufficient feedback, leading to errors and energy wastage, particularly in battery-operated devices.

Innovation Solution

The system employs improved user interfaces that utilize touch-sensitive displays, eye-tracking, hand-tracking, and tactile output generators to reduce the number and complexity of user inputs, providing intuitive interactions through visual and tactile feedback, and enabling efficient manipulation of virtual objects in three-dimensional environments.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If conventional user interfaces are used in augmented reality environments, then basic interaction functionality is provided, but user interaction efficiency deteriorates and cognitive burden increases due to cumbersome operations requiring multiple inputs

Engineering Contradiction:

Improveuser interaction efficiencyVSAvoidease of operation

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The system provides immediate visual and tactile feedback when users perform gestures in augmented reality environments. Visual feedback includes displaying highlighted boundaries or indicators on virtual objects when users approach or perform gestures, while tactile feedback uses haptic actuators to provide force feedback or vibration patterns that confirm gesture recognition and system response, reducing cognitive burden and improving interaction efficiency

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system automatically detects and interprets user gestures without requiring explicit command sequences. By using computer vision and machine learning to recognize gesture patterns, the system self-determines user intent and executes appropriate actions, eliminating the need for users to navigate complex input protocols or provide multiple sequential inputs

Inventive Principle:

Principle #25Self-service

2Productivity

If multiple inputs are required to achieve desired outcomes in augmented reality environments, then basic control functionality is provided, but time consumption increases and energy is wasted

Engineering Contradiction:

Improveoperation speedVSAvoidtime for completing operations

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The system pre-loads and pre-processes virtual objects and interface elements before they are needed for interaction. When users perform gestures, the system has already prepared the necessary computational resources, graphical rendering, and action sequences, enabling immediate execution without requiring users to wait for processing or provide multiple sequential inputs

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system combines multiple input detection modalities (visual gestures, voice commands, eye tracking) into a unified interaction framework. By merging these input methods and using intelligent routing, the system can achieve complex outcomes through single integrated gestures rather than requiring multiple separate inputs, reducing both time and energy consumption

Inventive Principle:

Principle #5Merging (Combining)

3Reliability

If conventional interaction methods are used, then basic functionality is provided, but error rate increases due to insufficient feedback and complex manipulation requirements

Engineering Contradiction:

Improveaccuracy of interactionVSAvoidfeedback information

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system provides predictive visual feedback that shows users the anticipated outcome of their gestures before execution. By displaying preview indicators or highlighted boundaries of affected virtual objects in advance, the system allows users to correct potential errors before they occur, thereby reducing the error rate and improving interaction reliability

Inventive Principle:

Principle #11Beforehand cushioning (Prior cushioning)

4Use of energy by moving object

If battery-operated devices are used in augmented reality applications, then portability is improved, but energy consumption becomes a limiting factor due to inefficient interactions and unnecessary processing

Engineering Contradiction:

Improvepower consumptionVSAvoidinteraction efficiency

Core Design Contradiction:

Use of energy by moving objectVSProductivity

Solution Approach 1:

The system uses periodic sampling and event-driven processing rather than continuous operation. Gesture detection and processing are triggered only when user interactions occur, and the system enters low-power states between interactions. This periodic activation pattern significantly reduces energy consumption while maintaining high interaction efficiency when users are actively engaged

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The system dynamically adjusts processing parameters such as frame rate, resolution, and computational intensity based on interaction context and device power state. During active interaction, the system increases processing power to maintain responsiveness, while during idle periods it reduces parameters to conserve battery energy, optimizing the balance between interaction efficiency and power consumption

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS20250278907A1Methods for sharing content and interacting with physical devices in a three-dimensional environment

Publication Date: 2025.09.04 APPLE INC

AI summary

In some embodiments, a computer system displays a user interface element corresponding to content shared with the computer system for interacting with the shared content in the three-dimensional environment. In some embodiments, a computer system displays one or more virtual control elements in a three-dimensional environment that are selectable to cause one or more corresponding operations involving a physical device to be performed.