Virtual Pointer Interaction with Physical Peripheral Device in AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial reality systems face challenges in providing precise and intuitive user interactions with virtual surfaces, often resulting in imprecise gesture detection and increased power consumption due to the need for separate interfaces on devices like smartphones or tablets.
Innovation Solution
The implementation of a physical peripheral device that interacts with an artificial reality system through pose tracking and gesture detection, allowing users to manipulate virtual surfaces with haptic feedback, eliminating the need for a separate interface on the device by using a presence-sensitive surface for input detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a separate interface on smartphone or tablet is used for user input, then the device can provide a user interface, but power consumption increases and interaction precision decreases
Solution Approach 1:
The patent extracts the interface display function from the peripheral device and presents it virtually on the AR surface instead. The peripheral device only retains input detection capabilities through presence-sensitive surface, eliminating the need for separate display hardware and reducing power consumption while maintaining interaction precision.
Solution Approach 2:
The patent merges the interface display with the AR virtual surface, combining the peripheral device's input detection with the HMD's display capabilities. This integration allows the interface to be rendered in the augmented reality environment where it can be interacted with using the same spatial gestures, improving precision while distributing computational load.
2Ease of operation
If gesture detection is used for interacting with virtual surfaces, then user interaction is enabled, but detection precision is insufficient
Solution Approach 1:
The patent introduces a presence-sensitive surface as an intermediary between the user and the virtual surface. This intermediary layer detects precise touch and proximity inputs on the physical peripheral device, translating them into accurate spatial gestures in the AR environment, thereby bridging the precision gap between physical input and virtual interaction.
3Ease of operation
If a physical peripheral device with separate interface is used, then user input is detected, but device complexity increases
Solution Approach 1:
The patent extracts the display function from the peripheral device and relocates it to the virtual AR environment. The peripheral device is reduced to an input controller with presence-sensitive surface, eliminating the need for separate display hardware, buttons, and interface management complexity while maintaining full input detection capability.
Data Source
AI summary
In some examples, a method comprises: obtaining, by an artificial reality system including a head mounted display (HMD), image data via an image capture device, the HMD configured to output artificial reality content; detecting, by the artificial reality system, a physical peripheral device from the image data; detecting, by the artificial reality system, a pose of the peripheral device; generating, by the artificial reality system, based on the pose of the peripheral device, a virtual pointer along at least a portion of a line between the peripheral device and a virtual surface, the virtual pointer pointing at a location of the virtual surface; performing, by the artificial reality system, one or more actions based on the location of the virtual surface; and rendering, by the artificial reality system, the virtual pointer and the virtual surface for display at the HMD.