Passthrough Window Object Locator in Artificial Reality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users of artificial reality systems often face difficulties in locating and obtaining physical objects while wearing a head-mounted display (HMD), leading to disruptions in the artificial reality experience as they may prefer to remove the HMD to find objects more easily in the physical environment.
Innovation Solution
The system generates artificial reality content that includes a passthrough window, allowing users to view their physical environment and locate objects within it, even when those objects are not in their possession, by using a head-mounted display, mapping engine, and application engine to detect the object's location and update the content accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a user wears a head-mounted display (HMD) to experience artificial reality content, then immersion and experience quality are improved, but the ability to locate and obtain physical objects in the real environment deteriorates
Solution Approach 1:
The field of view is segmented into multiple regions including a central region for primary artificial reality content and peripheral regions for passthrough windows. This allows simultaneous display of virtual content and real-world object locations without requiring the user to remove the HMD, resolving the contradiction between immersion and object locatability.
Solution Approach 2:
Passthrough windows act as an intermediary mechanism that bridges the artificial reality environment and the physical environment. These windows provide visual information about real-world objects directly within the artificial reality display, enabling users to locate objects while maintaining immersion in the virtual environment.
2Ease of operation
If the user removes the HMD to locate objects easily, then object finding capability is improved, but the artificial reality experience is disrupted
Solution Approach 1:
The system performs preliminary actions by pre-processing real-world visual data and integrating it into passthrough windows before the user needs to locate an object. This allows object location information to be prepared and displayed in advance, enabling quick object finding without breaking the artificial reality experience.
Solution Approach 2:
The passthrough windows maintain continuous visual feedback about the physical environment throughout the artificial reality experience. This continuous integration of real-world information allows users to locate objects seamlessly without removing the HMD, preserving experience continuity while improving object finding capability.
3Ease of operation
If passthrough windows are added to display real-world objects, then object locatability is improved, but system complexity increases
Solution Approach 1:
The content generation system is designed to be universal by handling multiple functions: generating artificial reality content, processing real-world visual data, tracking objects, and creating passthrough windows all through a single integrated system. This multi-functionality reduces the need for separate dedicated systems, thereby managing complexity while improving object locatability.
Data Source
AI summary
This disclosure describes an artificial reality system that assists a user in finding, locating, and/or taking possession of an object. In one example, this disclosure describes a system that includes a head-mounted display (HMD), capable of being worn by a user; a mapping engine configured to determine a map of a physical environment including position information about the HMD and an object; and an application engine configured to: detect execution of an application that operates using the object, determine that the object is not in possession of the user, and responsive to detecting execution of the application and determining that the object is not in possession of the user, generate artificial reality content that includes a passthrough window positioned to include the object.