AR Headset RF Sensing for Occluded Item Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR headsets struggle to perceive and locate occluded objects in dense and cluttered industrial environments due to their reliance on line-of-sight vision-based sensing, hindering efficiency in sectors like manufacturing and warehousing.
Innovation Solution
Integration of a conformal RF sensing system with AR headsets using ultra-high frequency RFID and synthetic aperture radar (SAR) to enable non-line-of-sight perception, allowing accurate localization and verification of occluded items through RF signals without obstructing the user's field of view.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Difficulty of detecting and measuring
If line-of-sight vision-based sensing is used in AR headsets, then the device complexity remains low and ease of operation is maintained, but the ability to perceive occluded objects in dense environments is severely limited
Solution Approach 1:
The patent combines multiple sensing modalities (vision-based cameras and RF sensing systems) into a single AR headset platform. The RF sensing system uses antennas integrated into the headset to detect occluded objects, while vision systems capture visible objects. These disparate sensing approaches are merged to provide comprehensive environmental perception that overcomes the limitations of either system alone.
Solution Approach 2:
The AR headset is designed with multi-functional sensing capabilities that can operate in different modes depending on the task. The same device can perform visual recognition for line-of-sight objects and RF-based detection for occluded objects, making it a universal platform that adapts to various perception needs in industrial environments.
2Reliability
If RF sensing systems are integrated into AR headsets to enable non-line-of-sight perception, then occluded objects can be detected and localized, but the device complexity and antenna design challenges increase
Solution Approach 1:
The patent implements different antenna types at different locations on the headset, each optimized for specific detection scenarios. The antenna configuration varies by position to maximize coverage and detection reliability for occluded objects from multiple angles, rather than using a single uniform antenna design.
Solution Approach 2:
The RF sensing system is nested within the existing AR headset structure. Antennas are integrated into the headset housing and form factors, with the RF sensing functionality embedded within the overall device architecture. This nesting approach allows the complex RF system to be incorporated without substantially increasing external device dimensions.
3Measurement precision
If synthetic aperture radar techniques are used for localization, then positioning accuracy improves, but the processing complexity and computational requirements increase
Solution Approach 1:
The system performs preliminary processing of RF signals by collecting measurements from multiple antenna elements and time samples before final localization computation. Data from the antenna array is pre-processed and organized into formats suitable for SAR algorithms, reducing the computational burden on the main processor and enabling accurate localization without overwhelming processing requirements.
4Difficulty of detecting and measuring
If multiple sensing modalities are integrated, then perception capability in cluttered environments improves, but the device complexity and power consumption increase
Solution Approach 1:
The system employs periodic sensing operations rather than continuous operation of all sensors. RF sensing and vision-based sensing are activated at different times or triggered by specific conditions, allowing the device to maintain occlusion penetration capability while managing power consumption through intermittent rather than continuous operation of power-intensive components.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables efficient identification and guidance towards occluded items, automating inventory control and reducing search time in cluttered environments by leveraging natural human mobility and integrating RF signals with visual information for precise localization and verification.
Implementation Method 1
RF signals can be used to sense and accurately localize items in non-line-of-sight and highly cluttered environments. Embodiments of the present disclosure make use of ultra-high frequency (UHF) RFID (Radio Frequency IDentification)
Implementation Method 2
disclosed systems and techniques opportunistically leverage natural human mobility using visual information from the AR headset camera along with RFID measurements collected during the user's motion to create a synthetic aperture radar (SAR) and localize RFID tagged items with high accuracy
Data Source
AI summary
According to one aspect of the disclosure, a mobile augmented reality (AR) system can include: a receiver configured to receive radio frequency (RF) signals from one or more items located within an environment; a tracking module configured to generate tracking data responsive to a location of the system within the environment over time; a display device; and one or more processors configured to determine a location of at least one of the one or more items within the environment using the received RF signals and the tracking data, and generate a visual representation of the location of the at least one item for display on the display device.