Mixed Reality Pose Estimation via Object Radio Communication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional mixed reality systems struggle to determine the visual pose of unknown objects in a user's real-world environment, such as furniture, due to variations in size, shape, texture, and color, and the inability to pre-store all possible characteristics, especially when objects are sold separately from the recognition device.

Innovation Solution

The system employs wideband radio communication between a recognition device and an object of interest to estimate its visual pose without pre-storing characteristics, using a combination of coarse and fine pose estimation methods, including radio frequency signals and video flux analysis, to project a visual representation of the object into a virtual reality environment.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If conventional methods use pre-stored object characteristics and known patterns for pose determination, then pose recognition accuracy is improved, but the system cannot recognize unknown objects sold separately from the headset

Engineering Contradiction:

Improvepose recognition accuracyVSAvoidability to recognize unknown objects

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The system performs preliminary actions by having the object of interest transmit its characteristics data before pose estimation is needed. The recognition device receives and stores these characteristics in advance, enabling subsequent accurate pose determination without requiring pre-stored databases of all possible objects.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary communication mechanism (wideband radio communication) between the object of interest and the recognition device. This intermediary allows the object to self-identify and transmit its characteristics, bridging the gap between unknown objects and the recognition system without requiring pre-programmed knowledge of the object.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If the system pre-stores all possible object characteristics in the headset, then pose determination accuracy is improved, but device complexity and memory requirements increase significantly

Engineering Contradiction:

Improvevisual pose estimation accuracyVSAvoidheadset complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts the object characteristics data from the recognition device and places it on the object of interest itself. Instead of the headset storing all possible object characteristics, each object carries its own characteristics data, which the headset receives temporarily during interaction. This transfers the storage burden from the headset to the individual objects.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The object of interest performs self-service by transmitting its own characteristics data to the recognition device. The object essentially introduces itself to the system, eliminating the need for the headset to have prior knowledge or extensive databases of object characteristics.

Inventive Principle:

Principle #25Self-service

3Speed

If the system uses conventional marker-based pose determination, then pose recognition speed is improved, but the system cannot determine poses of objects without pre-installed markers

Engineering Contradiction:

Improvepose determination speedVSAvoidability to handle markerless objects

Core Design Contradiction:

SpeedVSAdaptability or versatility

Solution Approach 1:

The patent replaces the mechanical/marker-based system with a communication-based system. Instead of using physical markers and LEDs that require line-of-sight and specific geometric patterns, the system uses wideband radio communication to exchange object characteristics and perform pose estimation, enabling markerless operation.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20240338906A1Method for visual pose determination of unknown objects in a mixed reality context

Publication Date: 2024.10.10 QORVO US INC

AI summary

The present disclosure relates to a mixed reality (MR) system merging computer-generated elements and real-world elements, and a method for determining visual poses of unknown real-world objects in the MR system. The disclosed system includes a recognition device and an object of interest that constitutes a real-world environment. Herein, the recognition device and the object of interest are capable of communicating with each other. The recognition device is capable of estimating a visual pose of the object of interest without pre-storing characteristics of the object of interest and configured to re-project a visual representation of the object of interest into a virtual reality (VR) environment based on the estimated visual pose.