Mixed Reality Localization via Virtual Environment Copying

Solution Overview

Problem

Existing augmented reality systems face challenges in accurately determining user location without additional hardware like location beacons or GPS devices, which increases cost and power requirements.

Innovation Solution

The system employs metadata collection from various networks such as WiFi, Bluetooth, and GPS, using keyframe analysis for coarse and fine localization to determine user position and orientation, enabling location-based content sharing without the need for additional hardware.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If additional hardware like location beacons or GPS devices is used to determine user location, then location accuracy is improved, but device cost and power requirements increase

Engineering Contradiction:

Improvelocation accuracyVSAvoiddevice cost

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system creates a virtual copy of the physical environment by capturing images and generating a three-dimensional model of the area. This virtual representation allows the system to determine user location through image recognition and matching against the pre-built model, eliminating the need for physical location beacons or GPS hardware while maintaining accurate location determination.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces mechanical/location-based hardware systems (GPS receivers, location beacons) with an optical/computational system using cameras and image processing algorithms. The system captures visual data, processes it to identify recognizable features, and matches these features against a pre-generated three-dimensional model to determine location, substituting physical positioning hardware with computational vision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If additional hardware like location beacons or GPS devices is used to determine user location, then location accuracy is improved, but power consumption increases

Engineering Contradiction:

Improvelocation accuracyVSAvoidpower consumption

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system creates a virtual copy of the physical environment by capturing images and generating a three-dimensional model of the area. This virtual representation allows the system to determine user location through image recognition and matching against the pre-built model, eliminating the need for physical location beacons or GPS hardware while maintaining accurate location determination.

Inventive Principle:

Principle #26Copying

Solution Approach 2:

The patent replaces mechanical/location-based hardware systems (GPS receivers, location beacons) with an optical/computational system using cameras and image processing algorithms. The system captures visual data, processes it to identify recognizable features, and matches these features against a pre-generated three-dimensional model to determine location, substituting physical positioning hardware with computational vision.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Measurement precision

If metadata from multiple networks is collected and processed for localization, then location determination accuracy is improved, but processing time and computational resources increase

Engineering Contradiction:

Improvelocation determination accuracyVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary action by pre-capturing images and generating a three-dimensional model of the entire area before the user needs localization. This pre-built virtual model serves as a reference database that can be quickly queried during actual localization operations, significantly reducing real-time processing requirements compared to analyzing raw sensor data from multiple networks in real-time.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The localization process is segmented into distinct phases: pre-processing phase where images are captured and three-dimensional models are generated, and execution phase where the pre-built model is queried to determine location. This segmentation allows heavy computational tasks to be performed offline during pre-processing, while online localization operations remain lightweight and fast.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11711668B2Localization determination for mixed reality systems

Publication Date: 2023.07.25 MAGIC LEAP INC

AI summary

To enable shared user experiences using augmented reality systems, shared reference points must be provided to have consistent placement (position and orientation) of virtual objects. Furthermore, the position and orientation (pose) of the users must be determinable with respect to the same shared reference points. However, without highly sensitive and expensive global positioning system (GPS) devices, pose information can be difficult to determine to a reasonable level of accuracy. Therefore, what is provided is an alternative approach to determining pose information for augmented reality systems, which can be used to perform location based content acquisition and sharing. Further, what is provided is an alternative approach to determining pose information for augmented reality systems that uses information from already existing GPS devices.