Cross Reality Localization With Normalized Images Across Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cross reality (XR) systems face challenges in interoperability and localization accuracy due to differences in image processing and sensor configurations across various devices, leading to inconsistent positioning of virtual content on devices with single or multiple cameras.
Innovation Solution
A cross reality system that interfaces with native AR components on devices to normalize image values, apply gamma correction and denoise operations, and generate feature descriptors for accurate localization, using a remote service to align device coordinates with stored maps, enabling shared user experiences across multiple device types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If image processing is performed locally on each device, then device independence is maintained, but localization accuracy and consistency across devices deteriorates
Solution Approach 1:
The patent introduces a remote service as an intermediary that receives images from multiple devices, performs centralized image processing and normalization, then returns processed images to the respective devices. This mediator approach enables consistent localization across devices without requiring each device to perform complex processing independently, thus improving localization accuracy while reducing individual device complexity.
Solution Approach 2:
The remote service provides universal image processing capabilities that work across different device types (smartphones, tablets, wearables) with varying camera configurations. The service normalizes images from different devices to a common reference frame, enabling the same processing pipeline to handle diverse input devices while maintaining consistent output for localization.
2Adaptability or versatility
If devices with different camera configurations are supported, then system versatility improves, but image normalization and feature matching reliability deteriorates
Solution Approach 1:
The patent applies parameter changes by adjusting image processing parameters dynamically based on device type and camera configuration. The remote service modifies parameters such as exposure, contrast, and color space normalization to compensate for differences between devices with single cameras, multiple cameras, or different sensor characteristics. This enables reliable feature extraction across diverse device types while maintaining descriptor consistency.
Solution Approach 2:
The system creates a standardized reference representation of the environment by processing images from multiple devices and consolidating their observations. The remote service generates a unified map and feature database that copies and integrates information from all devices, creating a consistent reference frame that all devices can use for localization regardless of their individual camera configurations.
3Productivity
If image processing is performed locally on each device, then processing speed is maintained, but interoperability between different device types deteriorates
Solution Approach 1:
The patent segments the image processing task into two parts: local devices perform rapid image capture and initial processing to extract features and generate descriptors, while the remote service handles the complex normalization and coordinate transformation tasks. This segmentation allows devices to maintain high processing speed for local operations while delegating interoperability challenges to the centralized service, thus preserving both productivity and adaptability.
Data Source
AI summary
A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.