Multi-device Pose Estimation via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision techniques for localization and mapping in computer-generated reality (CGR) often inaccurately display virtual objects within the physical environment, leading to a suboptimal user experience due to errors in determining the pose of devices.
Innovation Solution
A method using a first electronic device with a camera sensor and motion sensor, which obtains image and motion data, receives information from a second device, and generates a representation of its pose to accurately display virtual objects based on this data, potentially involving multiple devices and a server for improved accuracy through data sharing and minimization functions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single-device localization and mapping techniques are used, then device complexity is reduced, but measurement precision of pose estimation deteriorates
Solution Approach 1:
The patent combines data from multiple electronic devices to perform localization and mapping. The system merges image data, motion data, and pose information from multiple devices to generate a unified environmental map and determine accurate poses, thereby improving measurement precision through data fusion while distributing system complexity across multiple devices.
Solution Approach 2:
The patent introduces a server as an intermediary that receives data from multiple electronic devices, performs centralized processing including minimization functions to optimize pose estimates, and distributes results back to the devices. This intermediary architecture enables high-precision localization and mapping by consolidating computational complexity in a dedicated server while maintaining simple device-level implementations.
2Measurement precision
If multiple devices are used for data sharing, then measurement precision of localization improves, but device complexity increases
Solution Approach 1:
The server acts as an intermediary that manages the complexity of processing data from multiple devices. It receives image data, motion data, and initial pose estimates from multiple electronic devices, performs minimization functions to optimize localization accuracy, and distributes refined results back to the devices, thereby improving localization precision without increasing complexity at the device level.
Solution Approach 2:
The system segments the localization and mapping task across multiple devices and a server. Each device independently captures image and motion data, while the server performs centralized optimization. This segmentation allows the system to leverage multiple data sources for improved precision while distributing computational responsibilities to manage complexity.
3Manufacturing precision
If pose representation is generated using multiple data sources, then manufacturing precision of virtual object placement improves, but loss of time for data processing increases
Solution Approach 1:
The system performs preliminary actions by having multiple devices continuously capture and transmit image and motion data in advance. The server maintains ready-to-process data streams and pre-computes environmental features, so when virtual object placement is needed, the optimized pose estimation can be quickly generated from pre-processed data, reducing the time penalty of using multiple data sources.
Solution Approach 2:
The patent replaces traditional mechanical or single-sensor pose estimation methods with a computational system that uses image data, motion data, and minimization functions. This substitution enables high-precision virtual object placement by leveraging multiple data sources and mathematical optimization, achieving manufacturing precision through information processing rather than physical measurement systems.
Data Source
AI summary
The present disclosure relates generally to localization and mapping. In some examples, an electronic device obtains first image data and motion data using a motion sensor. The electronic device receives information corresponding to a second electronic device. The electronic device generates a representation of a first pose of the first electronic device using the first image data, the motion data, and the information corresponding to the second electronic device. The electronic device displays, on the display, a virtual object, wherein the displaying of the virtual object is based on the representation of the first pose of the first electronic device.