Fast AR Device Pairing with Monocular Depth Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AR devices face challenges in efficiently aligning coordinate systems due to resource constraints and privacy concerns with conventional depth sensors, leading to inefficient power consumption and computational demands.
Innovation Solution
A system that uses single-view depth predictions from monocular cameras and SLAM systems to align coordinate systems between AR devices, reducing the need for high computational resources by predicting depth from sparse 3D points and generating dense point clouds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional depth sensors are used to align coordinate systems, then alignment accuracy is improved, but power consumption and computational demands increase
Solution Approach 1:
The patent replaces conventional depth sensors (mechanical/optical hardware) with a computational approach using monocular camera images and neural network-based depth estimation. The system uses a single camera to capture images, then applies machine learning models to predict depth information, substituting physical depth sensing mechanisms with algorithmic depth synthesis.
Solution Approach 2:
The patent introduces an intermediary computational process (depth estimation neural network) between the monocular camera and the coordinate system alignment function. This intermediary transforms 2D images into predicted depth maps, which are then used for point cloud generation and coordinate alignment, mediating the information flow without requiring direct depth sensing.
2Measurement precision
If conventional depth sensors are used to align coordinate systems, then alignment accuracy is improved, but computational requirements increase
Solution Approach 1:
The patent performs preliminary depth prediction on individual images using pre-trained neural networks before coordinate system alignment is needed. By pre-processing images to generate depth maps and point clouds in advance, the system reduces real-time computational burden during the actual alignment operation, separating heavy computation from time-critical operations.
Solution Approach 2:
The patent segments the coordinate alignment process into distinct stages: image capture, depth prediction, point cloud generation, and alignment computation. This segmentation allows each stage to be optimized independently, with depth prediction handled by specialized neural networks and alignment handled by geometric computations, reducing overall computational complexity.
3Adaptability or versatility
If multiple AR devices share experiences, then user experience is improved, but coordinate system alignment becomes more complex
Solution Approach 1:
The patent uses copying by generating virtual point clouds from 2D images through depth prediction. Each AR device creates a computational copy of the 3D environment from its camera views, allowing coordinate systems to be aligned through point cloud registration without requiring direct sensor-to-sensor calibration between devices, simplifying multi-device synchronization.
Data Source
AI summary
A method for aligning coordinate systems from separate augmented reality (AR) devices is described. In one aspect, the method includes generating predicted depths of a first point cloud by applying a pre-trained model to a first single image generated by a first monocular camera of a first augmented reality (AR) device, and first sparse 3D points generated by a first SLAM system at the first AR device, generating predicted depths of a second point cloud by applying the pre-trained model to a second single image generated by a second monocular camera of the second AR device, and second sparse 3D points generated by a second SLAM system at the second AR device, determining a relative pose between the first AR device and the second AR device by registering the first point cloud with the second point cloud.