Markerless AR Positioning via Truncated 3D Model Transmission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems require graphical markers for accurate positioning and orientation of virtual objects, which can be disruptive and limit their application in environments without predefined markers, and they struggle with remote AR interactions that require local user assistance and high bandwidth for real-time data transmission.
Innovation Solution
A remote AR framework that enables markerless AR by using 3D data capture and transmission of truncated 3D models, allowing virtual objects to be positioned and oriented accurately without local user assistance, and reduces bandwidth requirements through perspective video transmission, supporting both synchronous and asynchronous interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If graphical markers are used for AR positioning, then positioning accuracy is improved, but the system becomes disruptive and limited to environments with predefined markers
Solution Approach 1:
The patent extracts the positioning function from graphical markers by using natural environment features (edges, corners, surfaces) detected through computer vision algorithms. The system captures images via camera, extracts geometric features from the environment, and uses these features for positioning and orientation without requiring any artificial markers to be placed in the scene.
Solution Approach 2:
The system creates a digital replica of the physical environment by capturing images and generating 3D models of natural features. This digital copy includes geometric information about surfaces, edges, and corners that can be used for positioning, effectively copying the positioning function from marker-based systems to markerless natural feature-based systems.
2Measurement precision
If real-time 3D data is transmitted for remote AR, then remote interaction accuracy is improved, but bandwidth consumption increases
Solution Approach 1:
The system extracts only the essential 3D geometric information needed for remote AR positioning from the full environment model. Instead of transmitting complete high-resolution 3D scans, it transmits extracted feature points, plane information, and transformed coordinate data that are sufficient for accurate positioning but require significantly less bandwidth.
Solution Approach 2:
The patent applies different levels of data quality to different parts of the system. The local device maintains high-resolution 3D models for accurate local rendering, while only transmitting compressed and transformed 3D data (position, orientation, scale information) to remote devices. This local quality approach ensures remote interaction accuracy while minimizing bandwidth consumption.
3Measurement precision
If complete 3D models are transmitted for remote AR, then positioning accuracy is improved, but data transmission volume increases
Solution Approach 1:
The system extracts only the critical 3D transformation parameters (position, orientation, scale) and essential geometric features from complete 3D models. These extracted parameters are sufficient for accurate AR object positioning and placement in remote environments, while requiring minimal data transmission volume compared to transmitting full 3D model datasets.
Solution Approach 2:
The patent performs preliminary processing of 3D data locally before transmission. The system pre-computes transformation matrices, extracts key feature points, and prepares condensed 3D information in advance. This preliminary action ensures that when data is transmitted to remote devices, it is already in the optimized format needed for accurate positioning, reducing both data volume and processing requirements at the receiving end.
Data Source
AI summary
Augmented reality (AR) telepresence systems and methods are disclosed for obtaining a 3D model of a physical location from a 3D-capture system comprising one or more 3D depth sensors disposed throughout the physical location, generating a truncated 3D model of the physical location, the truncated 3D model corresponding to the intersection of the generated 3D model and a field of view of a user terminal camera at the physical location, and transmitting the truncated 3D model to a remote location.