AR 3D Building Reconstruction with Multi-Track Pose Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3-D reconstruction methods for building structures face challenges with large image inputs requiring costly computing resources and sparse data failing to capture adequate information, while augmented reality (AR) devices suffer from sensor drift and noise, leading to location inaccuracies and lack of integration with other data for accurate 3-D model generation.
Innovation Solution
A method leveraging augmented reality frameworks to capture images and utilize world map data with non-camera anchors, combined with computer vision techniques and sensor data, to generate accurate 3-D models by selecting candidate poses and applying a scaling factor based on geometric transformations and illumination data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video input or large image sets are used for 3-D reconstruction, then measurement precision is improved, but computing resources and cost increase significantly
Solution Approach 1:
The patent applies partial action by selecting only a subset of images from the captured image set for 3-D reconstruction. Instead of processing all images, the system identifies and uses only those images that contain sufficient feature points and geometric information, thereby reducing computing resource consumption while maintaining measurement precision.
Solution Approach 2:
The patent extracts essential geometric information and feature points from the image set to create a simplified representation. By taking out only the necessary measurement data and geometric constraints from the full image set, the system achieves accurate 3-D reconstruction without processing the entire large dataset, thus reducing computational burden.
2Use of energy by moving object
If image sets with sparse data are used, then computing resources are reduced, but adequate information for realistic rendering and accurate measurements is not captured
Solution Approach 1:
The patent performs preliminary action by pre-processing the image set to identify and select images that contain sufficient geometric and illumination information before 3-D reconstruction. The system evaluates each image for feature point density, geometric constraints, and lighting conditions, selecting only those that meet predetermined thresholds, thereby ensuring adequate information is captured while minimizing data volume.
Solution Approach 2:
The patent changes parameters by adjusting the selection criteria for images based on geometric information density and illumination quality. By dynamically adjusting thresholds for feature point count, geometric constraint satisfaction, and lighting adequacy, the system optimizes the balance between data completeness and computing resource usage.
3Ease of operation
If AR device sensor data is used for 3-D reconstruction, then ease of operation is improved, but location accuracy deteriorates due to sensor drift and noise
Solution Approach 1:
The patent introduces an intermediary step where AR device sensor data is not directly used for 3-D reconstruction but serves as a guide for image capture. The sensor data (accelerometer, gyroscope, magnetometer) provides rough location and orientation information that guides the user to capture images from appropriate viewpoints, while the actual 3-D model is built from image features rather than raw sensor data, thereby eliminating sensor drift issues.
Solution Approach 2:
The patent replaces the mechanical sensor-based positioning system with an optical-based feature matching system. Instead of relying on physical sensors that drift, the system uses visual features from images to determine camera positions and orientations through photogrammetry and feature point matching, substituting a more accurate optical measurement system for the unreliable mechanical sensor system.
Data Source
AI summary
System and method are provided for scaling a 3-D representation of a building structure. The method includes obtaining world map data including a first track of real-world poses for a plurality of images. The plurality of images comprises non-camera anchors. The method also includes detecting a discrepancy in at least one real-world pose of the first track. The method also includes in response to detecting a discrepancy, generating a new track of real-world poses. The method also includes calculating a scaling factor for a 3-D representation of the building structure based on sampling across a plurality of tracks. The plurality of tracks comprises at least the first track and the new track.