BIM Camera Tracking for Drift-Resistant Indoor Pose Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In vast indoor environments like semiconductor manufacturing fabs, unrefined camera tracking methods lead to drift over time, making accurate camera pose estimation and correct overlay of building information modeling (BIM) on work sites impossible.
Innovation Solution
A camera tracking apparatus utilizing BIM-based methods that include edge and face detection from RGB and depth maps, combined with geometric features and deep learning-based segmentation, to improve accuracy and robustness by minimizing errors in camera pose estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If general camera tracking methods are used in vast indoor environments, then the system is simple to implement, but drift accumulates over time making visual inspection impossible
Solution Approach 1:
The patent segments the tracking process into multiple independent modules: edge detection module (detecting edges from RGB and depth images), face detection module (identifying planar surfaces), feature matching module (correlating scene features with BIM model features), and pose estimation module (calculating camera position and orientation). Each module handles a specific aspect of tracking, preventing error accumulation while maintaining overall system functionality.
Solution Approach 2:
The patent introduces BIM model features as an intermediary reference system. Instead of directly tracking between consecutive frames which causes drift, the system mediates tracking through static BIM model features (edges and faces of building structures). This intermediary reference framework provides a stable coordinate system that eliminates drift accumulation over time.
2Measurement precision
If edge and face detection algorithms are applied to pillars and floors, then measurement precision of camera pose is improved, but device complexity increases
Solution Approach 1:
The patent applies different detection algorithms to different local features: edge detection (Canny operator) is applied specifically to pillar boundaries where vertical edges are prominent, while face detection algorithms are applied to floor surfaces where planar geometry is characteristic. This localized approach optimizes measurement precision for each feature type without requiring complex universal algorithms.
Solution Approach 2:
The patent changes detection parameters based on feature type: for pillar edges, it uses depth map gradient thresholds and normal map orientations specific to vertical structures; for floor faces, it uses plane fitting parameters and angle constraints appropriate for horizontal surfaces. These parameter adjustments improve measurement precision while keeping the base algorithms relatively simple.
3Reliability
If multiple error minimization criteria are applied (edge error, face error, gravity error, camera pose error), then reliability of tracking is improved, but computational complexity increases
Solution Approach 1:
The patent implements periodic optimization where error minimization is performed in discrete stages rather than continuously. The system alternates between edge-based pose estimation, face-based pose estimation, and gravity vector correction in periodic cycles. This periodic approach maintains tracking robustness through multiple error criteria while reducing computational power requirements by avoiding continuous simultaneous optimization of all parameters.
Data Source
AI summary
A camera tracking apparatus for supporting segmentation based on a building information modeling (BIM) in an indoor environment receives a red/green/blue (RGB) image and a depth map including a pillar photographed by a red/green/blue-depth (RGB-D) camera, detects an RGB edge and a depth edge of a pillar using the RGB image and the depth map, calculates both end points and removes an outlier from a result of combining the RGB edge and the depth edge and detects a scene edge corresponding to the pillar, searches for a BIM edge corresponding to the scene edge using the scene edge, detects a scene face of a floor and the pillar using a face detection algorithm, searches for a BIM face corresponding to the scene face using a center point of the scene face, and removes an incorrect matching result.