Aligning Street-Level Imagery to 3D Building Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in accurately aligning street-level images with 3D building models due to location mismatches caused by signal distortions and mechanical inaccuracies, leading to misalignment issues.
Innovation Solution
A method that adjusts the origin point of image data systematically to achieve best match with 3D building models by combining image and high-resolution scanning, using street-level LIDAR data to extract building edges and skylines, and iteratively displacing the camera location to minimize alignment cost, incorporating images from both sides of a street for enhanced accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GPS and IMU are used to determine vehicle location, then positioning capability is provided, but signal distortions and mechanical tolerances cause location mismatch between image sensors and actual location
Solution Approach 1:
The system uses feedback by iteratively comparing extracted image features (edges, skylines) with corresponding 3D model features, calculating a cost function, and adjusting the camera location until the cost is minimized. This closed-loop feedback process compensates for initial GPS/IMU location errors and achieves accurate alignment between images and 3D models.
Solution Approach 2:
The patent replaces reliance on mechanical positioning systems (GPS/IMU) with an optical-based alignment system. Instead of trusting mechanical location data, the system uses image feature extraction and 3D model comparison to determine the actual camera location, substituting mechanical measurement with optical measurement and computational geometry.
2Productivity
If automatic alignment algorithms are used, then alignment speed is improved, but accuracy is insufficient due to unpredictable location mismatches
Solution Approach 1:
The system moves from 2D image space to 3D model space for alignment. By extracting features from images and comparing them with 3D model features (edges, skylines, facade depths), the system performs alignment in three-dimensional space, achieving both automation and high accuracy that 2D algorithms cannot provide.
Solution Approach 2:
The patent introduces 3D building models as an intermediary reference framework. Instead of directly aligning images with GPS coordinates, the system uses 3D models as a mediator that connects image features with geographic locations, enabling accurate automated alignment by comparing image-extracted features with model features.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides accurate alignment of images with 3D models, improving geographic location precision and enhancing the realism of 3D imagery by automating the image-to-model matching process, suitable for large-scale geographic modeling applications.
Implementation Method 1
airborne laser ranging (LIDAR)
Implementation Method 2
laser beams and measuring reflected light
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Point of origin information for image data may be inaccurately registered against a geographic location absolute. A process for aligning image and highly accurate model data adjusts a point of origin of the image data by matching elements in the image with corresponding elements of the model. In a street-level image, building skylines can be extracted and corresponding skylines from the building model can be placed over the image-based skyline. By adjusting the point of origin of the image, the respective skylines can be aligned. Building edge and facade depth information can similarly be matched by adjusting the image point of origin of the image. The adjusted point of origin of the image can be used to then automatically place images on the models for a long run of images.