Unobstructed Building Facade View Generation via Multi-View Image Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing navigation and mapping programs provide limited detail and obstructed views of building facades in street-level imagery, as images are captured from varying angles with potential obstructions like vehicles, making it difficult to obtain a clear, unobstructed view of objects.
Innovation Solution
A method to generate unobstructed images of building facades by capturing images from multiple points-of-view, normalizing them to create a merged image using average or median pixel values, and applying transformations like shift, rotation, or affine transformations to produce a two-dimensional view from a perspective normal to the facade, without requiring obstruction detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If images are captured from multiple points-of-view to reduce obstructions, then the completeness of the facade view is improved, but the image processing complexity increases
Solution Approach 1:
The method segments the image processing task into distinct steps: capturing multiple images from different points-of-view, normalizing each image individually (rectifying perspective, applying transformations), and then merging them. This segmentation allows complex multi-image processing to be broken down into manageable, systematic operations that can be automated.
Solution Approach 2:
The patent introduces normalized images as an intermediary step between the captured raw images and the final merged image. Each captured image is first normalized (rectified and transformed) to a standard format, creating intermediate normalized images that are then merged. This intermediary normalization process simplifies the merging operation and makes the overall process more systematic.
2Measurement precision
If images are normalized and merged using pixel comparison, then the accuracy of the unobstructed view is improved, but the processing time increases
Solution Approach 1:
The patent applies preliminary normalization (rectification and transformation) to each captured image before merging. By pre-processing each image to a standardized format with known geometric transformations, the subsequent merging process becomes more efficient. The preliminary action of normalization ensures that pixels from different images can be directly compared and merged without requiring complex real-time alignment calculations.
3Manufacturing precision
If rectification and transformation are applied to normalize images, then the quality of the two-dimensional view is improved, but the computational requirements increase
Solution Approach 1:
The patent changes key parameters of the images during normalization: applying perspective rectification to correct geometric distortion, applying rotation transformations to orient facades properly, and applying scaling transformations to unify dimensions. These parameter changes transform raw images into normalized images with consistent geometric properties, enabling high-quality merged views while making the transformations systematic and reusable across multiple images.
Data Source
AI summary
A method is provided for generating street-level imagery to provide unobstructed images, such as images of facades of a building, for use in on-line street viewing, three dimensional city rendering, research, or building based localization, among other uses. In particular, example methods may include obtaining a plurality of images of a first side of an object from a plurality of points-of-view, normalizing each of the plurality of images to obtain a plurality of normalized images of the first side of the object, where each of the plurality of normalized images includes a plurality of pixels, each with image location coordinates within a respective image. Method may further generate a merged image of the first side of the object, where the merged image is generated by comparing pixels between the plurality of normalized images with matching image location coordinates.