Hybrid Feature Matching for 3D Scan Colorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D scanners face challenges in efficiently matching features between intensity and color images, particularly near edges, due to limitations in active and passive sensor data integration, which affects the accuracy and speed of colorization and feature localization in scanned environments.
Innovation Solution
A system and method that utilize a 3D scanner capturing point clouds, an intensity image, and an ultrawide-angle color camera to perform hybrid feature matching by extracting features from intensity images, creating templates, and computing cost values for pixel matching in a transformed luminance image, enabling reliable correspondences and fast colorization of 3D data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If feature matching is performed between intensity image and color image using conventional methods, then feature correspondences can be established, but matching accuracy deteriorates near edges and in high-contrast regions
Solution Approach 1:
The patent segments the feature matching process into multiple stages: (1) detecting features in the intensity image, (2) creating templates around detected features, (3) transforming the color image to luminance, (4) creating search windows in the transformed image, and (5) computing cost values to determine matches. This segmentation allows each stage to be optimized independently, improving overall accuracy especially in challenging edge regions.
Solution Approach 2:
The patent introduces an intermediary transformed luminance image derived from the color image. This intermediate representation serves as a bridge between the intensity image and the original color image, enabling more robust feature matching by comparing features in a standardized luminance space before mapping back to color space for final colorization.
2Measurement precision
If multiple images with different exposures are acquired to enhance color accuracy, then color representation improves, but acquisition time increases significantly
Solution Approach 1:
The patent extracts the luminance information from the color image and uses it as the basis for feature matching and colorization. By separating the luminance component from chromatic information and using only the luminance for matching purposes, the system achieves accurate feature correspondence without requiring multiple exposure images, thus reducing acquisition time while maintaining color accuracy.
Solution Approach 2:
The patent creates a transformed luminance image as a copy of the color image's intensity information. This copied luminance representation is then used for feature matching instead of requiring multiple original color images with different exposures. The copying approach preserves the essential intensity information needed for accurate matching while avoiding the time cost of acquiring multiple exposures.
3Measurement precision
If feature matching is performed across the entire color image, then comprehensive colorization is achieved, but computational complexity increases
Solution Approach 1:
The patent applies local quality by creating search windows of specific sizes around detected features rather than performing exhaustive matching across the entire image. The template size and search window size are optimized locally around each feature point, providing sufficient context for accurate matching while significantly reducing the overall computational burden compared to full-image processing.
Solution Approach 2:
The patent performs preliminary actions by first detecting features in the intensity image and creating templates before proceeding to the matching stage. Features are pre-identified and organized, and the color image is pre-transformed to luminance space. This preliminary organization of data structures and feature detection results significantly reduces the computational complexity of the subsequent matching and colorization processes.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach improves feature matching between active and passive sensor data, providing accurate and efficient colorization of 3D scans, reducing computational complexity, and enhancing on-site compensation, thus addressing the technical challenges of edge localization and dynamic range variations.
Implementation Method 1
A TOF laser scanner is a scanner in which the distance to a target point is determined based on the speed of light in air between the scanner and a target point
Implementation Method 2
Laser scanners are typically used for scanning closed or open spaces such as interior areas of buildings, industrial installations and tunnels. They may be used, for example, in industrial applications and accident reconstruction applications. A laser scanner optically scans and measures objects in a volume around the scanner through the acquisition of data points representing object surfaces within the volume. Such data points are obtained by transmitting a beam of light onto the objects and collecting the reflected or scattered light to determine the distance, two-angles (i.e., an azimuth and a zenith angle), and optionally a gray-scale value.
Data Source
AI summary
A point cloud is colorized by mapping a color image using an intensity image. The mapping includes detecting multiple features from the intensity image using a feature-extraction algorithm. A feature is extracted that is not within a predetermined vicinity of an edge in the intensity image. A template is created by selecting a portion of a predetermined size from the intensity image with the feature at the center. A search window is created with the same size as the template by selecting a portion of a luminance image as a search space. The luminance image is obtained from the color image. A cost value is computed for each pixel of the search space by comparing image gradients of the template and the search window. A matching point is determined in the color image corresponding to the feature based on the cost value for each pixel of search space.