Point Cloud Colorization Using Occlusion-Aware Image Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for colorizing point clouds using images from 3D scanners result in inaccurate color assignment due to occlusions, leading to computational inefficiencies and low accuracy, especially when using ray casting algorithms.
Innovation Solution
Selecting images with the shortest subject distance to un-occluded points in the point cloud for color assignment, using a processing system to determine the most accurate color information from un-occluded images during post-processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If ray casting algorithms are used to search for un-occluded points in images, then color accuracy is improved, but computational expense increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-processing images to generate depth maps before colorization. The depth maps are created by comparing point cloud distances with image plane distances, allowing occlusion detection to be performed efficiently during colorization without requiring computationally expensive ray casting algorithms at processing time.
Solution Approach 2:
The patent replaces the mechanical ray casting algorithm with an optical-inspired depth map comparison approach. Instead of casting rays through 3D space to detect occlusions, the system uses pre-computed depth information from depth maps to quickly determine whether a point is occluded, significantly reducing computational requirements.
2Reliability
If images that are not spatially close to the point being colored are used, then occlusion problems are avoided, but mapping accuracy decreases due to sensor and position errors
Solution Approach 1:
The patent performs preliminary occlusion detection using depth maps before selecting images for colorization. By pre-processing depth information and comparing it with point cloud distances, the system identifies un-occluded points in advance, allowing nearby images to be used without the risk of applying occluding object colors.
Solution Approach 2:
The patent implements feedback by using depth map comparison results to guide image selection for colorization. The system continuously checks whether a point is occluded by comparing depth map distances with point cloud distances, and only selects images where the point is confirmed un-occluded, ensuring both accuracy and reliability.
3Productivity
If conventional geo-referenced image matching is used for colorization, then processing speed is maintained, but color accuracy deteriorates due to occlusion artifacts
Solution Approach 1:
The patent applies preliminary action by generating depth maps and identifying un-occluded points before the colorization process. This pre-processing step enables the system to quickly filter out occluded points and only apply color from images where the target point is visible, eliminating occlusion artifacts while maintaining efficient processing throughput.
Solution Approach 2:
The patent extracts and removes occluded points from the colorization process by using depth map comparison. Points that are occluded in the reference image are identified and excluded from colorization, ensuring that only un-occluded points receive color information, thereby eliminating occlusion artifacts while preserving processing efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods are described for colorizing a point cloud comprising a set of point cloud data points associated with a particular object using a plurality of candidate images. For each respective data point in the set of point cloud data points, an image is selected, from a subset of the plurality of candidate images that include an un-occluded point of the particular object corresponding to the respective data point, that has a shortest subject distance to the un-occluded point of the particular object. The respective data point is colorized based on a color in the selected image of the un-occluded point of the particular object corresponding to the respective data point.