Point Cloud Intensity Completion via Semantic Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current point cloud completion methods are limited to local objects and lack comprehensive reflection intensity completion, which is essential for accurate semantic segmentation in unmanned driving, especially in scenarios with sparse point clouds and adverse weather conditions.
Innovation Solution
A method and system that integrate RGB images and point cloud data from lidars to perform spatial transformation, feature stitching, and deep learning-based completion, enabling simultaneous depth and reflection intensity completion with semantic segmentation guidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a lidar with fewer beams is used to reduce cost, then device cost decreases, but point cloud density decreases making high precision calculations difficult
Solution Approach 1:
The patent uses an RGB camera as an intermediary to capture visual information that complements the sparse point cloud data from the low-beam lidar. The camera provides additional spatial and color information that helps reconstruct missing point cloud details, enabling high-precision calculations despite using a cost-effective lidar with fewer beams
Solution Approach 2:
The patent merges data from multiple sources - the point cloud from the lidar and the image data from the RGB camera - into a unified representation. This combination allows the system to achieve high point cloud density and precision while using an affordable lidar configuration
2Reliability
If lidars are used in adverse weather conditions, then data acquisition continues, but beam energy is lost due to atmosphere and distance resulting in undetected or weakened signals
Solution Approach 1:
The RGB camera serves as a mediator that can operate effectively in adverse weather conditions where lidar signals are weakened or lost. When the lidar fails to detect objects due to atmospheric interference or distance, the camera provides complementary visual information to maintain reliable data acquisition
Solution Approach 2:
The system uses feedback from the camera detection results to supplement or correct lidar data in adverse conditions. When lidar signals are lost, the camera's detection capability provides feedback that maintains the reliability of the overall data acquisition system
3Device complexity
If local dependent completion is used, then completion process is simple, but it lacks reference to regions with the same semantics in a larger range reducing accuracy
Solution Approach 1:
The patent transitions from local 2D interpolation to global 3D completion by utilizing the depth dimension and semantic information. The completion process references regions with the same semantics across the entire scene, not just adjacent areas, achieving higher accuracy while maintaining reasonable complexity through efficient algorithms
Data Source
AI summary
A point cloud intensity completion method and system based on semantic segmentation are provided. The point cloud intensity completion method includes: acquiring an RGB image and point cloud data of a road surface synchronously by a photographic camera and a lidar; performing spatial transformation on the point cloud data by using a conversion matrix to generate a two-dimensional reflection intensity projection map and a two-dimensional depth projection map; performing reflection intensity completion on the RGB image and the two-dimensional reflection intensity projection map to obtain a single-channel reflection intensity projection map; performing depth completion on the RGB image and the two-dimensional depth projection map to obtain a single-channel depth projection map; and performing coarse-grained completion on the RGB image, the single-channel reflection intensity projection map and the single-channel depth projection map to obtain a two-dimensional coarse-grained reflectance intensity projection map.