Dense Point-Cloud Prediction via 2D Projection and Error Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commercialized LIDAR sensors generate sparse point-clouds due to limited beams, leading to high cost and challenges in predicting depth accurately, especially in areas with high sparsity or abrupt depth changes, without RGB guidance, which is unreliable in 360° applications, night, or bad weather.
Innovation Solution
A method and system that project three-dimensional sensor data to a two-dimensional image space to obtain sparse depth data, predict a depth map and error-map using convolutional neural networks, and output a high-confidence point-cloud based on these predictions, without requiring RGB guidance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LIDAR sensors increase the number of beams to reduce sparsity, then depth prediction accuracy improves, but sensor cost increases substantially
Solution Approach 1:
The patent creates a dense point-cloud copy by projecting sparse LIDAR data to 2D image space and using convolutional neural networks to predict depth values for missing regions. This virtual dense point-cloud replicates the effect of having more LIDAR beams without the associated cost, resolving the contradiction between measurement precision and manufacturing cost
Solution Approach 2:
The patent introduces 2D image space as an intermediary representation between sparse 3D LIDAR data and the desired dense 3D point-cloud. By transforming data to 2D space where interpolation is more effective, then projecting back to 3D, the system achieves high-depth prediction accuracy with low-cost sensors
2Ease of manufacture
If commercialized LIDAR sensors are used with limited beams, then sensor cost is reduced, but point-cloud sparsity increases
Solution Approach 1:
The patent transforms the 3D sparse point-cloud problem into a 2D depth prediction problem by projecting to image space. In 2D, the spatial relationships are more compact and convolutional networks can effectively interpolate missing depth values. This dimensional transformation enables dense point-cloud generation without requiring dense 3D sampling, maintaining low sensor cost while increasing point-cloud density
3Measurement precision
If RGB guidance is used to improve depth prediction in sparse areas, then depth accuracy improves, but system complexity and cost increase due to additional sensor requirements
Solution Approach 1:
The patent extracts and removes the dependency on RGB guidance from the depth prediction system. By developing a standalone convolutional network that operates purely on sparse depth data in 2D image space, the solution achieves high depth accuracy without requiring additional RGB sensors, thereby reducing device complexity while maintaining measurement precision
Data Source
AI summary
Methods and systems may be used for obtaining a high-confidence point-cloud. The method includes obtaining three-dimensional sensor data. The three-dimensional sensor data may be raw data. The method includes projecting the raw three-dimensional sensor data to a two-dimensional image space. The method includes obtaining sparse depth data of the two-dimensional image. The method includes obtaining a predicted depth map. The predicted depth map may be based on the sparse depth data. The method includes obtaining a predicted error-map. The predicted error map may be based on the sparse depth data. The method includes outputting a high-confidence point-cloud. The high-confidence point-cloud may be based on the predicted depth map and the predicted error-map.