3D Road Surface Estimation Using Neural Network Densification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing 3D road surface reconstruction techniques for autonomous driving are inadequate due to the limitations of LiDAR sensors, such as high cost and limited range, and camera-based methods that struggle with accuracy and computational efficiency, especially in complex urban environments.
Innovation Solution
The use of cameras to capture images of the 3D environment, combined with densification techniques like Markov random fields and deep neural networks, to generate a dense representation of the 3D road surface, enabling accurate and efficient reconstruction for autonomous vehicle navigation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If LiDAR sensors are used for 3D road surface reconstruction, then measurement precision is improved, but device cost increases and measurement range is limited
Solution Approach 1:
The patent replaces LiDAR sensors with a camera-based vision system for 3D road surface reconstruction. The camera captures 2D images which are then processed through multi-view geometry and densification algorithms to reconstruct 3D road surface information, eliminating the need for expensive LiDAR hardware while maintaining measurement capability
Solution Approach 2:
The patent creates a virtual 3D representation of the road surface by processing 2D camera images through computational algorithms. Instead of directly measuring 3D points with LiDAR, the system creates a computational copy of the 3D road surface structure from 2D image data, achieving similar functional results with different hardware
2Device complexity
If conventional camera-based multi-view geometry is used for 3D reconstruction, then device cost is reduced, but measurement precision and computational efficiency deteriorate
Solution Approach 1:
The patent applies densification techniques to the sparse 3D points obtained from multi-view geometry before final reconstruction. By pre-processing the sparse point cloud to infer additional 3D points from 2D image correspondences, the system improves measurement precision while maintaining the cost advantage of camera-based systems
Solution Approach 2:
The patent introduces 2D projection images and densification algorithms as intermediary steps between camera capture and 3D reconstruction. This intermediate processing stage allows the system to bridge the gap between 2D image data and accurate 3D road surface representation, improving precision without requiring LiDAR hardware
3Productivity
If conventional interpolation or plane fitting post-processing is applied to camera-based reconstruction, then computational complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent employs iterative densification algorithms that dynamically refine 3D point estimates by repeatedly processing 2D image correspondences. Rather than using static interpolation or plane fitting, the system adaptively improves precision through multiple refinement passes, balancing computational efficiency with accuracy for complex road surfaces
Data Source
AI summary
In various examples, to support training a deep neural network (DNN) to predict a dense representation of a 3D surface structure of interest, a training dataset is generated using a parametric mathematical modeling. A variety of synthetic 3D road surfaces may be generated by modeling a 3D road surface using varied parameters to simulate changes in road direction and lateral surface slope. In an example embodiment, a synthetic 3D road surface may be created by modeling a longitudinal 3D curve and expanding the longitudinal 3D curve to a 3D surface, and the resulting synthetic 3D surface may be sampled to form a synthetic ground truth projection image (e.g., a 2D height map). To generate corresponding input training data, a known pattern that represents which pixels may remain unobserved during 3D structure estimation may be generated and applied to a ground truth projection image to simulate a corresponding sparse projection image.