Convolutional Neural Network Depth Map Estimation for Autonomous Vehicles
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current depth map estimation algorithms, such as stereo matching, face challenges in real-time computation and accuracy due to dependency on window size and complexity in processing stereo images, making them inefficient for autonomous vehicle piloting.
Innovation Solution
A convolutional deep neural network (CDNN) system is implemented to process stereo images, using multiple layers for disparity calculation and upsampling, trained with ground truth data to determine accurate depth maps, enabling real-time processing and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional stereo matching algorithms are used for depth map estimation, then processing can be performed with simpler computational structures, but real-time processing capability and accuracy are compromised due to dependency on window size and computational complexity
Solution Approach 1:
The patent replaces traditional mechanical stereo matching algorithms with a deep neural network-based computational system. The CDNN architecture processes stereo images through multiple convolutional layers, replacing iterative window-based matching with parallel neural network computations that achieve real-time performance while maintaining or improving accuracy.
Solution Approach 2:
The patent introduces a multi-layer neural network architecture that adds computational dimensions beyond traditional 2D image processing. By stacking convolutional layers and using 3D convolutions across stereo pairs, the system processes spatial and depth information in additional dimensional spaces, enabling real-time depth estimation without being constrained by traditional window size dependencies.
2Measurement precision
If larger window sizes are used in stereo matching algorithms, then measurement accuracy may be improved, but processing time and computational load increase significantly
Solution Approach 1:
The patent segments the depth estimation problem into multiple processing stages through the neural network architecture. Different layers of the CDNN handle different aspects of feature extraction and disparity calculation, allowing the system to achieve high accuracy without requiring large window sizes. Each layer processes localized features and combines them hierarchically, replacing the need for large computational windows.
Solution Approach 2:
The patent performs preliminary feature extraction and matching in early neural network layers before final depth calculation. By pre-processing stereo image pairs through multiple convolutional layers that extract relevant features and reduce dimensionality, the system prepares optimized representations that enable accurate depth estimation with reduced computational requirements and faster processing.
Data Source
AI summary
Vehicles can be equipped to operate in both autonomous and occupant piloted mode. While operating in either mode, an array of sensors can be used to pilot the vehicle including stereo cameras and 3D sensors. Stereo camera and 3D sensors can also be employed to assist occupants while piloting vehicles. Deep convolutional neural networks can be employed to determine estimated depth maps from stereo images of scenes in real time for vehicles in autonomous and occupant piloted modes.