Neural Depth Mapping From Camera Images for Occluded Obstacle Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional autonomous vehicles rely on limited-range laser and radar sensors for depth mapping, which struggle to detect objects hundreds of meters away, and lack accurate depth information for occluded obstacles, limiting their navigation and control decisions.
Innovation Solution
Implementing a fully-learned neural network that generates a depth map from camera images, using known depth data to assign depth values to locations, allowing for better estimation of free space and occluded obstacles, and integrating this data with other sensor inputs for enhanced navigation and control.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If laser and radar sensors are used for depth mapping, then depth information can be obtained, but the sensing range is limited and cannot detect objects hundreds of meters away
Solution Approach 1:
The patent combines camera images with known depth data from multiple sources (laser, radar, and previously captured images) to create a comprehensive depth map. This merging allows the system to leverage the long-range capability of cameras while supplementing with accurate depth measurements from other sensors, thereby extending the effective sensing range beyond what any single sensor can achieve alone.
Solution Approach 2:
The patent introduces a neural network as an intermediary that processes camera images and infers depth information for distant objects. This neural network mediator bridges the gap between camera images (which have long range but no inherent depth) and the need for accurate depth measurement, enabling depth estimation for objects hundreds of meters away that would be beyond the direct sensing capability of laser or radar alone.
2Loss of information
If conventional sensors are used, then depth map can be generated, but accurate depth information for occluded obstacles is not available
Solution Approach 1:
The patent captures and stores known depth data in advance from multiple viewpoints and sensor types, building a comprehensive depth database before the actual navigation task. This preliminary action allows the system to have depth information about occluded obstacles already available, reducing the need for complex real-time sensing of hidden objects during critical moments.
Solution Approach 2:
The patent uses a neural network that learns from training data consisting of images paired with known depth information. The network continuously improves its depth estimation capability by processing training examples and adjusting its parameters, creating a feedback loop that enhances the system's ability to infer depth of occluded objects without adding physical sensors.
3Length of stationary object
If camera systems are used instead of laser and radar, then higher range and resolution are achieved, but direct depth measurement capability is lost
Solution Approach 1:
The patent replaces direct mechanical depth measurement (laser ranging, radar time-of-flight) with a computational approach using neural networks. Instead of using physical sensors that directly measure depth, the system uses image processing and machine learning to infer depth from visual information, substituting a computational system for a physical measurement system.
Solution Approach 2:
The patent creates a composite depth map that combines information from multiple sources: camera images providing visual detail and range, plus known depth data from laser and radar sensors. This composite approach allows the system to maintain the long-range advantage of cameras while incorporating accurate depth measurements where available, creating a unified depth representation that leverages the strengths of different data types.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a depth map of a scene from a camera image using a neural network. One of the methods includes obtaining an image captured by a first sensor. A neural network processes the image to generate a respective score for each of a plurality of locations in the image. Known depth data specifying respective known depth values for some of the locations in the image is obtained. A depth output is generated that assigns a depth value to some of the locations in the image, including determining whether the score for a location exceeds a threshold; and when the score exceeds the threshold and the known depth value is available for the location, assigning the known depth value for the location to the location in the depth output.