Range-Fused TTC Mapping for Monocular Obstacle Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current obstacle detection systems for autonomous navigation face challenges in providing accurate and rapid time-to-contact (TTC) estimates, especially in cluttered environments, due to limitations in depth sensing, high computational requirements, and high SWaP constraints of existing sensors like LIDAR and monocular cameras.
Innovation Solution
A system that combines two-dimensional and three-dimensional sensing to generate accurate time-to-contact maps by segmenting images, calculating motion fields, and fusing range depth data, using a spiking neural network for neuromorphic foreground detection and range-seeded propagation to integrate salient features for precise obstacle boundary detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If active range sensing (LIDAR, RADAR, 3D LIDAR) is used to provide accurate depth data and TTC estimation, then measurement precision and reliability are improved, but device complexity, size, weight, and power consumption increase significantly
Solution Approach 1:
The patent combines 2D image data from a monocular camera with 1D range data from a laser scanner to create a fused TTC map. This merging allows the system to achieve accurate depth estimation and TTC calculation without requiring complex 3D LIDAR systems, thereby reducing device complexity while maintaining measurement precision.
Solution Approach 2:
The system uses a monocular camera that can serve multiple functions: capturing 2D images for obstacle detection, providing texture information for feature matching, and contributing to TTC estimation when fused with range data. This multi-functionality reduces the need for specialized sensors, lowering overall device complexity.
2Device complexity
If monocular cameras are used to meet low SWaP requirements, then device complexity and power consumption are reduced, but the ability to provide accurate depth data and TTC estimation deteriorates
Solution Approach 1:
The patent introduces a laser scanner as an intermediary sensor that provides 1D range data to complement the monocular camera's 2D image data. This intermediary range information enables accurate depth estimation and TTC calculation without requiring the monocular camera to perform depth sensing alone, thus maintaining measurement precision while keeping device complexity low.
Solution Approach 2:
The system transitions from 2D image data alone to a fusion of 2D image data with 1D range data, effectively adding a depth dimension to the monocular camera's capabilities. This dimensional enhancement allows accurate depth and TTC estimation while maintaining the simplicity of using a monocular camera as the primary sensor.
3Device complexity
If passive vision methods (optical flow, stereo, structure from motion) are used for obstacle detection, then device complexity is reduced, but reliability and accuracy of TTC estimation deteriorate due to inconsistent feature tracking, texture-less environments, and intensive computation
Solution Approach 1:
The system performs preliminary segmentation of the image into foreground and background regions before TTC estimation. This preliminary action isolates relevant obstacle regions, improving the reliability of detection by focusing computational resources on salient features and reducing the impact of texture-less environments and inconsistent feature tracking.
Solution Approach 2:
The patent segments the image into multiple regions of interest (foreground objects) and processes each region separately for TTC estimation. This segmentation improves reliability by focusing on salient obstacle regions, reducing the impact of background noise and texture-less areas, and enabling more robust feature tracking in complex environments.
4Measurement precision
If high computational methods (scale change detection, feature tracking) are used to improve TTC estimation accuracy, then measurement precision is improved, but productivity and processing speed deteriorate
Solution Approach 1:
The system performs preliminary segmentation to identify foreground regions before TTC estimation, which reduces the computational domain to only relevant obstacle areas. This preliminary action maintains measurement precision by focusing on salient features while improving processing speed by eliminating unnecessary computation in background regions.
Solution Approach 2:
The patent applies computational methods selectively to only the segmented foreground regions rather than processing the entire image. This partial action maintains TTC estimation accuracy for obstacles while significantly reducing overall processing time and computational load, thereby improving productivity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Described is a robotic system for detecting obstacles reliably with their ranges by a combination of two-dimensional and three-dimensional sensing. In operation, the system receives an image from a monocular video and range depth data from a range sensor of a scene proximate a mobile platform. The image is segmented, into multiple object regions of interest and time-to-contact (TTC) value are calculated by estimating motion field and operating on image intensities. A two-dimensional (2D) TTC map is then generated by estimating average TTC values over the multiple object regions of interest. A three-dimensional TTC map is then generated by fusing the range depth data with image. Finally, a range-fused TTC map is generated by averaging the 2D TTC map and the 3D TTC map,