Time-to-Contact Estimation via Vision-Range Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current obstacle detection systems for autonomous navigation face challenges in providing accurate and rapid time-to-contact (TTC) estimates, especially in cluttered environments, due to limitations in depth sensing, high computational requirements, and high SWaP constraints of existing sensors like LIDAR and monocular cameras.
Innovation Solution
A system that combines two-dimensional and three-dimensional sensing to generate accurate time-to-contact maps by segmenting images, calculating motion fields, and fusing range depth data, using a spiking neural network for neuromorphic foreground detection and range-seeded propagation to integrate salient features for precise obstacle boundary detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If monocular cameras are used for obstacle detection, then SWaP requirements are met, but depth data and TTC estimation become unreliable
Solution Approach 1:
The patent combines monocular camera data with active range sensor data to create a fused depth map. The camera provides wide-field coverage while the range sensor provides accurate depth measurements at key locations, merging their strengths to achieve both low SWaP and high measurement precision for TTC estimation.
Solution Approach 2:
The patent introduces an intermediary processing layer that uses optical flow from the camera to guide range sensor measurements. The optical flow identifies moving objects and their velocities, which then inform where to place range sensor beams, creating an intermediary step that bridges the gap between passive vision and active ranging.
2Measurement precision
If active range sensors like LIDAR are used, then accurate depth measurement is achieved, but SWaP constraints are violated
Solution Approach 1:
The patent segments the scene into regions of interest based on optical flow detection, then applies range sensing only to those specific regions rather than scanning the entire field of view. This segmentation approach maintains measurement precision for critical objects while dramatically reducing the SWaP requirements compared to full-field LIDAR.
Solution Approach 2:
The patent uses partial action by applying range sensing only where necessary (at detected object locations) rather than continuously scanning the entire environment. This provides sufficient depth measurement accuracy for obstacle avoidance while using minimal sensor resources.
3Device complexity
If passive vision methods are used for TTC estimation, then computational requirements are reduced, but reliability and speed deteriorate
Solution Approach 1:
The patent performs preliminary optical flow computation to identify objects and their velocities before conducting range measurements. This preliminary action allows the system to focus computational resources on critical objects and use simple geometric calculations for TTC estimation, achieving both low complexity and high reliability.
Solution Approach 2:
The patent replaces complex computer vision algorithms with a simpler physics-based approach. By measuring object velocity via optical flow and distance via range sensor, then applying basic kinematic equations (TTC = distance/velocity), the system achieves reliable TTC estimation with minimal computational complexity.
4Measurement precision
If stereo vision is used for depth estimation, then depth information is obtained, but working range and texture dependency are limited
Solution Approach 1:
The patent merges stereo vision depth estimation with active range sensor measurements. The stereo provides depth information across the entire field of view, while the range sensor provides accurate measurements at specific distances. This combination extends the working range and improves adaptability to texture-less environments where stereo alone would fail.
Data Source
AI summary
Described is a robotic system for detecting obstacles reliably with their ranges by a combination of two-dimensional and three-dimensional sensing. In operation, the system receives an image from a monocular video and range depth data from a range sensor of a scene proximate a mobile platform. The image is segmented into multiple object regions of interest and time-to-contact (TTC) value are calculated by estimating motion field and operating on image intensities. A two-dimensional (2D) TTC map is then generated by estimating average TTC values over the multiple object regions of interest. A three-dimensional TTC map is then generated by fusing the range depth data with image. Finally, a range-fused TTC map is generated by averaging the 2D TTC map and the 3D TTC map.