Virtual Stereo Ranging with Thermographic Imaging for Low Visibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing advanced driving assist systems (ADAS) face challenges in stereo matching and ranging due to the limitations of using two RGB cameras, particularly in low light conditions, glare, and low visibility, necessitating the integration of IR cameras, which increases space, cost, and weight on vehicles.
Innovation Solution
A system utilizing a single visible-light camera and a single thermographic camera, combined with an ADAS ECU, performs monocular ranging to generate depth maps and stereo matching using a parallax-based method to overcome the limitations of RGB cameras, enabling effective stereo ranging even with mixed image types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If two RGB cameras are used for stereo matching, then depth estimation can be performed, but performance deteriorates in low light conditions, glare, and low visibility
Solution Approach 1:
The patent combines RGB camera data and IR camera data to perform stereo matching and generate a fused depth map. The system integrates information from both camera types to overcome the limitations of RGB cameras alone in challenging environmental conditions, thereby improving depth estimation reliability.
Solution Approach 2:
The system uses an IR camera that can capture thermal radiation patterns, providing universal functionality across different lighting conditions. The IR camera serves multiple purposes: it works in low light, through glare, and in low visibility conditions where RGB cameras fail, making the overall system more versatile and reliable.
2Reliability
If IR cameras are integrated to improve depth estimation in challenging conditions, then reliability improves, but vehicle space, cost, and weight increase
Solution Approach 1:
The patent segments the depth estimation task into two parallel processing streams: one for RGB images and one for IR images. Each stream is processed independently through monocular ranging and stereo matching, then the results are fused. This segmentation allows the system to leverage IR capabilities without requiring a complete redesign of the entire vision system, thus managing complexity better.
Solution Approach 2:
The patent introduces a fusion module as an intermediary that combines depth maps from both RGB and IR processing streams. This mediator integrates the complementary information from both camera types, allowing the system to achieve improved reliability through IR technology while managing the added complexity through a structured fusion approach rather than direct integration.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances ADAS functionality by improving depth estimation in various environmental conditions, reducing the likelihood of collisions by accurately detecting hazardous objects, without the need for additional IR cameras, thus optimizing space, cost, and weight.
Implementation Method 1
receiving, from an infrared image sensor, IR image data of the surrounding environment of the vehicle
Implementation Method 2
stereo matching using a parallax-based method to overcome the limitations of RGB cameras
Data Source
AI summary
Aspects of the subject technology relate to systems and methods for virtual stereo ranging in a vehicle. A visible-light image and a thermographic image of a surrounding environment of a vehicle are respectively received from a visible-light image sensor and a thermographic image sensor. A visible-light depth map and a thermographic depth map of the surrounding environment are respectively generated based on the visible-light image and the thermographic image using a monocular ranging method. A first boundary surrounding a first object and a second boundary surrounding a second object in the surrounding environment are respectively generated in the visible-light image and the thermographic image. Based on the first and second boundaries, a stereo depth map of the surrounding environment is generated using a stereo ranging method. Driving of the vehicle is controlled based on the visible-light depth map, the thermographic depth map, and the stereo depth map.