Integrated LiDAR-Thermal Imaging for Adverse Weather Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current LiDAR and camera systems in autonomous vehicles struggle to accurately recognize animate objects, especially in adverse weather conditions such as rain, snow, and fog, due to limitations in wavelength response and scattering, leading to difficulties in distinguishing between animate and inanimate objects.
Innovation Solution
A two-waveband image acquisition device that integrates LiDAR and thermal imaging using a single focal plane array with LiDAR photodetectors and thermal photodetectors, operating in different wavebands, allowing for simultaneous image acquisition and spatial correlation of LiDAR and thermal frames to enhance object detection and recognition.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional LiDAR systems use near infrared wavelengths (750 nm to 1.0 μm) with silicon photodetectors, then the system can achieve reliable object detection in clear weather, but the system performance deteriorates in foul weather due to water vapor scattering and attenuation
Solution Approach 1:
The patent changes the operating wavelength parameter from near infrared (750 nm to 1.0 μm) to mid-wave infrared (3 μm to 5 μm). This parameter change allows the LiDAR system to operate in a wavelength range that is less affected by water vapor scattering and attenuation, thereby maintaining reliable object detection in foul weather conditions while using advanced photodetector materials like InSb or MCT that are sensitive to mid-wave infrared wavelengths
Solution Approach 2:
The patent employs composite material approaches by integrating multiple photodetector types (silicon for visible/NIR, InSb or MCT for mid-wave infrared) within a single LiDAR system. This allows the system to leverage the advantages of different materials for different wavelength ranges, achieving robust performance across varying weather conditions through multi-material detection capabilities
2Ease of operation
If current LiDAR and camera systems operate in visible and near infrared wavebands, then the systems can function in clear weather conditions, but they fail to reliably distinguish animate objects in adverse weather due to scattering and blinding effects
Solution Approach 1:
The patent changes the detection wavelength parameter to mid-wave infrared (3 μm to 5 μm), where atmospheric transmission is better and scattering by rain, snow, and fog is reduced. This allows the LiDAR system to maintain ease of operation for distinguishing animate objects in adverse weather conditions by operating in a wavelength range that penetrates obscurants more effectively
Solution Approach 2:
The patent uses thermal radiation as an intermediary mechanism for detection. By detecting the thermal infrared radiation emitted by objects in the mid-wave infrared band, the system can distinguish animate objects (which have distinct thermal signatures) from inanimate objects even in adverse weather, using thermal radiation as a mediator that is less affected by scattering than visible or near infrared light
3Adaptability or versatility
If separate LiDAR and thermal imaging systems are used for object detection, then comprehensive sensing coverage is achieved, but system complexity and computational requirements increase
Solution Approach 1:
The patent merges LiDAR and thermal imaging functions into a single integrated system that operates in the mid-wave infrared band. By combining the ranging capability of LiDAR with the thermal detection capability of thermal imaging in one unified system using a single focal plane array, the patent achieves comprehensive sensing coverage while reducing system complexity and computational requirements compared to using separate systems
Solution Approach 2:
The patent creates a universal imaging system that performs both LiDAR ranging and thermal imaging functions simultaneously using a single focal plane array detector. This multi-functional system can operate in both active (LiDAR) and passive (thermal imaging) modes, providing adaptability and versatility while avoiding the complexity of maintaining separate dedicated systems for each function
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This integrated approach enables more effective and efficient object detection and recognition, even under inclement weather, by utilizing wavelengths that penetrate obscurants and utilize heat signatures, simplifying the system and improving sensing performance.
Implementation Method 1
Conventional LiDAR systems illuminate the environment with pulses of light, and then use silicon photodetectors to register the reflections
Implementation Method 2
The at least one photodetector array is also operable to receive radiated infrared beams within a different second waveband of light from the one or more objects of interest
Implementation Method 3
Using a single focal plane array predominately co-locates LiDAR and thermal photodetectors and therefore objects of interest are also optically correlated
Data Source
AI summary
A method for image acquisition includes receiving, by an image acquisition computing device, a digitized LiDAR image frame and a thermal image frame of a region of interest from a read out integrated circuit of an image acquisition device coupled to the image acquisition computing device. The LiDAR image frame and the thermal image frame are processed to detect one or more objects of interest located in the region of interest. The detected one or more objects of interest are correlated between the LiDAR image frame and the thermal image frame. The detected one or more objects of interest are identified based on the correlation between the LiDAR image frame and the thermal image frame. An integrated LiDAR and thermal image acquisition device is also disclosed.