Thermal Sensor Fusion for Occluded Vehicle Object Perception
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in detecting and tracking agents under certain conditions and with respect to specific types of agents.
Innovation Solution
Implementing thermal imaging in conjunction with non-thermal imaging for object detection, which improves detection of occluded objects, prioritizes heat-emitting objects in the perception pipeline, and differentiates between inactive and active objects, reducing processing requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If thermal imaging is implemented in conjunction with non-thermal imaging for object detection, then detection accuracy is improved, but device complexity increases
Solution Approach 1:
The patent combines thermal imaging sensors with non-thermal imaging sensors (such as visible light cameras) into a unified perception system. This merging allows the system to detect objects using multiple modalities simultaneously, improving detection accuracy by leveraging the complementary strengths of thermal and non-thermal data, while the integrated architecture manages the complexity through coordinated processing.
Solution Approach 2:
The perception system is designed to handle multiple types of sensors and multiple detection modes (thermal and non-thermal) within a single unified framework. This multi-functionality allows the system to adapt to different detection scenarios and object types, improving overall detection accuracy while managing device complexity through a versatile processing architecture.
2Measurement precision
If thermal imaging is used to detect occluded objects, then detection capability is improved, but processing requirements increase
Solution Approach 1:
The system applies partial processing by focusing computational resources on specific tasks where thermal imaging provides the most value, such as detecting occluded objects or objects in low-visibility conditions. Rather than processing all sensor data at full resolution and detail, the system selectively processes thermal data to supplement non-thermal detection, reducing overall processing requirements while maintaining improved detection capability for challenging scenarios.
3Loss of time
If heat-emitting objects are prioritized in the perception pipeline, then reaction time is improved, but measurement precision may be compromised
Solution Approach 1:
The perception pipeline performs preliminary processing and prioritization of thermal data early in the processing sequence, identifying heat-emitting objects before full analysis is complete. This preliminary action allows the system to quickly react to thermally active objects (such as living beings or engines) while subsequently applying more rigorous precision analysis, thus improving reaction time without permanently compromising measurement precision.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances detection accuracy and speed, improves reaction time, and optimizes processing by prioritizing heat-emitting objects in the perception pipeline.
Implementation Method 1
obtain thermal sensor data associated with an environment in which an autonomous vehicle is operating
Data Source
AI summary
Provided are methods for thermal sensor data vehicle perception, which can include obtaining thermal sensor data, obtaining non-thermal sensor data, and determining, based on the thermal sensor data and the non-thermal sensor data, a perception parameter indicative of an object. Some methods described also include generating a trajectory for an autonomous vehicle. Systems and computer program products are also provided.