Multispectral Object Detection for Small Roadway Hazards
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting small animals or debris on the roadway due to inaccuracies inherent in single-modality sensor systems, which can lead to inadequate collision avoidance.
Innovation Solution
Utilize multi-modal or multispectral data from various sensors, including visible light cameras, infrared cameras, LiDAR, and RADAR, to enhance object detection through machine-learned models, and employ data augmentation techniques to generate training data using synthetic and real-world data, along with image translation and style transfer networks to improve detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If single-modality sensor systems are used for object detection, then device complexity is reduced, but measurement precision deteriorates due to inaccuracies in detecting small objects
Solution Approach 1:
The patent combines multiple sensor modalities (visible light camera, infrared camera, LiDAR, RADAR) into an integrated sensor system that captures data across different spectral bands. This merging of sensors allows the system to detect small objects more accurately by leveraging the complementary strengths of each modality, resolving the contradiction between measurement precision and device complexity through unified multi-modal processing
2Measurement precision
If extensive labeled training data is collected for machine-learned models, then detection accuracy is improved, but loss of time increases due to data collection and labeling efforts
Solution Approach 1:
The system performs preliminary actions by using unlabeled multi-modal sensor data to pre-train machine-learned models before deployment. By preparing models in advance with synthetic and unlabeled real-world data, the system reduces the need for extensive manual labeling during actual operation, thereby decreasing time loss while maintaining high detection accuracy
Solution Approach 2:
The patent uses synthetic data that copies and simulates real-world scenarios to train machine-learned models. This copying approach allows the system to generate unlimited training examples without requiring actual physical data collection and labeling, significantly reducing time loss while improving detection accuracy through diverse training samples
Data Source
AI summary
Techniques for determining a presence of an object, especially an object such as animal or debris, in a path of a vehicle, are discussed herein. For example, sensors of various modalities, which may include multispectral sensors, may capture data representing an environment the vehicle is traversing. In examples, one or more trained machine learned (ML) models, operating on a vehicle computing system, may detect and/or classify objects in the environment, based on input data of one or more modalities or spectral bands. The ML models may be pre-trained using training data including real sensor data, synthetic data, and/or augmented data, along with auto-generated annotations. In some examples, hyperspectral data may be used to identify materials associated with detected objects. A confidence score associated with the detection of the object may also be computed. The vehicle may be controlled based on detection of the object and its classification.