Autonomous Vehicle Sound Detection via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles lack effective methods to detect and respond to critical auditory cues, such as railroad crossing bells and reversing truck beeps, which are essential for safe navigation, especially when visual obstructions are present.
Innovation Solution
The integration of microphones and a classifier system that identifies sound types and increases likelihood values based on associated sensor data, allowing the vehicle to respond autonomously to sounds by analyzing sensor data and map information, even when objects are occluded from other sensors like LIDAR and cameras.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If autonomous vehicles rely only on visual sensors (LIDAR, cameras) to detect objects, then the system complexity is reduced, but the ability to detect critical hazards (especially when visually occluded) deteriorates
Solution Approach 1:
The patent combines multiple sensor types (microphones for audio, LIDAR for visual, radar for electromagnetic detection) into an integrated perception system. This merging allows the vehicle to detect hazards through multiple modalities simultaneously, improving reliability when visual sensors are occluded while maintaining manageable system complexity through unified processing architecture.
Solution Approach 2:
The perception system is designed to perform multiple functions using a single integrated framework: detecting objects visually, audibly, and through electromagnetic signals. The system can identify trains, pedestrians, vehicles, and environmental hazards through any or all sensor types, making the detection system universally applicable across diverse hazard scenarios.
2Speed
If the vehicle responds immediately to all detected sounds, then the response time is improved, but the false alarm rate increases
Solution Approach 1:
The system implements feedback loops where detected sounds trigger cross-verification with other sensor data. When a sound is detected, the system checks for corresponding visual or radar detections, and adjusts the likelihood value based on corroboration. This feedback mechanism enables rapid response to genuine hazards while filtering out false alarms through continuous verification.
Solution Approach 2:
The system performs preliminary cross-verification of sound detections with other sensor data before triggering a full response. By pre-checking for corroboration and calculating likelihood values in advance, the system prepares rapid responses only for high-confidence detections, maintaining speed while reducing false alarms.
3Reliability
If the system processes all sensor data continuously to identify additional signals, then the detection completeness is improved, but the computational load increases
Solution Approach 1:
The system processes sensor data selectively rather than continuously analyzing all data streams. When a sound detection occurs, the system processes additional sensor data to a degree sufficient for verification (partial action), rather than performing exhaustive analysis. This approach ensures adequate detection completeness while managing computational energy consumption.
Data Source
AI summary
The technology relates to detecting and responding to sounds for a vehicle having an autonomous driving mode. In one example, an audible signal corresponding to a sound received at one or more microphones of the vehicle may be received. Sensor data generated by a perception system of the vehicle identifying objects in an environment of the vehicle may be received. A type of sound may be determined by inputting the audible signal into a classifier. A set of additional signals may be determined based on the determined type of sound. The sensor data may be processed in order to identify one or more additional signals of the identified set of additional signals. The vehicle may be controlled in the autonomous driving mode in order to respond to the sound based on the one or more additional signals and the type of sound.