Autonomous Vehicle Sound Classification and Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles lack sufficient environmental information for effective route planning and operation, particularly in dynamic environments, despite using LiDAR, RADAR, and cameras, highlighting the need for additional sensory inputs like sound classification and localization to enhance decision-making.
Innovation Solution
An autonomous vehicle system that captures ambient sound using microphones, classifies sounds through frequency analysis and machine learning, and localizes sound sources to determine actions, such as avoiding vehicles or pedestrians, by integrating sound data into its planning and control modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If LiDAR, RADAR and cameras are used for environmental sensing, then high-resolution image data is obtained, but sufficient environmental information for effective route planning and operation is still lacking
Solution Approach 1:
The autonomous vehicle system integrates multiple sensing modalities (visual, acoustic, radar) into a unified environmental perception framework. The sound classification module and sound source localization module work alongside existing LiDAR, RADAR and camera systems to provide complementary information about the environment, particularly detecting objects and events that may be invisible to optical sensors alone.
2Reliability
If sound classification and sound source localization are added to enhance environmental perception, then decision-making capability is improved, but system complexity increases
Solution Approach 1:
The acoustic processing system is divided into distinct functional modules: a sound classification module that identifies types of sounds (e.g., sirens, horns, pedestrian speech), and a sound source localization module that determines the spatial location of sound sources. This segmentation allows each module to specialize in specific tasks and process acoustic data independently before integrating results with other sensor inputs.
Solution Approach 2:
The patent introduces an acoustic environment perception module as an intermediary between raw microphone inputs and the autonomous vehicle's decision-making system. This intermediate layer processes and interprets acoustic signals, converting them into meaningful environmental information that can be effectively utilized by the planning and control algorithms.
Data Source
AI summary
An ambient sound environment is captured by a microphone array of an autonomous vehicle traveling in the ambient sound environment. A perception module of the autonomous vehicle classifies sounds and localizes sound sources in the ambient sound environment. Classification is performed using spectrum analysis and/or machine learning. In an embodiment, sound sources within a field of view (FOV) of an image sensor of the autonomous vehicle are localized in a visual scene generated by the perception module. In an embodiment, one or more sound sources outside the FOV of the image sensors are localized in a static digital map. Localization is performed using parametric or non-parametric techniques and/or machine learning. The output of the perception module is input into a planning module of the autonomous vehicle to plan a route or trajectory for the autonomous vehicle in the ambient sound environment.