Autonomous Vehicle Audio Labeling With Onboard Sound Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional motion planning and control for autonomous driving vehicles lack accuracy due to the use of unrefined sensor data, particularly in obstacle identification and sound source classification, which are often manually labeled and prone to human error.
Innovation Solution
A method and system for onboard validation of audio data that records, labels, and refines audio samples from obstacles within the driving environment, incorporating visual sensor data to generate refined labeled audio data for training machine learning algorithms, improving the accuracy of sound source recognition during autonomous driving.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual data labelling is performed by humans, then the labelling process can be completed, but the accuracy of the labelled data deteriorates due to human error
Solution Approach 1:
The system enables autonomous self-labeling by having the vehicle perform sensing, logging, and validation operations independently during normal operation. The vehicle captures sensor data, automatically labels it through validation against ground truth, and stores it without requiring external human intervention, thus achieving both high throughput and high accuracy.
Solution Approach 2:
The system implements a feedback mechanism where sensor data is continuously validated against ground truth information. The validation process compares predicted outcomes with actual observations, and discrepancies are used to refine and improve the labeling accuracy over time, creating a self-correcting system that maintains high precision.
2Device complexity
If conventional motion planning uses standard curvature and speed estimation, then the planning process is simple, but the accuracy deteriorates because vehicle-specific features are not considered
Solution Approach 1:
The system applies local quality by tailoring motion planning parameters to specific vehicle characteristics. Instead of using uniform planning for all vehicles, the system adjusts curvature and speed estimates based on individual vehicle features such as dimensions, mass, and performance capabilities, thereby improving accuracy without requiring completely complex custom planning for each vehicle.
3Reliability
If audio data is collected during autonomous driving operation, then real-world validation data is obtained, but data processing complexity increases due to the need for refinement and validation
Solution Approach 1:
The system performs preliminary validation and filtering of audio data during the data collection phase itself. By validating data against ground truth information and filtering out invalid samples before storage, the system reduces the burden of post-processing while ensuring high data reliability. This preliminary action prevents accumulation of invalid data that would require extensive later processing.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Systems and methods for generating labelled audio data and onboard validation of the labelled audio data utilizing an autonomous driving vehicle (ADV) while the ADV is operating within a driving environment are disclosed. The method includes recording a sound emitted by an object within the driving environment of the ADV, and converting the recorded sound into audio samples. The method further includes labelling the audio samples, and refining the labelled audio samples to produce refined labelled audio data. The refined labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV. The method further includes generating a performance profile of the refined labelled audio data based at least on the audio samples, a position of the object, and a relative direction of the object. The position of the object and the relative direction of the object are determined by a perception system of the ADV.