Sound Source Localization via Deep Neural Network and Computer Vision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current sound source localization (SSL) techniques in electronic devices are computationally expensive and difficult to tune, especially under varying acoustic conditions, due to the need for pre-processing and post-processing operations like noise reduction and voice activity detection.
Innovation Solution
The implementation of deep learning techniques using well-labelled datasets to develop deep neural network (DNN) approaches for SSL, which learn pre-processing and post-processing operations implicitly, and a probabilistic framework that incorporates computer vision data to improve accuracy and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional SSL algorithms with pre-processing and post-processing operations are used, then SSL functionality is achieved, but computation cost increases and tuning difficulty increases
Solution Approach 1:
The patent extracts and removes the complex pre-processing and post-processing operations from the SSL system. By using a neural network to directly process raw audio data and output sound source directions, the system eliminates the need for separate noise reduction, voice activity detection, and other intermediate processing steps, thereby reducing computation cost while maintaining SSL accuracy
Solution Approach 2:
The neural network is designed to perform multiple functions simultaneously: it processes raw audio data, performs noise reduction, detects voice activity, and determines sound source directions all in a single unified model. This multi-functional approach replaces multiple separate traditional algorithms, reducing both computation cost and system complexity
2Reliability
If traditional SSL algorithms with pre-processing and post-processing operations are used, then SSL functionality is achieved, but tuning difficulty increases
Solution Approach 1:
The neural network is trained offline on labeled datasets to automatically learn optimal parameters and processing strategies for various acoustic conditions. During deployment, the system uses the pre-trained model without requiring manual tuning of noise reduction parameters, voice activity detection thresholds, or other processing parameters, making the system easy to operate across different environments
Solution Approach 2:
The system handles varying acoustic conditions by using the neural network's learned parameters that were trained on diverse datasets representing different noise levels and acoustic environments. The network automatically adapts to different conditions through its learned representations without requiring manual parameter adjustments, thereby reducing tuning difficulty while maintaining reliability
3Productivity
If deep learning techniques are used for SSL, then processing resources are reduced, but implementation complexity increases
Solution Approach 1:
The patent uses a neural network model that can be deployed as a standalone processing component. The model is trained offline using computational resources, then the trained weights are deployed to the electronic device where they require minimal processing power during inference. This approach trades offline training complexity for efficient online operation, improving processing efficiency while keeping implementation complexity manageable through standardized deep learning frameworks
Data Source
AI summary
Techniques for improving sound source localization (SSL) are provided. A method for probabilistic SSL using a deep neural network (DNN) may include receiving audio data including a representation of audio such as a wakeword from a microphone array. The audio data may be processed by a DNN to output a plurality of values where each value indicates a probability that the audio originated from a direction corresponding to that value. A sensor may provide computer vision or other data which may be used to inform the plurality of values based on detecting presence of a human or obstacle. A probability that the audio originated from one of the directions of the plurality of directions may be determined based at least in part on the DNN output and the computer vision or other data.