Target Likelihood Fusion for Reflection-Robust Sound Localization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Electronic devices struggle to accurately distinguish between direct sound and acoustic reflections of desired speech due to strong signal reflections from walls and other acoustically reflective surfaces, especially when in motion, degrading sound source localization (SSL) processing.
Innovation Solution
Fusing SSL data with object information to generate a combined target likelihood estimate that accounts for the surrounding environment, using sensors to detect objects and calculate likelihood values, thereby associating acoustic events with direct sound sources and ignoring reflections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If SSL processing is performed in environments with strong acoustic reflections from walls and reflective surfaces, then the device can identify sound sources, but the accuracy and resolution of sound source localization degrades due to confusion between direct sound and reflections
Solution Approach 1:
The patent introduces an intermediary processing stage between raw SSL output and final sound source identification. This intermediary fuses SSL data with object information (floorplan data, sensor data) to create a combined target likelihood estimate. The intermediary acts as a mediator that reconciles the conflicting SSL measurements by incorporating environmental context, thereby resolving the ambiguity caused by acoustic reflections and improving localization accuracy.
2Adaptability or versatility
If the device moves through the environment to capture audio data, then it can access multiple locations and improve coverage, but the changing environment continuously degrades SSL processing accuracy
Solution Approach 1:
The patent implements dynamic adaptation by continuously updating the fusion of SSL data with object information as the device moves. The system dynamically adjusts target likelihood estimates based on real-time sensor data and floorplan information, allowing it to adapt to changing environmental conditions. This dynamic approach maintains measurement precision despite the device's movement through different locations with varying acoustic characteristics.
3Reliability
If the device uses only SSL processing to identify sound sources, then the system complexity remains low, but the device cannot distinguish between direct sound and acoustic reflections
Solution Approach 1:
The patent merges multiple data sources (SSL data, object information, floorplan data, sensor data) into a unified processing framework. By combining these diverse information streams, the system achieves reliable sound source identification that can distinguish direct sound from reflections. The merging of data sources creates a more robust system that overcomes the limitations of SSL processing alone, while the integrated approach manages complexity through unified target likelihood estimation.
Data Source
AI summary
A system configured to improve SSL processing and/or target goal detection by fusing SSL data with object information to generate a combined target likelihood estimate that takes into account what the device knows about the surrounding environment. For example, the device may generate object information by performing object detection, floorplan estimation, distance measurements, and/or the like. Using this object information, the device may calculate a likelihood estimate value for each direction around the device, with known objects (e.g., walls) corresponding to low likelihood values. In response to an acoustic event (e.g., wakeword detection), the device may fuse the target likelihood estimates generated using SSL data and/or object information to generate the combined target likelihood estimate. Thus, the combined target likelihood estimate enables the device to accurately associate the acoustic event with a corresponding SSL track (e.g., direct sound) and ignore reflections caused by objects in the environment.