Mobile Robot Audio Perception for Navigation Beyond Visual Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile robots, such as vacuum robots, face challenges in navigating around objects, especially in scenarios with insufficient lighting, camera malfunctions, or object occlusions.
Innovation Solution
The implementation of a computer-implemented method that utilizes a microphone array on the mobile robot to receive audio signals, extract audio feature data, generate Direction-of-Arrival (DOA) data, and use machine learning models to generate audio event data. This data is then used to query a knowledge graph, generating semantic audio scene data that informs the robot's actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If visual sensors (camera) are used for navigation, then the robot can detect objects in well-lit environments, but the robot fails to navigate when there is insufficient lighting, camera malfunction, or object occlusion
Solution Approach 1:
The patent implements multi-functional sensing by integrating both visual sensors (camera) and audio sensors (microphone array) in the mobile robot. The system can switch between visual navigation in well-lit conditions and audio-based navigation in challenging environments, making the robot universally adaptable to different environmental conditions. The audio perception system serves as a complementary function that activates when visual sensors fail.
Solution Approach 2:
The audio perception system acts as an intermediary sensing modality between the robot and objects in environments where visual sensing fails. By capturing acoustic reflections and sound signatures of objects, the system mediates the navigation process when direct visual detection is unavailable, allowing the robot to maintain navigation capability through alternative acoustic information channels.
2Adaptability or versatility
If the robot uses audio signals for navigation, then it can operate in challenging environments, but the system complexity increases with microphone array and audio processing
Solution Approach 1:
The patent segments the navigation system into distinct functional modules: visual sensing module, audio sensing module (microphone array), audio processing module, and navigation control module. This segmentation allows the audio-based navigation to be implemented as a separate, interchangeable subsystem that can be activated only when visual sensing fails, thereby managing overall system complexity through modular architecture.
Solution Approach 2:
The system implements partial action by activating the audio perception system only when visual sensing fails, rather than continuously operating both systems. The microphone array and audio processing are engaged partially - only in challenging environments - reducing the effective complexity and resource consumption while maintaining adaptability when needed.
3Measurement precision
If the robot processes audio signals through multiple stages (feature extraction, DOA estimation, event detection), then accurate semantic audio scene data is generated, but the processing time and computational load increase
Solution Approach 1:
The patent implements preliminary action by pre-training machine learning models for audio event detection and classification before deployment. The system pre-processes and stores acoustic signatures of various objects and events during the training phase, so that during actual navigation, the robot can quickly match incoming audio features against pre-established patterns, reducing real-time processing time while maintaining high recognition accuracy.
Data Source
AI summary
A mobile robot includes a microphone array with a set of microphones. The microphone array is at least partially disposed on the mobile robot. The mobile robot receives audio signals from the microphone array. Audio feature data of acoustic activity is extracted from the audio signals. Direction of arrival (DOA) data of the acoustic activity is generated based on the audio signals. A machine learning model is configured to generate audio event data using the audio feature data. The audio event data identifies at least one sound source of the audio feature data. A knowledge graph is queried using the audio event data to obtain entity data. The entity data has a predetermined relation with the audio event data. Semantic audio scene data is generated using the audio event data, the DOA data, and the entity data. The mobile robot performs an action based on the semantic audio scene data.