Telepresence Robot Voice Control With Operator-Only Command Routing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing robotic telepresence systems struggle with differentiating between operators and observers during multi-party interactions, leading to incorrect command execution and energy inefficiency due to continuous speech recognition by all participants.
Innovation Solution
A robotic telepresence system with a dedicated secured bidirectional WebSocket P2P connection between an ASR cloud and an operator client, enabling speech signal recognition and command validation, allowing only the operator's commands to be executed by the tele-robot, while reducing energy consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ASR is performed continuously by the robot to recognize speech commands, then command recognition capability is improved, but energy consumption increases
Solution Approach 1:
The system performs preliminary speech activity detection before initiating full ASR processing. The speech activity detector continuously monitors audio input and only triggers the ASR component when speech is detected, avoiding continuous full ASR processing and reducing energy consumption while maintaining reliable command recognition.
Solution Approach 2:
The system uses periodic speech activity detection at predetermined time intervals to determine whether to activate ASR processing. This periodic checking mechanism ensures that the resource-intensive ASR component is only activated when necessary, balancing reliable command recognition with energy efficiency.
2Measurement precision
If the robot performs ASR on all speech signals in multi-user sessions, then speech recognition accuracy is improved, but command differentiation capability deteriorates
Solution Approach 1:
The system segments the multi-user audio environment by detecting speech activity and isolating speech signals from non-speech participants. By dividing the audio processing into speech and non-speech segments, the system maintains high speech recognition accuracy for commands while preventing misattribution of observer chatter to the operator.
Solution Approach 2:
The speech activity detector acts as an intermediary between the audio input and ASR component. It filters and prepares speech signals before they reach the ASR system, ensuring that only actual speech from the operator is processed, thereby maintaining both recognition accuracy and command differentiation capability.
3Ease of operation
If the robot executes commands from any participant in multi-user sessions, then ease of operation is improved, but command execution accuracy deteriorates
Solution Approach 1:
The system applies different processing qualities to different audio sources based on their role. Speech activity detection is applied specifically to identify operator commands, while observer audio is treated differently. This localized quality approach ensures that only operator speech triggers command execution, maintaining both accessibility and accuracy.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
Complexity of voice based control of tele-robots increases when used in collaborative multi-presence systems, as tele-robot fail to differentiate between operator and observers, and hence may end up executing command from wrong person. Further, the tele-robot ends up losing energy as it listens continuously to speech from multiple people, which may or may not contain commands. Embodiments disclosed herein provide a telepresence system in which an Automatic Speech Recognition (ASR) component hosted on a cloud receives and processes speech signals, infers commands, and only the inferred commands are transmitted to the tele-robot, thereby causing energy saving at the tele-robot end as it doesn't have to listen to all the speech from all the operator and observers for the execution of the voice based navigation control command. Further, a dedicated channel is established between the operator at any instance and the tele-robot, enabling direct communication between the operator and the tele-robot.