Conversational Software Agent Speech Inactivity Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication systems, particularly in VoIP, face challenges in creating a lifelike conversation experience with software agents due to false recognition of speech disfluencies and background noise, leading to disjointed interactions.
Innovation Solution
The use of Automatic Speech Recognition (ASR) systems to identify complete sentences and a speech detection module that determines speech inactivity based on the ASR output, rather than sound level detection, to generate responses only after a pause, ensuring a more conversational and realistic interaction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sound level detection is used to determine speech activity, then the system can detect when a user is speaking, but it causes false recognition of speech disfluencies and background noise leading to disjointed interactions
Solution Approach 1:
The patent introduces ASR (Automatic Speech Recognition) as an intermediary component between the sound level detector and the response generator. The ASR system processes the voice input and provides word-level information that mediates the detection of speech inactivity, filtering out false positives from background noise and disfluencies while maintaining accurate speech activity detection.
Solution Approach 2:
The patent replaces the traditional mechanical sound level detection system with an ASR-based speech activity detection system. Instead of relying solely on acoustic energy thresholds, the system uses linguistic processing to determine whether speech is actively occurring, substituting a more intelligent recognition mechanism for the simpler physical measurement approach.
2Speed
If the system generates responses immediately upon detecting speech inactivity, then the response time is reduced, but it may interrupt the user if they are still speaking
Solution Approach 1:
The system performs preliminary speech activity verification using ASR before generating a response. When speech inactivity is detected, the system preemptively checks whether any new words are identified by the ASR system in a subsequent time interval, ensuring that the user has truly finished speaking before the response is generated, thus avoiding interruptions.
Solution Approach 2:
The system implements a feedback mechanism where the ASR system continuously monitors for new words after speech inactivity is detected. This feedback loop allows the system to verify speech completion status before proceeding with response generation, creating a more reliable and natural conversational flow.
Data Source
AI summary
Voice input is received from a user. An ASR system generates in memory a set of words it has identified in the voice input, and update the set each time it identifies a new word in the voice input to add the new word to the set. A condition indicative of speech inactivity in the voice input is detected. A response for outputting to the user is generated based on the set of identified words, in response to the detection of the speech inactivity condition. The generated response is outputted to the user after an interval of time—commencing with the detection of the speech inactivity condition—has ended and only if no more words have been identified in the voice input by the ASR system in that interval of time.