ASR Endpoint Detection via Relative Likelihood Score Trends
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing human-machine interfaces with language understanding systems face challenges in reducing latency during speech recognition due to inefficient speech endpoint detection, particularly in noisy environments, leading to increased system latency and error rates.
Innovation Solution
The implementation of a methodology that tracks the probabilities of ASR hypotheses over time using relative likelihood score trends, allowing for earlier detection of speech endpoints without increasing error rates, by calculating a trend of the relative likelihood score as additional speech is collected and processed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If voice activity detection based on signal energy is used for speech endpoint detection, then the system can detect speech endpoints in quiet environments, but it fails in noisy environments where background noise is misrecognized as voice signal energy
Solution Approach 1:
The patent introduces an intermediary mechanism (score trend analysis of ASR hypotheses) between the raw audio signal and the endpoint detection decision. Instead of directly detecting endpoints from audio energy, the system uses ASR hypothesis scores as an intermediate indicator to infer speech continuity, thereby filtering out noise interference that would otherwise directly corrupt the endpoint detection.
Solution Approach 2:
The patent replaces the traditional acoustic-based endpoint detection mechanism (voice activity detection using signal energy) with an information-processing-based mechanism (analyzing trends in ASR hypothesis scores). This substitution moves the detection from the acoustic domain to the computational domain, where noise has less direct impact.
2Reliability
If a predefined period of time is used for speech endpoint detection, then the system can ensure complete phrase recognition, but it increases system latency
Solution Approach 1:
The patent makes the endpoint detection dynamic by continuously monitoring the trend of ASR hypothesis scores over time. Instead of using a fixed time window, the system adapts the detection timing based on the observed score trends, allowing earlier detection when confidence is high and longer observation when uncertainty exists, thus optimizing the trade-off between completeness and latency.
Solution Approach 2:
The system uses feedback from the ASR hypothesis scoring process to guide endpoint detection. The trends in hypothesis scores provide continuous feedback about the likelihood of speech continuation, enabling the system to make informed decisions about when to declare an endpoint without relying on fixed time parameters.
3Loss of time
If speech endpoint detection is performed early to reduce latency, then system response time improves, but error rates increase due to insufficient speech data
Solution Approach 1:
The patent performs preliminary analysis of ASR hypothesis score trends during the speech processing window before making the final endpoint decision. By evaluating the trajectory of confidence scores in advance, the system can determine whether sufficient information has been gathered to make a reliable detection, allowing early endpoint declaration only when the trend indicates high confidence.
Solution Approach 2:
The system changes the parameter being monitored from acoustic signal properties to ASR hypothesis confidence scores. This parameter change enables detection based on recognition confidence rather than raw signal characteristics, allowing the system to declare endpoints earlier when confidence is high without sacrificing accuracy.
Data Source
AI summary
Techniques are provided for reducing the latency of automatic speech recognition using hypothesis score trend analysis. A methodology implementing the techniques according to an embodiment includes generating complete-phrase hypotheses and partial-phrase hypotheses, along with associated likelihood scores, based on a segment of speech. The method also includes selecting the complete-phrase hypothesis associated with the highest of the complete-phrase hypotheses likelihood scores, and selecting the partial-phrase hypothesis associated with the highest of the partial-phrase hypotheses likelihood scores. The method further includes calculating a relative likelihood score based on a ratio of the likelihood score associated with the selected complete-phrase hypothesis to the likelihood score associated with the selected partial-phrase hypothesis. The method further includes calculating a trend of the relative likelihood score as a function of time and identifying an endpoint of the speech based on a determination that the trend does not decrease over a selected time period.