Speech Endpointing Using Weighted Non-Speech Probability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face inefficiencies in bandwidth usage, privacy concerns, and resource wastage due to continuous audio processing, especially when no commands are being issued, and traditional endpointing techniques are imprecise, leading to errors in determining the end of a spoken utterance.
Innovation Solution
A distributed speech processing system that uses a local device to detect a wakeword to activate processing, and employs a method to determine the endpoint of an utterance by considering hypotheses based on pause lengths and their weighted probabilities, comparing them to thresholds to accurately estimate when a spoken command has concluded.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If continuous audio processing is performed, then speech recognition accuracy is maintained, but bandwidth usage increases and privacy concerns arise
Solution Approach 1:
The system performs preliminary endpoint detection locally on the device to identify when speech utterances begin and end. By detecting endpoints beforehand, the system can selectively transmit only relevant audio segments containing actual speech, rather than continuously streaming all audio data. This preliminary action at the endpoint detection stage enables subsequent bandwidth optimization while maintaining recognition accuracy.
2Device complexity
If traditional endpointing techniques are used, then system complexity is reduced, but accuracy in determining utterance end deteriorates
Solution Approach 1:
The system introduces an intermediary endpoint detection module that bridges simple threshold-based detection and complex full speech recognition. This intermediary component uses prosodic features (pitch contours, energy variations, pause detection) as intermediate signals to accurately identify utterance boundaries. By processing these intermediate prosodic indicators, the system achieves high endpoint detection accuracy without requiring full speech recognition processing, thus maintaining reasonable system complexity.
3Reliability
If audio data is transmitted continuously, then processing completeness is ensured, but resource wastage increases
Solution Approach 1:
The system extracts and transmits only the essential audio segments containing actual speech utterances, separating useful speech data from unnecessary silent or non-speech audio. By extracting only the relevant portions between detected endpoints and transmitting selectively, the system ensures complete processing of all speech content while dramatically reducing resource wastage associated with transmitting and processing irrelevant audio data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An automatic speech recognition (ASR) system detects an endpoint of an utterance using the active hypotheses under consideration by a decoder. The ASR system calculates the amount of non-speech detected by a plurality of hypotheses and weights the non-speech duration by the probability of each hypotheses. When the aggregate weighted non-speech exceeds a threshold, an endpoint may be declared.