Speech Endpoint Detection Using Silent Phonemes and WFST States
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current automatic speech recognition (ASR) systems face challenges in accurately determining the endpoint of speech due to imprecision in traditional pause detection methods, leading to incorrect cessation of speech processing, and require significant bandwidth and computing resources for remote processing, raising privacy concerns and inefficiencies.
Innovation Solution
A hybrid approach using a WFST accelerator with hardware and software components on a local device for decoding speech, incorporating a WFST language model and a personalized grammar model to improve accuracy and reduce remote processing needs, along with an endpointing module to accurately detect speech endpoints by counting silent frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional pause detection methods are used to determine speech endpoint, then the system is simple to implement, but the endpoint detection precision deteriorates leading to incorrect cessation of speech processing
Solution Approach 1:
The speech processing system is segmented into local and remote components. The local device performs preliminary speech decoding and endpoint detection using a WFST accelerator and silent frame counting, while the remote server handles final recognition. This segmentation allows precise endpoint detection locally without requiring complex remote processing for every frame.
Solution Approach 2:
The system performs preliminary speech decoding and endpoint detection locally before transmitting to the remote server. By counting silent frames and identifying speech endpoints in advance, the system prepares data locally, reducing the need for complex continuous remote monitoring and enabling more precise endpoint detection.
2Reliability
If remote processing is used for speech recognition, then the system can leverage powerful computing resources, but bandwidth consumption and computing resource usage increase
Solution Approach 1:
The system extracts and processes only the essential speech data locally using a WFST accelerator for decoding and silent frame counting for endpoint detection. By taking out and handling the preliminary processing locally, the system reduces the amount of data that needs to be transmitted and processed remotely, thereby reducing bandwidth consumption and remote computing resource usage while maintaining recognition accuracy.
Solution Approach 2:
The local device performs self-service speech decoding using the WFST accelerator and conducts its own endpoint detection by counting silent frames. This self-service capability reduces dependency on remote processing for every speech frame, thereby reducing bandwidth consumption and remote computing resource usage while maintaining speech recognition accuracy through local preprocessing.
3Reliability
If all speech data is transmitted to remote server for processing, then the system can achieve high recognition accuracy, but privacy concerns increase and processing efficiency decreases
Solution Approach 1:
The speech processing workflow is segmented into local decoding and endpoint detection, followed by selective remote recognition. By dividing the processing tasks, the system maintains high recognition accuracy through remote server capabilities while improving processing efficiency by handling routine tasks locally and reducing data transmission requirements.
Solution Approach 2:
The system performs preliminary speech decoding and endpoint detection locally before transmitting to the remote server. This preliminary action reduces the amount of data requiring remote processing, thereby improving processing efficiency while maintaining recognition accuracy through the combined local-remote approach.
Data Source
AI summary
Embodiments describe a method for speech endpoint detection including receiving identification data for a first state associated with a first frame of speech data from a WFST language model, determining that the first frame of the speech data includes silence data, incrementing a silence counter associated with the first state, copying a value of the silence counter of the first state to a corresponding silence counter field in a second state associated with the first state in an active state list, and determining that the value of the silence counter for the first state is above a silence threshold. The method further includes, determining that an endpoint of the speech has occurred in response to determining that the silence counter is above the silence threshold, and outputting text data representing a plurality of words determined from the speech data that was received prior to the endpoint.