Speech Recognition Endpoint Detection with Acoustic-Semantic Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current speech recognition systems face challenges in accurately determining the speech ending state due to delayed or premature detection, often caused by background noise or pauses, leading to inaccurate responses and user frustration.
Innovation Solution
A method that combines sound category and semantics extraction from audio frames to accurately determine the speech ending point, using energy thresholds and a speech endpoint classification model to adapt to different environments and improve processing efficiency and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automatic speech recognition is used to determine speech ending state, then speech-based operations can be automated, but speech ending detection accuracy deteriorates due to background noise and pauses
Solution Approach 1:
The patent combines multiple detection mechanisms (energy threshold detection, zero-crossing rate detection, and semantic analysis) into a unified speech ending detection system. By merging these approaches, the system achieves both automation and improved accuracy, as each mechanism complements the others in handling different speech scenarios such as background noise and pauses.
Solution Approach 2:
The system implements feedback by continuously monitoring audio features (energy, zero-crossing rate) and comparing them against thresholds during real-time speech recognition. When the speech ending conditions are met, the system provides feedback to stop audio collection and triggers the corresponding operation, ensuring accurate timing while maintaining automation.
2Duration of action of moving object
If fixed waiting time is used to determine speech ending, then response timing can be controlled, but timeliness and accuracy of response deteriorate
Solution Approach 1:
The patent replaces fixed waiting time with dynamic detection based on real-time audio feature analysis. The speech ending point is determined dynamically by monitoring energy thresholds and zero-crossing rates, allowing the system to respond immediately when speech naturally ends, thereby reducing unnecessary waiting time while maintaining precise response timing control.
Solution Approach 2:
The system performs preliminary detection of speech ending conditions by continuously analyzing audio features before finalizing the speech ending point determination. This preliminary action allows the system to prepare for immediate response once the speech ending is confirmed, reducing overall waiting time while ensuring accuracy.
3Loss of information
If semantic analysis is performed on voice data, then understanding of voice instructions improves, but processing time increases
Solution Approach 1:
The patent performs semantic analysis preliminarily during the audio collection phase rather than after speech ending detection. By analyzing the semantics of voice instructions in advance and comparing them with detected speech ending points, the system achieves both accurate understanding and optimized processing time, as the analysis is conducted parallel to audio collection rather than sequentially.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
This application provides a speech recognition method, a speech recognition apparatus, and a system, and relates to the artificial intelligence field. The method includes: obtaining audio data, where the audio data includes a plurality of audio frames; extracting sound categories of the plurality of audio frames and semantics; and obtaining a speech ending point of the audio data based on the sound categories and the semantics. In the solution, the speech ending point of the audio data is obtained by extracting and combining the sound category and the semantics in the audio data, so that the speech ending point of the audio data can be determined more accurately, and a speech-based subsequent operation is responded to more accurately, thereby improving user experience.