Spoken Language Understanding Using Relevant Prosody Cues
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech recognition systems struggle to accurately interpret user intent due to limitations in utilizing prosodic data, leading to ambiguities in understanding spoken language.
Innovation Solution
Incorporating prosodic features such as intonational, rhythmic, and tonal properties at both syllable/word and sentence/utterance levels within the SLU system to disambiguate user intentions and resolve ambiguities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If speech recognition systems use basic spectral audio features only, then the system complexity remains low, but the intent recognition accuracy deteriorates due to ambiguities in spoken language
Solution Approach 1:
The patent transitions from using only spectral audio features to incorporating prosodic features (pitch, energy, duration) as an additional dimension of analysis. This multi-dimensional approach allows the system to disambiguate user intent by examining both what is said and how it is said, thereby improving intent recognition accuracy without excessive complexity increase
Solution Approach 2:
The patent segments the audio signal analysis into distinct prosodic feature extractions (pitch contour, energy contour, duration contour) that operate independently at different granularities (word-level and utterance-level). This segmentation allows the system to process prosodic information systematically and integrate it with spectral features to resolve ambiguities
2Reliability
If prosodic features are incorporated at multiple levels, then the disambiguation capability improves, but the processing complexity increases
Solution Approach 1:
The patent divides prosodic analysis into two distinct levels: word-level prosodic features (pitch, energy, duration for individual words) and utterance-level prosodic features (overall intonation patterns, pause structures). This segmentation enables the system to capture different aspects of prosody systematically, improving disambiguation capability while organizing processing complexity into manageable segments
Solution Approach 2:
The patent merges word-level and utterance-level prosodic features into a unified analysis framework that works together with spectral audio features. By combining these different levels of prosodic information, the system achieves enhanced disambiguation capability for determining user intent, resolving syntactic ambiguities, and identifying rhetorical questions
Data Source
AI summary
Techniques for determining and using relevant prosody information for spoken language understanding (SLU) processing are described. In some embodiments, a system determines local prosody data for individual audio frames of input audio data representing a spoken input. The system also determines global prosody data based on the entire spoken input. A portion of the local prosody data is determined to be relevant for a respective audio frame. A portion of the global prosody data is determined to relevant for the spoken input. The relevant portions are used to determine at least an intent corresponding to the spoken input. Audio features corresponding to the input audio data may be used to determine relevant portions of the prosody data.