End-to-End Spoken Language Understanding Model Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing spoken language understanding systems face inefficiencies due to large memory and storage usage from combining automatic speech recognition (ASR) and natural language understanding (NLU) models, leading to cascading issues that degrade performance and increase model size, especially when utilizing dialog history in text form.
Innovation Solution
The integration of a system that encodes both speech-based and text-based content of an utterance into a uniform representation, utilizing a hierarchical model with a lower-level encoder for speech and an upper-level encoder for text, allowing for end-to-end integration of dialogue history in spoken form without the need for text-based context during performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If ASR and NLU models are combined to process dialog history in text form, then spoken language understanding capability is improved, but model size and memory usage increase significantly
Solution Approach 1:
The patent merges ASR and NLU models into a unified end-to-end model that processes both speech and text inputs through a single architecture. This combines the functionality of separate models while sharing computational resources, thereby improving spoken language understanding without proportionally increasing model size.
Solution Approach 2:
The unified model serves multiple functions: it performs both automatic speech recognition and natural language understanding, and can handle both speech-based and text-based inputs. This multi-functionality reduces the need for separate specialized models, decreasing overall system size while maintaining comprehensive capability.
2Reliability
If dialog history is integrated in text form, then context understanding is improved, but training time and computational resources increase
Solution Approach 1:
The patent changes the input parameter representation by accepting both speech-based and text-based inputs uniformly. This allows the model to learn from multiple modalities simultaneously during training, improving context understanding while optimizing training efficiency through shared processing pathways.
Solution Approach 2:
The model performs preliminary encoding of both speech and text inputs into uniform representations before processing. This preprocessing step prepares the data in an optimized format that reduces computational burden during main processing, thereby decreasing training time while maintaining comprehensive context understanding.
3Device complexity
If separate ASR and NLU models are used, then modularity is maintained, but system complexity and cascading issues increase
Solution Approach 1:
The patent combines separate ASR and NLU models into a single integrated system that processes speech and text through unified layers. This eliminates the cascading architecture where errors propagate between separate models, improving performance stability while managing complexity through a streamlined single-model structure.
Data Source
AI summary
Systems, computer-implemented methods, and computer program products to facilitate end to end integration of dialogue history for spoken language understanding are provided. According to an embodiment, a system can comprise a processor that executes components stored in memory. The computer executable components comprise a conversation component that encodes speech-based content of an utterance and text-based content of the utterance into a uniform representation.