Intended Query Detection in End-to-End Continued Conversation ASR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing speech-enabled environments require users to repeatedly use hotwords to interact with digital assistants, disrupting the flow of conversation and imposing a cognitive burden.
Innovation Solution
Integrate an automatic speech recognition system with an intended query detection model that utilizes an end-to-end recurrent neural network-transducer (RNN-T) model to detect intended queries within spoken utterances, allowing for continuous conversation without the need for repeated hotword activation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional topic models or classifiers are used for intent detection, then the system can handle general queries, but it fails to accurately detect intents in continued conversations where pronouns and ellipses are used
Solution Approach 1:
The patent applies preliminary action by generating and storing reference expressions for each intent category before actual intent detection occurs. During continued conversations, the system retrieves these pre-generated references and compares them against the current input, enabling accurate intent detection even when users employ pronouns or ellipses. This pre-computation of reference expressions allows the system to handle the adaptability challenge while maintaining high detection accuracy.
2Measurement precision
If the system generates comprehensive reference expressions for all possible intents, then intent detection accuracy improves, but the complexity of the system increases
Solution Approach 1:
The patent segments the intent detection process into distinct modules: a reference generation module that creates template expressions for each intent category, a retrieval module that fetches relevant references based on conversation context, and a comparison module that matches input against references. This segmentation allows the system to manage complexity by organizing functions into separate, manageable components rather than implementing a monolithic complex system.
Solution Approach 2:
The system changes parameters dynamically by adjusting which reference expressions are generated and retrieved based on the specific conversation context and intent category. Rather than maintaining all possible reference expressions simultaneously in a fixed structure, the system flexibly generates and retrieves references as needed, changing the active parameter set based on contextual requirements to manage complexity while maintaining accuracy.
3Measurement precision
If the system uses detailed context analysis to improve intent detection in continued conversations, then accuracy improves, but processing time increases
Solution Approach 1:
The system performs preliminary action by pre-generating reference expressions that encapsulate typical user inputs for each intent category. During actual intent detection in continued conversations, the system retrieves these pre-computed references and performs quick string matching or similarity comparison, avoiding the need for time-consuming detailed context analysis while maintaining high accuracy. The heavy lifting is done in advance when references are generated.
Data Source
Figure 1A
Figure 1B
Figure 2
AI summary
A method (500) includes receiving, as input to a speech recognition model (200), audio data (110) corresponding to a spoken utterance (106). The method also includes performing speech recognition on the audio data by, at each of a plurality of time steps, encoding, using an audio encoder (220), the audio data corresponding to the spoken utterance into a corresponding audio encoding (224), and decoding, using a speech recognition joint network (240), the corresponding audio encoding into a probability distribution (242) over possible output labels. At each of the plurality of time steps, the method also includes determining, using an intended query (IQ) joint network (230) configured to receive a label history representation (350) associated with a sequence of non-blank symbols (252) output by a final softmax layer (250), an intended query decision (212) indicating whether or not the spoken utterance includes a query intended for a digital assistant (105).