Digital Assistant Latency Reduction via Preliminary Task Flow Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital assistants on mobile platforms face significant latency issues due to limited computing resources, leading to poor user experience and reduced adoption, as they require longer processing times for natural language processing, task flow processing, and speech synthesis, which are initiated only after detecting a speech end-point condition.
Innovation Solution
Implementing techniques that allow partial execution of operations such as natural language processing and task flow processing during the time needed to detect a speech end-point condition, by determining candidate user intents and task flows and executing them without immediate output, and generating text dialogues that can be stored for later output once the end-point is confirmed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If digital assistants wait to detect speech end-point condition before processing, then processing accuracy is improved, but response latency increases
Solution Approach 1:
The system performs preliminary natural language processing and task flow processing on the first portion of the audio stream before the speech endpoint is detected. Candidate user intents and task flows are identified and executed in advance, so that when the endpoint is confirmed, the results are already prepared and can be immediately presented to the user, eliminating the traditional waiting period.
Solution Approach 2:
The audio stream is divided into multiple portions: a first portion processed preliminarily before endpoint detection, and subsequent portions processed after confirmation. This segmentation allows the system to work on different parts of the processing pipeline at different times, reducing overall latency while maintaining accuracy.
2Speed
If digital assistants process audio in real-time on mobile platforms, then response speed is improved, but computing resource consumption increases
Solution Approach 1:
The system performs preliminary processing of the first audio portion using available computing resources, then waits passively during the endpoint detection period. This approach utilizes otherwise idle time for computation, achieving faster response without requiring sustained high processing power that would drain mobile device resources.
Solution Approach 2:
The system performs partial processing on the first audio portion to generate candidate intents and task flows, rather than waiting to process the complete audio stream. This partial action is sufficient to prepare results in advance, and the system can refine or adjust based on the confirmed endpoint without redoing all the work.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
In an example process, low-latency operation of a digital assistant is provided. In this example, natural language processing, task flow processing, dialogue flow processing, speech synthesis, or any combination thereof can be at least partially performed while awaiting detection of a speech end-point condition. Upon detection of a speech end-point condition, results obtained from performing the operations can be presented to the user. In another example, robust operation of a digital assistant is provided. In this example, task flow processing by the digital assistant can include selecting a candidate task flow from a plurality of candidate task flows based on determined task flow scores. The task flow scores can be based on speech recognition confidence scores, intent confidence scores, flow parameter scores, or any combination thereof. The selected candidate task flow is executed and corresponding results presented to the user.