Digital Assistant Latency Reduction via Preliminary Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Digital assistants on mobile platforms face high latency due to limited computing resources, leading to poor user experience and reduced adoption, as they require longer processing times for natural language processing, task flow processing, and speech synthesis, which are initiated only after detecting a speech end-point condition.
Innovation Solution
Initiating natural language processing, task flow processing, and speech synthesis during the detection of a speech end-point condition, such as a short pause, allowing partial completion of these processes before the condition is confirmed, and evaluating multiple candidate task flows based on scores to enhance accuracy and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If digital assistants are implemented on mobile computing platforms with limited computing resources, then the device can operate portably, but processing times increase and latency increases
Solution Approach 1:
The system initiates natural language processing, task flow processing, and speech synthesis operations during the speech end-point detection period (between the second time and third time) before the user releases the button. This preliminary action allows computation to begin earlier, reducing overall latency while maintaining portability on mobile devices with limited resources.
2Reliability
If processing operations are initiated only after speech end-point condition is detected, then processing accuracy is maintained, but overall latency increases
Solution Approach 1:
The system performs preliminary processing operations during the detection window before speech end-point is confirmed. Multiple candidate task flows are evaluated with scores, and the highest-scoring candidate is selected in advance. This approach reduces latency by starting computation earlier while maintaining reliability through candidate evaluation and scoring mechanisms.
Solution Approach 2:
The system performs partial processing operations during the speech end-point detection period, completing a subset of necessary computations before the final end-point confirmation. This partial action reduces overall latency while the remaining operations complete after confirmation, balancing speed and accuracy.
3Reliability
If multiple candidate task flows are evaluated with scores, then accuracy and reliability improve, but computational complexity increases
Solution Approach 1:
The system evaluates multiple candidate task flows and assigns scores during the speech end-point detection period. By performing this evaluation preliminarily, the system reduces the computational burden at the final decision point while improving selection accuracy through comprehensive candidate assessment.
Data Source
AI summary
Systems and processes for operating a digital assistant are provided. In an example process, low-latency operation of a digital assistant is provided. In this example, natural language processing, task flow processing, dialogue flow processing, speech synthesis, or any combination thereof can be at least partially performed while awaiting detection of a speech end-point condition. Upon detection of a speech end-point condition, results obtained from performing the operations can be presented to the user. In another example, robust operation of a digital assistant is provided. In this example, task flow processing by the digital assistant can include selecting a candidate task flow from a plurality of candidate task flows based on determined task flow scores. The task flow scores can be based on speech recognition confidence scores, intent confidence scores, flow parameter scores, or any combination thereof. The selected candidate task flow is executed and corresponding results presented to the user.