Automated Assistant Action Ordering for Low-Latency Multi-Step Requests
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants often prioritize actions based on user-specified orders, leading to inefficiencies and computational delays when multiple actions are requested, particularly when dialogue-initiating queries require additional user input or resources.
Innovation Solution
Automated assistants determine an optimal execution order for multiple actions based on predicted interactions and resource requirements, prioritizing dialogue-initiating queries last and adjusting based on historical data and machine learning models to minimize latency and resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If the automated assistant executes actions in the order specified by the user, then the ease of operation is improved, but the productivity deteriorates due to unnecessary delays and computational inefficiencies
Solution Approach 1:
The patent applies inversion by automatically reordering actions in reverse of the user-specified sequence when optimization is beneficial. The system analyzes execution characteristics and reverses the conventional user-specified order to achieve faster overall completion, thereby resolving the contradiction between ease of operation (maintaining user control) and productivity (reducing delays).
Solution Approach 2:
The system dynamically adjusts the execution order based on real-time analysis of action characteristics, resource availability, and interdependencies. Rather than rigidly following user-specified order or using fixed rules, the system adapts the execution sequence dynamically to optimize productivity while still respecting user intent when appropriate.
2Reliability
If the automated assistant prioritizes dialogue-initiating queries, then the reliability is improved, but the loss of time increases due to delaying other actions
Solution Approach 1:
The system performs preliminary analysis of all requested actions before execution, identifying which actions are dialogue-initiating and which can be executed independently. Actions that do not require user dialogue are executed in advance while awaiting user response for dialogue-initiating actions, thereby reducing overall time loss while maintaining reliability.
Solution Approach 2:
The patent segments actions into different categories based on their execution characteristics: dialogue-initiating actions, independent actions, and dependent actions. This segmentation allows the system to handle different action types differently, executing independent actions concurrently or in advance while prioritizing dialogue-initiating actions appropriately, thus balancing reliability and time efficiency.
3Device complexity
If the automated assistant executes actions sequentially in user-specified order, then the device complexity is reduced, but the productivity deteriorates due to inability to optimize execution order
Solution Approach 1:
The system applies self-service by automatically analyzing and optimizing its own action execution order without requiring complex external control mechanisms. The automated assistant independently evaluates execution characteristics, resource requirements, and interdependencies to determine the optimal execution sequence, thereby improving productivity while avoiding the need for significantly increased system complexity.
Solution Approach 2:
The patent changes the execution parameters dynamically by adjusting the order parameter based on analyzed characteristics of each action. Rather than maintaining a fixed execution order or using complex scheduling algorithms, the system modifies the execution sequence parameter adaptively based on real-time conditions, achieving productivity improvement without proportionally increasing device complexity.
4Reliability
If the automated assistant delays actions requiring user input, then the reliability is improved, but the loss of time increases for actions that could be performed quickly
Solution Approach 1:
The system performs preliminary identification and classification of actions based on their execution characteristics. Actions that can be executed quickly and independently are identified and scheduled for early execution, while actions requiring user input are flagged for later execution after user response is received. This preliminary action approach reduces overall time loss while maintaining reliability.
Solution Approach 2:
The patent maintains continuity of useful action by executing independent actions continuously while awaiting user input for dialogue-initiating actions. Rather than pausing all actions when user input is required, the system continues executing actions that do not depend on user response, thereby minimizing time loss while ensuring reliability through proper sequencing.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Implementations are set forth herein for creating an order of execution for actions that were requested by a user, via a spoken utterance to an automated assistant. The order of execution for the requested actions can be based on how each requested action can, or is predicted to, affect other requested actions. In some implementations, an order of execution for a series of actions can be determined based on an output of a machine learning model, such as a model that has been trained according to supervised learning. A particular order of execution can be selected to mitigate waste of processing, memory, and network resources-at least relative to other possible orders of execution. Using interaction data that characterizes past performances of automated assistants, certain orders of execution can be adapted over time, thereby allowing the automated assistant to learn from past interactions with one or more users.