Automated Assistant Action Initialization Across Active Apps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated assistants often have limited availability and functionality, leading to inefficiencies and waste of computational resources when users attempt to control other applications, as they may terminate dialog sessions inadvertently or require repeated spoken utterances to perform actions.
Innovation Solution
An automated assistant processes spoken utterances on-device to identify and prioritize actions for applications currently running, allowing simultaneous interaction with those applications without needing separate modules for voice control, thus conserving computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated assistant processes spoken utterances while application is running, then user interaction efficiency is improved, but computational resources are wasted when dialog session is terminated
Solution Approach 1:
The system dynamically adjusts the availability and functionality of the automated assistant based on the current application state. When an application is in the foreground, the assistant processes spoken utterances; when another application opens, the assistant automatically terminates the dialog session, adapting its operational state to match user context and avoid wasting computational resources on abandoned dialog sessions.
Solution Approach 2:
The system monitors user actions and application state changes as feedback signals. When a user opens a new application during an ongoing dialog session, the system detects this context change and automatically terminates the assistant dialog, using the application state as feedback to determine when to stop processing spoken utterances and conserve computational resources.
2Loss of energy
If automated assistant has limited availability when user operates other applications, then computational resources are conserved, but user functionality is reduced and repeated utterances are required
Solution Approach 1:
The automated assistant's availability is dynamically adjusted based on application context. When the user is actively using an application, the assistant remains available to process spoken utterances and initialize actions, providing full functionality. When the user switches to another application, the assistant automatically becomes unavailable, conserving computational resources while maintaining ease of operation during active use.
3Loss of energy
If dialog session is canceled when another application is opened, then system resources are managed, but user progress is impeded and repeated utterances are required
Solution Approach 1:
The system uses application state changes as feedback signals to manage dialog session lifecycle. When another application is opened, the system detects this as feedback that the user has shifted context, and automatically cancels the ongoing dialog session. This feedback mechanism manages system resources effectively while the rapid session termination prevents wasted time on abandoned tasks.
Data Source
AI summary
Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.