Automated Assistant Background Actions During App Use

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Automated assistants often have limited availability and functionality, leading to inefficiencies and waste of computational resources when users attempt to control other applications, as dialog sessions are terminated or require repeated commands due to limited functionality.

Innovation Solution

An automated assistant processes spoken utterances on-device to identify actions for active and background applications, accessing application data to prioritize and execute relevant actions without interrupting user interaction, conserving computational resources by avoiding redundant ASR and NLU modules in multiple applications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If automated assistant processes spoken utterances while user interacts with other applications, then user can control multiple applications simultaneously, but computational resources are wasted when dialog sessions are terminated without performing actions

Engineering Contradiction:

Improvemulti-application control capabilityVSAvoidcomputational resource waste

Core Design Contradiction:

Adaptability or versatilityVSLoss of energy

Solution Approach 1:

The system performs preliminary actions by initializing application actions in response to spoken utterances before the user actually needs them. The automated assistant processes the spoken input, identifies the intended action, and prepares it for execution, ensuring resources are not wasted when dialog sessions would otherwise be terminated.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system implements feedback mechanisms to monitor whether a spoken utterance successfully initializes an application action. If the action is successfully initialized, the dialog session continues; if not, the system can adjust its response strategy, preventing unnecessary computational waste from continuing futile dialog sessions.

Inventive Principle:

Principle #23Feedback

2Ease of operation

If automated assistant terminates dialog session when user opens another application, then system assumes user is no longer interested, but this wastes computational resources when user intends to perform dialog-related actions

Engineering Contradiction:

Improvedialog session continuityVSAvoidtime to re-invoke automated assistant

Core Design Contradiction:

Ease of operationVSLoss of time

Solution Approach 1:

The automated assistant is designed to handle multiple functions simultaneously - it can process spoken utterances, monitor application states, and initialize application actions across different applications. This multi-functionality allows the dialog session to remain active and useful even when the user switches between applications, eliminating the need to re-invoke the assistant.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system maintains continuity of useful action by keeping the dialog session active and capable of initializing application actions regardless of which application the user currently has in focus. The automated assistant continues to process spoken utterances and can trigger actions in background applications, ensuring the dialog remains productive without requiring re-invocation.

Inventive Principle:

Principle #20Continuity of useful action

3Adaptability or versatility

If each application has separate ASR and NLU modules for voice control, then each application can independently process voice commands, but this increases device complexity and computational overhead

Engineering Contradiction:

Improveindependent voice control capabilityVSAvoidnumber of voice processing modules

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The automated assistant serves as a universal voice processing interface that can control multiple applications. Instead of each application having its own ASR and NLU modules, the single automated assistant performs these functions for all applications, reducing device complexity while maintaining the ability to control multiple applications through voice commands.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Solution Approach 2:

The system merges the ASR and NLU functionality from multiple applications into a single automated assistant. This consolidation reduces the total number of voice processing modules on the device, lowering complexity and computational overhead, while the automated assistant maintains the capability to initialize actions across different applications.

Inventive Principle:

Principle #5Merging (Combining)

Data Source

PatentUS12437764B2Initializing non-assistant background actions, via an automated assistant, while accessing a non-assistant application

Publication Date: 2025.10.07 GOOGLE LLC

AI summary

Implementations set forth herein relate to a system that employs an automated assistant to further interactions between a user and another application, which can provide the automated assistant with permission to initialize relevant application actions simultaneous to the user interacting with the other application. Furthermore, the system can allow the automated assistant to initialize actions of different applications, despite being actively operating a particular application. Available actions can be gleaned by the automated assistant using various application-specific schemas, which can be compared with incoming requests from a user to the automated assistant. Additional data, such as context and historical interactions, can also be used to rank and identify a suitable application action to be initialized via the automated assistant.