Cross-App Routine Automation from Detected User Actions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Users often perform repetitive tasks on mobile devices that are time-consuming and prone to errors due to lack of awareness of device capabilities and require multiple application interactions, leading to inefficient battery usage and memory consumption.
Innovation Solution
Automated assistant suggests routines based on user actions identified through machine learning models and interface analysis, allowing users to perform tasks with minimal input by selecting an interface element or uttering a command, thus reducing the need for repetitive actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If users perform tasks manually through multiple applications, then task completion is achieved, but time consumption and error probability increase
Solution Approach 1:
The system creates automation routines that copy and replicate manually performed task sequences. Once a user performs a task manually through multiple applications, the system captures the action sequence and creates an automated routine that replicates the same workflow, allowing future executions to be performed automatically without manual repetition.
Solution Approach 2:
The system enables tasks to serve themselves through automated routines. After initial manual execution, the automation framework allows tasks to be performed autonomously by the device itself, with routines executing automatically based on detected user actions or triggers, reducing the need for continuous manual intervention.
2Productivity
If users manually perform tasks through multiple applications, then task completion is achieved, but computing resources are consumed unnecessarily
Solution Approach 1:
The automation framework allows the device to perform tasks autonomously through routines that execute automatically. Once a routine is created from manual user actions, the system can execute these routines automatically without requiring continuous active user engagement, thereby reducing the computational overhead and energy consumption associated with manual task performance.
Solution Approach 2:
The system performs preliminary analysis of user actions to identify and create automation routines. By analyzing and pre-processing user interaction patterns, the system prepares automation routines in advance that can be executed efficiently, reducing the computational resources needed during actual task execution.
3Adaptability or versatility
If users perform multiple steps in multiple applications, then complex tasks are completed, but user input errors increase
Solution Approach 1:
The system captures the exact sequence of user actions across multiple applications and creates faithful copies in the form of automated routines. This copying process preserves the intended workflow while eliminating human error, as the automated routine replicates the correct sequence of actions consistently without deviation.
Solution Approach 2:
The system monitors and detects user actions across multiple applications, providing feedback about detected patterns and suggesting automation routines. This feedback mechanism allows the system to learn from user behavior and improve the accuracy of automated routines, ensuring they correctly replicate the intended task sequence while reducing errors.
Data Source
AI summary
Implementations relate to identifying actions performed by a user while the user is interacting with multiple applications and providing a routine suggestion to the user based on the identified actions. While a user is interacting with one or more applications, the user actions are determined. The user actions are compared to one or more template routines and a template routine is selected that matches the actions of the user and provided to the user as a suggested routine. The suggested routine can be implemented by an automated assistant to perform the actions of the template by providing a corresponding command.