Web Task Automation via Recorded Demonstration Adaptation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional web task automation methods require extensive user interaction and are not adaptable to changes in web page structures and programming, making them inefficient and rigid.
Innovation Solution
A system utilizing a modelling component, a recorder component, and a playback component to automate web tasks through recorded demonstrations, allowing users to execute tasks with voice commands or text snippets, and adapt to changes in web page structures by generating a repository of demonstrations and merging multiple recordings into a single action sequence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional web browser automation methods are used, then basic web tasks can be performed, but the system requires extensive user interaction and is not adaptable to changes in web page structures
Solution Approach 1:
The system performs preliminary actions by recording user demonstrations of web tasks in advance. These recorded demonstrations capture the sequence of actions, web page structures, and element identifiers. When automating similar tasks later, the system retrieves and adapts these pre-recorded demonstrations rather than requiring users to manually program each task from scratch, thereby reducing user interaction requirements while maintaining adaptability to different web page structures.
Solution Approach 2:
The system creates copies of recorded task demonstrations and adapts them to new web pages. Instead of requiring users to re-program tasks when web page structures change, the system copies the core logic from recorded demonstrations and automatically adjusts element identifiers and selectors based on the current web page structure, enabling adaptable automation with minimal user intervention.
2Extent of automation
If rigid automation models are used, then task execution can be automated, but the system cannot adapt when web pages change their internal structures and programming
Solution Approach 1:
The system implements dynamic automation by continuously monitoring web page structures during task execution. When changes in web page internal structures or programming are detected, the system automatically updates its automation scripts by comparing current page structures with recorded demonstrations and adjusting element selectors, identifiers, and task sequences accordingly. This dynamic adaptation maintains high automation capability while responding to web page changes.
Solution Approach 2:
The system incorporates feedback mechanisms that monitor web page structures during task execution. When automation attempts fail or web page changes are detected, the system receives feedback about the structural changes and automatically retrieves relevant recorded demonstrations to update its automation logic. This feedback loop enables the system to maintain high automation capability while adapting to web page changes without requiring manual reprogramming.
3Productivity
If manual mouse and keyboard interactions are used for web tasks, then precise control is achieved, but the process is time-consuming and inefficient
Solution Approach 1:
The system implements self-service automation where recorded task demonstrations are automatically retrieved, adapted, and executed without requiring manual mouse or keyboard interactions. The system autonomously navigates web pages, fills forms, clicks buttons, and performs tasks by replaying and adapting recorded demonstrations. This eliminates time-consuming manual interactions while maintaining task execution precision, significantly improving productivity.
4Reliability
If exhaustive mouse clicking and keyboard typing are required for web tasks, then complete task execution is achieved, but the complexity and time required increase significantly
Solution Approach 1:
The system copies the essential task logic from recorded demonstrations rather than requiring exhaustive manual programming of every mouse click and keyboard type. By capturing and reusing demonstrated task sequences, the system achieves complete task execution with simpler automation scripts, reducing device complexity while maintaining task completion accuracy through faithful reproduction of proven task workflows.
Data Source
AI summary
A system and method for automating a task on a web page. The system and method include a recording engine, wherein a user can record a new task. Based on template matching, the system and method can be used to generate a model for carrying out the task on a new website. Based on a natural language input, the system can determine the task, specifics for implementing the task, and the web page or web pages to carry out the task on.