Operation Automation with Adaptive Web Element Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robotic process automation (RPA) systems struggle to find and operate on web elements when the description of a web page changes, leading to failed operations due to the inability to adapt to updates in web page structures.
Innovation Solution
An operation automation system that stores scenario information including web page identifiers and analysis information about operation target web elements, uses a scenario executor to determine the presence of these elements, and employs machine learning to identify new operation targets based on analysis of web page changes, utilizing a model to calculate the likelihood of candidate elements being the new targets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional RPA systems use fixed web element identifiers to automate web operations, then operation precision is maintained, but adaptability to web page changes deteriorates
Solution Approach 1:
The system performs preliminary actions by storing analysis information about web elements (such as surrounding elements, parent elements, and contextual attributes) before web page changes occur. This preliminary data is saved and later used to identify corresponding elements even when their identifiers change, allowing the system to adapt to web page updates while maintaining operation precision.
Solution Approach 2:
The patent introduces analysis information as an intermediary between the fixed identifier and the actual web element. Instead of directly relying on immutable identifiers, the system uses contextual analysis information (surrounding elements, parent elements, positional relationships) as a mediator to identify web elements, enabling both precision and adaptability.
2Reliability
If RPA systems manually update scenario descriptions when web pages change, then operation reliability is maintained, but productivity deteriorates
Solution Approach 1:
The system performs self-service by automatically updating scenario descriptions when web pages change. The analysis process compares stored analysis information with current web page elements, automatically identifies corresponding elements, and updates the scenario without human intervention, maintaining reliability while improving productivity.
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors web page changes, compares them with stored analysis information, and automatically adjusts scenario descriptions based on the detected changes. This feedback loop ensures operation reliability is maintained while eliminating manual update requirements, thereby improving productivity.
3Adaptability or versatility
If RPA systems reconfigure automation scenarios when web elements cannot be found, then adaptability improves, but device complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the web page structure into hierarchical components (parent elements, surrounding elements, and target elements). This segmented approach allows the system to identify web elements through their contextual relationships rather than relying on a single complex identifier, improving adaptability while managing complexity through structured breakdown.
Data Source
Figure 1
Figure 2
Figure 3A~3B
AI summary
A storage stores a scenario including a web page identifier, an operation target web element identifier, and a web operation on the operation target web element and stores auxiliary information for identifying the operation target in a web page. A scenario executor reads a web page of the web page identifier included in the scenario with a web browser, determines whether or not the operation target web element identifier is present within the read web page. The scenario executor selects a new operation target web element by analyzing content of the read web page using the auxiliary information when it is determined that the operation target web element identifier is absent in the determination process, and performs a web operation described in the scenario for the web element selected in the analysis process.