UI Object Matching for RPA Under Interface Changes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional robotic process automation (RPA) systems fail to adapt to UI changes, requiring manual intervention and error analysis, leading to increased downtime and potential new errors.
Innovation Solution
An object search method for automation scenarios that utilizes a computing device to search for a target object using multiple attributes with weighted sums, switching to a similar object when the target is not found, and updating search conditions based on successful matches.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional RPA uses exact object matching for automation scenarios, then execution precision is maintained, but the system fails when UI changes occur requiring manual intervention
Solution Approach 1:
The system changes the search parameters from exact matching to similarity-based matching with configurable thresholds. When UI changes occur, the system automatically adjusts search conditions by calculating similarity scores between target objects and available objects, allowing execution to continue with adapted parameters rather than failing completely.
Solution Approach 2:
The object search mechanism transitions from static exact matching to dynamic similarity-based matching. The system continuously evaluates multiple objects against the target using similarity algorithms, and the search strategy adapts in real-time based on what objects are available in the current UI state, enabling the automation to flow smoothly around changes.
2Reliability
If manual error analysis and scenario modification is performed for UI changes, then execution accuracy can be restored, but execution time increases significantly
Solution Approach 1:
The system performs self-service by automatically detecting when object search fails, analyzing the discrepancy between target and available objects, and modifying search conditions without human intervention. The error analysis and scenario adaptation are handled autonomously by the system's similarity calculation engine, eliminating the need for manual developer intervention.
Solution Approach 2:
The system implements feedback loops where the result of each object search is immediately evaluated. When a target object is not found, the system receives feedback about available objects and automatically adjusts search conditions based on similarity calculations, creating a closed-loop system that continuously adapts to maintain execution accuracy without manual input.
3Reliability
If incorrect error analysis is performed to modify automation scenarios, then some issues may be addressed, but new errors may be introduced
Solution Approach 1:
The system replaces manual mechanical error analysis with an automated computational similarity-based analysis system. Instead of relying on human developers to correctly identify and fix errors, the system uses algorithms to calculate similarity scores and automatically adjust search conditions, eliminating human error in the analysis process.
Solution Approach 2:
The system creates copies of the target object's characteristics and compares them against available objects using similarity algorithms. Rather than attempting to correct the original target object definition, the system copies relevant features and finds the best matching available object, preserving the original scenario logic while adapting to UI changes.
4Measurement precision
If multiple search conditions are used to precisely identify target objects, then object identification accuracy improves, but search complexity and processing time increase
Solution Approach 1:
The system applies partial action by using only the necessary subset of search conditions needed to achieve sufficient similarity. Rather than requiring all possible attributes to match exactly, the system evaluates objects based on available attributes and uses similarity scoring to determine adequate matches, reducing the effective complexity while maintaining identification accuracy.
Solution Approach 2:
The search process is segmented into multiple stages: first evaluating objects against primary search conditions, then applying similarity calculations only to candidates that partially match. This segmentation allows the system to handle complex multi-attribute searches efficiently by breaking down the evaluation process into manageable steps rather than processing all conditions simultaneously.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of searching for an object for executing an automation scenario and an apparatus for performing the method are provided. The method of searching for an object for executing an automation scenario according to an exemplary embodiment is performed by a computing device comprising one or more processors and a memory storing one or more programs to be executed by the one or more processors are stored and the method includes searching for a target object based on search information including a plurality of search conditions for the target object that is a target for which an event is to be executed according to an automation scenario, searching for a similar object for the target object based on some of the plurality of search conditions when the target object is not found, and executing the event on the similar object when the similar object is successfully found.