Automated UI Element Healing via Multi-Metric Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automation scripts often become unsynchronized with web page elements due to changes in UI element identifiers or the web page structure, leading to errors such as unfound element errors, which account for approximately 13-15% of script failure errors.
Innovation Solution
A method is described that computes a similarity score between expected and candidate objects by using a combination of similarity coefficients, edit distances, and embedding similarities, allowing for the automatic rectification of object locators in automation scripts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automation scripts are created separately from web pages, then script development is independent and efficient, but scripts become unsynchronized with web page elements over time leading to errors
Solution Approach 1:
The system implements feedback by automatically detecting when web page elements change and notifying script authors about synchronization issues. The platform monitors element properties, compares them against script expectations, and provides feedback loops that alert developers to updates needed in their scripts, thereby maintaining reliability without sacrificing independent development efficiency
Solution Approach 2:
The system performs preliminary actions by proactively detecting element changes before scripts fail. It continuously monitors web page elements and prepares synchronization updates in advance, so when scripts are executed, they already have the correct element references, preventing errors before they occur
2Adaptability or versatility
If web page elements are frequently updated, then the web application remains current and functional, but automation scripts fail due to unfound element errors
Solution Approach 1:
When web page elements are updated, the system provides feedback to script authors through notifications and alerts. The platform detects element changes, compares them against script expectations, and communicates the discrepancies to developers, enabling them to update their scripts to match the new element states while maintaining script functionality
Solution Approach 2:
The system handles parameter changes by detecting modifications in element properties such as IDs, classes, or other identifying attributes. When such changes occur, the platform automatically identifies affected scripts and guides authors through the necessary parameter updates, allowing the web application to adapt while maintaining script reliability
3Reliability
If manual updates to automation scripts are performed, then scripts can be synchronized with web page changes, but time and resources are consumed for maintenance
Solution Approach 1:
The system implements self-service by automatically detecting element changes, identifying affected scripts, and providing targeted updates or notifications to script authors. This reduces manual maintenance time by automating the detection and notification processes, allowing authors to focus only on the specific changes needed rather than manually reviewing entire scripts
Solution Approach 2:
The system replaces manual mechanical updating processes with automated detection and notification mechanisms. Instead of requiring authors to manually compare and update scripts against web page changes, the platform uses automated element monitoring and change detection algorithms to identify and communicate necessary updates, significantly reducing maintenance time and effort
Data Source
AI summary
In some implementations, the techniques described herein relate to a method including: receiving an expected object and a candidate object; computing a similarity coefficient between the expected object and the candidate object; computing an edit distance between the expected object and the candidate object; computing an embedding similarity between the expected object and the candidate object; and computing a matching score between the expected object and the candidate object based on the similarity coefficient, the edit distance, and the embedding similarity, the matching score representing a likelihood that the candidate object has replaced the expected object.