Software Test UI Identifier Updates Through Semantic Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated software tests fail due to changes in user interface element identifiers, such as renaming, despite underlying functionality remaining the same, leading to inefficiencies and resource waste in manual reconfiguration.
Innovation Solution
Techniques for automatically identifying semantically equivalent user interface elements in updated software artifacts by analyzing screenshots or code, updating test definitions to reference these elements, and modifying affected tests to prevent future failures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If automated tests use fixed user interface element identifiers, then test execution is straightforward and automated, but tests fail when UI element identifiers change even though functionality remains the same
Solution Approach 1:
The patent changes the parameter used to identify UI elements from fixed identifiers to dynamic identifiers based on semantic equivalence. Instead of relying on static element IDs that may change, the system uses screenshots and AI-based image recognition to dynamically identify elements by their visual and functional characteristics, allowing tests to adapt when identifiers change.
Solution Approach 2:
The patent replaces the traditional mechanical approach of string-matching UI element identifiers with an AI-based visual recognition system. Instead of programmatically searching for element IDs in code, the system uses machine learning models to analyze screenshots and identify UI elements based on their visual appearance and spatial position, substituting automated image processing for traditional code-based identification.
2Reliability
If manual reconfiguration is performed when tests fail, then test accuracy can be maintained, but time and resources are wasted
Solution Approach 1:
The patent implements self-service by enabling the test system to automatically detect and adapt to UI changes without human intervention. When a test fails due to UI element changes, the system automatically captures screenshots, uses AI to identify the changed elements, updates the test definitions with new element identifiers, and reexecutes tests - completely automating what previously required manual reconfiguration.
Solution Approach 2:
The patent introduces feedback loops where test execution results automatically trigger analysis and correction processes. When tests fail, the system feeds this information back into the test definition through automated analysis of screenshots and AI-based element identification, continuously improving test accuracy without manual intervention.
3Adaptability or versatility
If UI element identifiers are changed during software updates, then software can be improved and updated, but existing automated tests break
Solution Approach 1:
The patent introduces dynamics into the test system by making element identification adaptive rather than static. The system continuously adapts to UI changes by using AI-based visual recognition to dynamically identify elements based on their current state in screenshots, allowing the test system to remain effective despite ongoing software updates and identifier changes.
Solution Approach 2:
The patent performs preliminary action by proactively detecting UI changes before they cause test failures. The system continuously monitors UI elements using screenshot analysis and AI recognition, identifying changes in element identifiers or positions ahead of time and automatically updating test definitions to prevent future test breaks.
Data Source
AI summary
The present disclosure provides techniques and solutions for automatically correcting software tests. When a test failure is detected, it is determined whether a screenshot or code associated with a second version of a software artifact includes a user interface element that has a semantically equivalent identifier to a user interface element in a screenshot or code associated with a first version of the software artifact. Identifying a semantically equivalent identifier can include determining a section of the user interface, in the screenshot of the code, of the first version of the software artifact where the user interface is located and searching the corresponding section of the user interface of the second version of the software artifact. A definition of the software test can be updated to reference the semantically equivalent user interface element.