Adaptive UI Test Framework Using Dynamic Label Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Automated testing of software applications is unreliable and high maintenance due to brittleness from small changes in the application or test environment, requiring frequent updates and specific programming knowledge.
Innovation Solution
A framework that characterizes graphical user interface elements, identifies labels and input fields, and uses a test data repository to execute testing scripts, allowing for dynamic injection of test data and generation of reports, which can adapt to different application templates and reduce maintenance costs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated test scripts are used to test software applications, then testing productivity is improved, but the test scripts become brittle to small changes in the application under test or test environment
Solution Approach 1:
The patent introduces an intermediary layer between the test execution engine and the application under test. This intermediary uses machine learning models to interpret UI elements dynamically, allowing test scripts to adapt to changes without direct modification. The intermediary translates stable test intentions into dynamic UI interactions, resolving the brittleness issue while maintaining automation benefits.
Solution Approach 2:
The system dynamically changes parameters such as UI element identification methods from static locators to dynamic descriptions generated by machine learning models. Test data, UI element properties, and interaction parameters are adjusted based on real-time analysis of the application state, enabling scripts to remain reliable despite application changes.
2Loss of time
If automated test scripts are created, then testing speed is improved, but maintenance effort increases due to brittleness
Solution Approach 1:
The system implements self-service capabilities where the machine learning models automatically adapt test scripts to application changes without human intervention. The system self-updates its understanding of UI elements by learning from interactions, automatically recovering from failures and adjusting test paths, thereby reducing maintenance effort while preserving testing speed.
Solution Approach 2:
The system incorporates continuous feedback loops where test execution results, UI element states, and interaction outcomes are fed back to the machine learning models. This feedback enables automatic refinement of test scripts, allowing the system to learn from failures and successes, reducing manual maintenance while maintaining efficient testing cycles.
3Reliability
If traditional automated testing is implemented, then testing coverage is improved, but programming knowledge is required which increases complexity
Solution Approach 1:
The machine learning models serve as an intermediary that abstracts away the complexity of writing and maintaining test scripts. Users with limited programming knowledge can define high-level test intentions, and the intermediary handles the complex task of translating these into detailed test actions, achieving comprehensive coverage without requiring deep programming expertise.
Solution Approach 2:
The system uses machine learning to create copies or replicas of UI element behaviors and interactions. Instead of manually programming each test scenario, the system learns from examples and generates test scripts automatically, reducing the need for programming knowledge while maintaining thorough testing coverage.
Data Source
AI summary
Data is received that characterizes graphical user interface elements of a software application. Thereafter, labels within the received data are identified. Further, those labels having corresponding input fields are identified. These identified input fields are used to poll a test data repository for test data appropriate for such input fields. Using the test data obtained as part of the polling, a testing script for the software application is executed which injects the test data into the identified input fields. Related apparatus, systems, techniques and articles are also described.