Automated GUI Testing via Self-Learning Menu Structure
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated testing methods for graphical user interfaces require manual updates and additional workload when the GUI structure changes, as test steps are device-specific and need to be updated for each software type, leading to increased time and labor costs.
Innovation Solution
A computer-implemented method for automated testing that learns the GUI menu structure by navigating through the interface, creating a structure file with screen header text and images, and uses OCR to analyze screens, allowing for automatic detection and comparison of menu elements, thereby eliminating the need for device-specific test steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual test step updates are performed for each GUI structure change, then testing accuracy is maintained, but labor costs and time consumption increase
Solution Approach 1:
The system performs self-learning of the GUI structure by automatically navigating through menus and capturing screen information. The test script engine independently updates the GUI structure model without requiring manual intervention from testers, thereby maintaining testing accuracy while eliminating the time loss associated with manual updates.
Solution Approach 2:
The system performs preliminary learning of the GUI structure before actual testing begins. By pre-capting menu structures, screen images, and navigation paths, the system prepares the test environment in advance, so that when GUI changes occur, the updated structure is already available for testing without requiring time-consuming manual reconfiguration.
2Measurement precision
If device-specific test steps are used, then testing precision for specific devices is improved, but adaptability to different software types deteriorates
Solution Approach 1:
The test script engine is designed with universal functionality to adapt to different software types and device configurations. Instead of requiring separate test steps for each device, the system learns the specific GUI structure of each device during initialization and then applies the same adaptive testing framework across all devices, achieving both precision and versatility.
Solution Approach 2:
The testing system transitions from static, device-specific test steps to a dynamic, adaptive approach. The test script engine dynamically learns and updates the GUI structure model based on the actual device being tested, allowing the same testing framework to adapt to different software types and device configurations while maintaining testing precision.
3Productivity
If automated test scripts with fixed steps are implemented, then productivity is improved, but flexibility to handle GUI changes deteriorates
Solution Approach 1:
The test script engine implements dynamic adaptation by automatically learning and updating the GUI structure model when changes are detected. Instead of requiring manual reconfiguration of fixed test steps, the system dynamically adjusts its internal model of the GUI structure, maintaining high productivity while gaining the flexibility to handle GUI changes automatically.
Solution Approach 2:
The system incorporates feedback mechanisms where the test script engine continuously monitors the actual GUI structure during testing and compares it with the learned model. When discrepancies are detected, the system automatically updates its internal model, creating a closed-loop system that maintains both productivity and flexibility by adapting to changes based on feedback from actual testing operations.
Data Source
AI summary
A computer-implemented method for automated testing of a graphical user interface (GUI) for a customer device, the method comprising: receiving GUI navigation keys (801); automated learning (802) of the GUI MENU structure by navigating the GUI to create a GUI structure file comprising the screen header text and the screen image for each sub-MENU detected in the MENU structure; receiving (803) data related to the sub-MENU to be tested; performing (804) a testing procedure on the sub-MENU.