Automated Behavior Model Derivation for Software Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Creating and maintaining formal behavior models for model-based testing (MBT) in software engineering is challenging due to the need for extensive knowledge of the system under test (SUT) and the modeling language, which hinders the adoption of MBT in industrial settings, and automating system tests is time-consuming and requires frequent updates with changes in the SUT.
Innovation Solution
A computer-implemented method to determine a behavior model of a computing system under test by executing initial test scripts, recording states, and iteratively refining the model through generating new test scripts to explore unobserved behavior, using a user interface to automate web browsers and filter events, and storing the model as a directed graph for regression testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If formal behavior models are created and maintained for model-based testing, then test automation reliability is improved, but the complexity of the process increases due to requiring extensive knowledge of the system and modeling language
Solution Approach 1:
The system performs self-service by automatically generating behavior models through iterative exploration of the system under test. The modeling process does not require external experts to manually create models, but rather the system autonomously discovers states and transitions by executing test scripts and observing behavior, thereby reducing the knowledge barrier while maintaining reliability
Solution Approach 2:
The approach changes the parameter of model creation from manual expert-driven process to automated iterative exploration. By transforming the modeling process into an automated sequence of test executions and state recordings, the complexity parameter is reduced while preserving the reliability benefits of formal behavior models
2Productivity
If manual test cases are automated, then testing productivity is improved, but the time required for automation development increases
Solution Approach 1:
The system performs preliminary actions by automatically generating the behavior model before test case execution. The model is constructed in advance through iterative exploration, so that subsequent test cases can be automatically derived from it, reducing the time needed for automation development while maintaining high productivity
Solution Approach 2:
The behavior model serves as an intermediary between manual test cases and automated testing. Instead of directly converting manual test cases to automation, the system uses the behavior model as a intermediate representation that captures system behavior, from which automated test cases can be systematically generated, thereby reducing automation development time
3Measurement precision
If the behavior model is refined iteratively to cover unobserved behavior, then measurement precision is improved, but the time required for model determination increases
Solution Approach 1:
The iterative refinement process maintains continuity of useful action by systematically exploring unobserved behavior through generated test scripts. Each iteration continues the exploration where the previous one left off, progressively improving behavior coverage without redundant repetitions, thereby achieving high measurement precision while managing time efficiently
Solution Approach 2:
The system uses feedback from observed system behavior to guide iterative model refinement. By monitoring which states and transitions have been observed and which remain unexplored, the system adjusts its test script generation to target unobserved behavior, improving coverage precision while avoiding unnecessary exploration of already-covered areas
Data Source
AI summary
Methods, systems, and computer-readable storage media determining a behavior model of a computing system under test. In some implementations, actions include executing, using a user interface of a computing SUT, an initial test script on the SUT; recording, after executing the initial test script, a state of the SUT in the behavior model by observing one or more events that can be triggered using the user interface of the SUT; and iteratively refining the behavior model until an end condition is reached by generating one or more new test scripts, executing the new test scripts on the SUT to test unobserved behavior, and recording one or more new states reached by executing the new test scripts on the SUT in the behavior model.