Test Scenario Template Generation for Software Configuration Validation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex software systems, such as ERP systems, face challenges in efficiently validating configuration changes due to the overwhelming cost and time required for full regression testing, and the difficulty in deciding which tests to run and what they should involve, often resulting in ineffective and prolonged testing processes.
Innovation Solution
A method and system for generating and selecting test scenario templates relevant to validating software systems by clustering runs of test scenarios from multiple organizations, identifying connections between configuration changes and test scenarios, and selecting representative templates based on the number of associated organizations, to streamline the testing process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If full regression testing is performed to validate configuration changes, then testing completeness and reliability are improved, but testing time and cost increase significantly
Solution Approach 1:
The patent segments the testing process by dividing test scenarios into groups based on configuration elements. Instead of running all possible tests, the system identifies and executes only the relevant test groups associated with specific configuration changes, thereby reducing testing time while maintaining reliability for the changed elements.
Solution Approach 2:
The system applies partial action by selecting and executing only the subset of test scenarios that are relevant to the configuration changes being made. This avoids the excessive action of running the complete regression test suite, reducing time and resource consumption while still achieving adequate validation coverage.
2Reliability
If comprehensive test scenarios are designed to cover all configuration changes, then testing coverage is improved, but device complexity and difficulty of managing tests increase
Solution Approach 1:
The patent segments test scenarios into groups associated with specific configuration elements. This segmentation reduces the complexity of managing comprehensive tests by organizing them into manageable, logically-connected groups that can be selectively executed based on the configuration changes being made.
Solution Approach 2:
The system creates a universal framework that handles diverse configuration changes through a common process. The test scenario grouping mechanism provides a multi-functional solution that works across different types of configuration elements and changes, reducing the need for separate complex testing procedures for each scenario.
3Adaptability or versatility
If each organization independently develops its own test knowledge, then organizational autonomy is maintained, but knowledge accumulation time and testing effectiveness are reduced
Solution Approach 1:
The patent merges test knowledge across multiple organizations by collecting and aggregating test scenario data from different sources. This combining approach allows organizations to benefit from collective testing experience and knowledge while maintaining their own autonomous testing capabilities, reducing the time required for individual knowledge accumulation.
Solution Approach 2:
The system acts as an intermediary that collects, processes, and distributes test knowledge between organizations. This intermediary framework enables knowledge sharing and accumulation without requiring direct organizational collaboration, maintaining autonomy while accelerating knowledge development through centralized aggregation and distribution.
4Reliability
If extensive test data is collected from multiple organizations, then collective knowledge and testing effectiveness are improved, but data processing complexity and time increase
Solution Approach 1:
The patent extracts only the essential and relevant test scenario data from the extensive collected information. By filtering and extracting key elements such as test scenario groupings and configuration element associations, the system reduces data processing complexity while retaining the core knowledge needed for effective testing.
Solution Approach 2:
The system segments the extensive test data into organized groups based on configuration elements and test scenarios. This segmentation structure makes the large volume of collected data more manageable and processable, reducing complexity by breaking down the data into logical, organized units that can be efficiently analyzed and applied.
Data Source
AI summary
System, method, and non-transitory medium for selecting a test scenario template useful for testing a configuration change involves execution of the following: identifying runs of test scenarios run by users belonging to different organizations implementing configuration changes on software systems; identifying first connections between configuration changes and the runs; clustering the runs into clusters of similar runs; identifying, from the first connections and the clustering, second connections between configuration changes and the clusters; receiving a certain configuration change of a certain user; identifying a certain cluster of similar runs which correspond to the certain configuration change by comparing the certain configuration change with configuration changes that take part in the second connections; generating test scenario templates based on runs belonging to the certain cluster; and selecting, based on number of different organizations associated with the templates, a representative test scenario template to represent the certain cluster.