Test Scenario Clustering by Organization Count
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large and complex software systems, such as SAP ERP or Oracle EBS, require extensive testing to verify their integrity, leading to significant IT budget allocation, and organizations often run similar tests, making it difficult to select relevant tests due to diversity in software systems and customizations, resulting in suboptimal testing results.
Innovation Solution
A system and method for rating clusters of test scenarios based on the number of organizations associated with them, where higher popularity ratings indicate more useful tests, allowing for the generation of test scenario templates and suggesting relevant tests to users, while removing less relevant values from the templates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If organizations run many tests to verify software system integrity, then testing coverage and reliability are improved, but testing time and cost increase significantly
Solution Approach 1:
The system performs preliminary actions by collecting and analyzing test scenario data from multiple organizations in advance, identifying popular and effective test scenarios before a new organization needs to test. This pre-analysis enables new organizations to skip the trial-and-error phase and directly adopt proven test scenarios, significantly reducing their testing time while maintaining high reliability.
Solution Approach 2:
The system creates copies of effective test scenarios from one organization and applies them to other organizations. By copying popular test scenarios that have proven effective across multiple organizations, the system enables new organizations to benefit from others' testing experience without conducting extensive their own tests, thus reducing testing time while maintaining verification reliability.
2Reliability
If organizations run many tests to verify software system integrity, then testing coverage is improved, but IT budget allocation increases
Solution Approach 1:
The system performs preliminary analysis of test scenario effectiveness across multiple organizations, identifying which tests provide the best value. This pre-computed knowledge enables new organizations to allocate their IT budgets more efficiently by focusing on proven effective tests rather than running extensive trial tests, thereby reducing overall IT spending while maintaining verification reliability.
Solution Approach 2:
The system enables organizations to copy and reuse effective test scenarios from other organizations, eliminating redundant testing efforts. By sharing test scenarios across the community, the system reduces duplicate spending on the same testing activities and allows organizations to allocate their IT budgets to more critical areas while maintaining comprehensive verification coverage.
3Reliability
If test scenarios are selected from a large diverse body of tests, then testing comprehensiveness is improved, but difficulty in selecting relevant tests increases
Solution Approach 1:
The system implements feedback mechanisms by analyzing which test scenarios are most effective across multiple organizations and using this feedback to rank and prioritize test scenarios. This feedback-driven approach automatically identifies relevant tests for new organizations based on community-wide effectiveness data, making the selection process easier while maintaining comprehensive testing coverage.
Solution Approach 2:
The system changes the parameter of test scenario selection from random or manual choice to popularity-based ranking. By transforming the selection criterion into a measurable parameter (popularity score based on cross-organization usage and effectiveness), the system automatically identifies relevant tests without requiring manual evaluation, thus simplifying the selection process while ensuring comprehensive coverage.
4Productivity
If popular test scenarios are used across organizations, then testing efficiency is improved, but adaptability to organization-specific customizations decreases
Solution Approach 1:
The system applies local quality by allowing organizations to start with popular generic test scenarios and then customize them with organization-specific details. The framework supports both standardized popular tests for efficiency and localized customizations for specific needs, enabling organizations to achieve high testing efficiency on common scenarios while maintaining adaptability for organization-specific requirements.
Data Source
AI summary
System, method, and non-transitory medium for rating popularity of clusters of runs of test scenarios. An interface receives runs of test scenarios, run by users belonging to different organizations, run essentially on same packages of software systems. A clustering module clusters the runs into clusters that include similar runs of test scenarios. An organization counter counts the number of different organizations associated with a cluster; an organization may be considered associated with a certain cluster if the certain cluster includes a run of a test scenario run by a user belonging to the organization. A cluster rating module computes popularity ratings of at least some of the clusters based on the number of different organizations associated with the clusters; the higher the number of different organization associated with a cluster, the higher the popularity rating of the cluster.