Regression Test Selection Using AHP for Coverage-Time Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies fail to effectively quantify and optimize the selection of regression test cases in information processing systems, particularly in storage products, due to limited resources and the inability to consider multiple factors such as feature overlap, test point overlap, execution time, and regression test case creation time, leading to inadequate regression testing coverage.
Innovation Solution
A test selection system utilizing the Analytic Hierarchy Process (AHP) to assign weights to regression test cases, considering multiple factors like feature coverage, code coverage, defect tickets, and execution time, and optimizing the selection process through visualization and normalization to identify the optimal subset of test cases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all regression test cases are executed to ensure comprehensive testing coverage, then testing coverage is improved, but resource consumption and execution time increase significantly
Solution Approach 1:
The patent extracts and selects only the most valuable subset of regression test cases from the complete test suite. By calculating scores based on multiple factors (feature coverage, defect detection capability, execution time, creation time) and selecting top-scoring cases, the system extracts the essential testing coverage needed while excluding redundant or low-value test cases, thus reducing execution time while maintaining reliability.
Solution Approach 2:
The patent changes the evaluation parameters from simple binary selection (run/not run) to a multi-dimensional scoring system. By introducing weighted factors including feature coverage, defect tickets, execution time, and creation time, the system transforms the test case selection into a parameter-optimized problem where the best subset can be identified based on multiple criteria simultaneously, balancing coverage and resource usage.
2Measurement precision
If multiple factors are considered for test case selection to optimize coverage, then selection quality is improved, but system complexity increases
Solution Approach 1:
The patent segments the complex selection problem into distinct evaluable factors: feature coverage, defect detection capability, execution time, and creation time. Each factor is calculated and weighted separately, then combined to produce an overall score. This segmentation allows the system to handle multiple considerations systematically without becoming unmanageably complex, as each segment can be independently computed and then integrated.
3Productivity
If limited resources are used for regression testing, then resource efficiency is improved, but testing coverage decreases
Solution Approach 1:
The patent introduces a multi-factor scoring system that explicitly incorporates resource-related parameters (execution time, creation time) alongside coverage metrics. By weighting these parameters appropriately, the system identifies test cases that provide maximum coverage value per unit of resource consumed, thus achieving high resource efficiency without sacrificing coverage quality.
Solution Approach 2:
The patent applies different weighting priorities to different test cases based on their local characteristics. Test cases with better defect detection records, higher feature coverage, or more efficient execution receive higher scores and are prioritized. This local quality adjustment ensures that limited resources are allocated to the most valuable test cases rather than treating all cases equally, maintaining coverage while improving efficiency.
Data Source
AI summary
Methods, system, and non-transitory processor-readable storage medium for a test selection system are provided herein. An example method includes selecting a regression test case from a plurality of regression test cases in a software testing lifecycle system. A test selection system calculates a score for the regression test case using an Analytic Hierarchy Process (AHP) assigned weight. Using the score, the test selection system visualizes a relationship between the regression test case and goals associated with a regression testing effort. The test selection system selects the regression test case for use in the regression testing effort based on the visualized relationship, and executes the regression test case on a system.