Covert Minimum Set Cover for Test Case Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In the context of hardware and software development, especially in safety-related systems like automotive and avionics, the increasing complexity of designs leads to an exponential growth in the number of tests required for verification, making it computationally expensive to select the most efficient tests within stringent space and time constraints, and existing methods like mutation analysis are impractical due to the large number of simulations needed to determine fault detection capabilities.
Innovation Solution
The approach involves using a covert minimum set cover technique to rank and select a subset of test cases that can detect most faults efficiently, focusing on poorly covered faults and iteratively selecting test cases that detect the largest number of faults, thereby reducing the number of necessary simulations and ensuring that each selected test case can detect at least one fault, while using coverage scores for ranking and minimizing computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If mutation analysis is used to evaluate test quality by injecting artificial bugs, then test detection capability is improved, but the number of simulations required grows exponentially with design complexity
Solution Approach 1:
The patent extracts and analyzes only the most critical fault detection scenarios by identifying poorly covered faults and focusing simulations on those specific cases rather than exhaustively simulating all possible fault-test combinations. This selective extraction reduces the simulation burden while maintaining detection reliability for critical faults.
Solution Approach 2:
The patent performs preliminary fault coverage analysis to identify which faults are poorly covered before conducting mutation analysis. By pre-filtering the fault list to focus only on poorly covered faults, the system prepares the simulation process in advance to target only the most critical detection scenarios, reducing overall simulation requirements.
2Reliability
If the number of test cases increases to cover more faults, then fault detection coverage is improved, but regression testing time increases
Solution Approach 1:
The patent segments the test suite into prioritized groups based on fault detection effectiveness. By dividing and ranking test cases according to their coverage of poorly covered faults, the system enables selective execution of the most critical tests first, reducing the time needed for regression testing while maintaining comprehensive coverage.
Solution Approach 2:
The patent applies partial action by selecting and executing only the most essential test cases that provide maximum fault detection coverage, rather than running the complete test suite. This partial execution approach maintains adequate detection coverage for critical faults while significantly reducing regression testing time.
3Productivity
If test cases are selected to reduce regression testing time, then productivity is improved, but the ability to detect all faults may be compromised
Solution Approach 1:
The patent applies local quality by differentiating the treatment of different faults based on their coverage characteristics. Poorly covered faults receive prioritized attention and are targeted by selected test cases, while well-covered faults are handled by the reduced test set. This localized focus ensures that critical detection gaps are addressed without requiring full regression testing.
Data Source
AI summary
Techniques and systems for test case selection and ordering with covert minimum set cover for functional qualification are described. Some embodiments can determine a first set of test cases by, iteratively, identifying a set of faults that is covered by a smallest set of test cases, determining whether or not a test case that covers a fault is able to detect the fault, and selecting and adding a test case to the first set of test cases. Next, the embodiments can execute a minimum set cover process on the first set of test cases by using coverage scores for test cases in the first set of test cases for ranking.