Iterative Test Vector Generation for Combinatorial Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current Combinatorial Test Design (CTD) techniques face challenges in achieving higher order test space coverage efficiently, as they often require testing all possible combinations of attribute values, which can be astronomically large, making it impractical to detect higher order defects or faults, especially in complex systems.
Innovation Solution
The method involves modeling inputs to a System Under Test (SUT) as attributes with possible values, generating an initial set of test vectors for complete pairwise coverage, and iteratively reducing the test space to generate new unique test vectors that provide incremental m-wise coverage, ensuring no overlapping test vectors with previous sets, thereby increasing cumulative higher order test coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all possible combinations of attribute values are tested to achieve complete test space coverage, then higher order defect detection capability is improved, but test execution time and resource consumption increase astronomically
Solution Approach 1:
The patent segments the test space coverage achievement into multiple iterative passes. Each pass generates test vectors that cover a specific portion of the test space, progressively building up higher order coverage without requiring all combinations to be tested simultaneously. This divides the overwhelming task into manageable segments executed over time.
Solution Approach 2:
The patent performs preliminary action by generating test vectors in iterative passes before complete coverage is achieved. Each pass prepares a subset of test vectors that contribute to higher order coverage, allowing the system to accumulate coverage progressively rather than requiring all tests to be prepared and executed at once.
2Reliability
If all possible combinations of attribute values are generated to ensure complete coverage, then test space coverage is improved, but the complexity of test case management increases
Solution Approach 1:
The patent segments the complete test space coverage into multiple iterative passes, where each pass generates a manageable subset of test vectors. This segmentation reduces the complexity of test case management by breaking down the overwhelming task of managing all possible combinations into smaller, more tractable segments that can be generated, executed, and managed independently.
Solution Approach 2:
The patent applies partial action by generating test vectors that provide incremental coverage in each pass rather than attempting to generate and manage all possible combinations at once. Each pass produces a partial set of test vectors that contributes to the overall coverage goal, making the management process more manageable.
3Productivity
If iterative test vector generation is performed to achieve incremental higher order coverage, then test efficiency is improved, but the number of iterations required increases
Solution Approach 1:
The patent employs feedback mechanisms where each iterative pass uses information from previously generated test vectors to guide the generation of subsequent test vectors. The system monitors coverage achievement and uses this feedback to optimize the generation process in subsequent passes, potentially reducing the total number of iterations needed compared to naive iterative approaches.
Solution Approach 2:
The patent changes parameters between iterative passes, such as modifying the criteria for test vector selection or adjustment based on coverage achieved in previous passes. By dynamically adjusting generation parameters based on progress, the system can improve efficiency and potentially reduce the number of iterations required to achieve target coverage levels.
Data Source
AI summary
Systems, methods, and computer-readable media are described for expanding test space coverage for testing performed on a System Under Test (SUT) through iterative test case generation from combinatoric pairwise outputs. At each test case generation iteration, a new set of test vectors is generated that provides complete pairwise coverage of the test space but that does not include any overlapping test vector with any previously generated set of test vectors. As such, cumulative m-wise test space coverage (where 2<m≤n) is incrementally increased through each iteration until the iterative process ceases when a desired percentage of m-wise test space coverage is achieved.