Machine Learning Test Case Generation via K-Wise Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Creating reliable machine learning models is challenging due to the complexity of determining test cases for all possible states, which is an exponential process, making it computationally expensive and time-consuming.
Innovation Solution
A computer-implemented method that selects a subset of states for test cases using k-wise coverage, where each subset of dimensions has at least one test case, and applies constraints to reduce the number of meaningful states, allowing for efficient reduction of computing time and test case determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If test cases are determined for all possible states, then model reliability is improved, but computing time and complexity increase exponentially
Solution Approach 1:
The patent applies partial action by determining test cases for only a subset of states rather than all possible states. Specifically, it selects states that achieve k-wise coverage (where k is typically 2 or 3), meaning every combination of k dimensions is covered by at least one test case. This partial coverage approach maintains sufficient model reliability while avoiding the exponential complexity of complete state enumeration.
Solution Approach 2:
The patent changes the parameter of test case selection from exhaustive enumeration to coverage-based sampling. By introducing the coverage parameter k and selecting states based on achieving k-wise coverage rather than complete enumeration, the system transforms the problem from exponential complexity to polynomial complexity while maintaining acceptable model reliability.
2Manufacturing precision
If all dimensions and options are used for test case determination, then model quality is improved, but computing time increases
Solution Approach 1:
The patent extracts and focuses only on the most critical dimensions and options for test case determination. By identifying and selecting a subset of dimensions that achieve k-wise coverage, it extracts the essential test cases needed for model quality while excluding redundant combinations, thereby reducing computing time without significantly compromising model quality.
Solution Approach 2:
The patent applies partial action by using only the necessary subset of dimensions and options required to achieve k-wise coverage rather than exhaustively testing all possible dimension combinations. This selective approach maintains model quality by ensuring critical dimension interactions are covered while avoiding the computational overhead of testing all possible combinations.
3Productivity
If k-wise coverage is achieved with a subset of states, then computing time is reduced, but the number of test cases may be insufficient
Solution Approach 1:
The patent changes the parameter of test case selection from exhaustive enumeration to coverage-based sampling. By introducing the coverage parameter k and selecting states based on achieving k-wise coverage rather than complete enumeration, the system transforms the problem from exponential complexity to polynomial complexity while maintaining acceptable model reliability.
Solution Approach 2:
The patent employs feedback mechanisms to iteratively improve test case selection. The system evaluates whether the selected subset of states achieves the desired k-wise coverage and uses this feedback to adjust and refine the test case set, ensuring that model validation reliability is maintained while benefiting from reduced computing time.
Data Source
AI summary
A device and a computer-implemented method for machine learning. First input data are provided which encompass information concerning dimensions and options for the machine learning. At least one of the options is associated with at least one of the dimensions as a function of information concerning the dimensions and options for at least one test case for the machine learning. A combination of options for a subset of the dimensions that is lacking in the set of test cases is determined, and a test case is determined for this combination.