Intelligent OOM Test Baselines Through Score Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional technologies lack effective methods to determine the proper memory usage workload needed to trigger a valid Out of Memory (OOM) error, failing to provide optimized baseline testing scenarios that address critical product issues in information processing systems.
Innovation Solution
An Out of Memory test baseline system is implemented using a machine learning system trained with test case scores to predict a baseline test score for unexecuted test cases, employing a test case configuration tuning module to adjust configurations and identify OOM issues, utilizing multidimensional scaling and mean-shift clustering to categorize test cases.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional testing methods are used to determine OOM baseline configuration, then testing coverage can be achieved, but the ability to accurately trigger valid OOM errors is insufficient
Solution Approach 1:
The system performs preliminary actions by executing a subset of test cases before full testing to establish baseline OOM configurations. The machine learning model is trained on results from this preliminary subset to predict outcomes for remaining test cases, enabling accurate OOM error triggering without exhaustive testing of all cases first.
Solution Approach 2:
The machine learning model creates a predictive copy of the testing process by learning patterns from executed test cases and applying this knowledge to predict outcomes for unexecuted test cases. This allows the system to identify which test cases will trigger OOM errors without actually executing them all, improving both reliability and productivity.
2Reliability
If all test cases are executed to ensure comprehensive coverage, then testing completeness is improved, but time consumption increases significantly
Solution Approach 1:
The system applies partial action by executing only a subset of test cases (e.g., 10-30%) to gather training data for the machine learning model. This partial execution is sufficient to learn patterns and predict outcomes for the remaining test cases, achieving comprehensive coverage insights without the time cost of executing all cases.
Solution Approach 2:
The system implements feedback by using results from executed test cases to train the machine learning model, which then predicts outcomes for unexecuted cases. This feedback loop allows the system to identify high-value test cases for OOM error triggering and prioritize them, reducing overall testing time while maintaining completeness.
3Productivity
If machine learning prediction is used to predict baseline test scores, then testing time is reduced, but prediction accuracy may be insufficient
Solution Approach 1:
The system performs preliminary training of the machine learning model using a carefully selected subset of test cases with known outcomes. This preliminary action ensures the model learns accurate patterns before being deployed for prediction, improving prediction accuracy while maintaining testing speed benefits.
Solution Approach 2:
The system dynamically adjusts the testing process by using machine learning predictions to identify and prioritize test cases most likely to trigger OOM errors. The model can be retrained incrementally as more data becomes available, improving accuracy over time while maintaining fast testing throughput.
Data Source
AI summary
Methods, system, and non-transitory processor-readable storage medium for an Out of Memory test baseline system are provided herein. An example method includes executing a plurality of test cases on a system. A test score calculation module calculates a test case score for each of the executed test cases in a subset of the plurality of test cases. An Out of Memory (OOM) test baseline configuration system trains a machine learning system, using the subset test scores, to predict a baseline test score for an unexecuted test case. A test case score prediction module predicts the baseline test score for the unexecuted test case. A test case configuration tuning module tunes the unexecuted test case to determine a baseline configuration for the unexecuted test case, to identify OOM issues when the unexecuted test case is executed on a test system.