Regression Testing Variability Mitigation via KNN Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional systems for regression testing are limited by computational constraints, only considering a subset of diagnostic results and requiring substantial resources to identify and compute baselines and thresholds, leading to inefficiencies and false positives in detecting software regressions.
Innovation Solution
A computer-implemented method using a KNN algorithm to identify similar historical regression tests and builds, predict elapsed time for profiled regression tests, and apply mitigation actions based on historical builds to address variability in regression testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional systems compute baselines and thresholds for all diagnostic results, then measurement precision improves, but use of energy and computational resources worsen
Solution Approach 1:
The patent extracts only the most relevant diagnostic results for baseline computation rather than processing all diagnostic data. This selective extraction reduces computational resources while maintaining detection accuracy by focusing on critical performance indicators that most significantly indicate regressions.
Solution Approach 2:
The system computes baselines and thresholds for only a subset of diagnostic results rather than all possible metrics. This partial action approach achieves sufficient measurement precision for regression detection without the excessive computational cost of analyzing every available diagnostic parameter.
2Reliability
If traditional systems analyze all diagnostic results, then reliability of regression detection improves, but productivity worsens due to substantial resource requirements
Solution Approach 1:
The patent extracts and analyzes only the most relevant diagnostic results for regression detection. This selective approach maintains reliable detection by focusing on critical indicators while improving productivity by reducing the time and resources needed to process and evaluate diagnostic data.
Solution Approach 2:
The diagnostic results are segmented into critical and non-critical categories. Only critical diagnostic results are used for baseline computation and regression detection, which maintains detection reliability while significantly improving testing efficiency by eliminating unnecessary processing of less important metrics.
3Productivity
If traditional systems use limited subset of diagnostic results, then productivity improves, but measurement precision and false positive reduction worsen
Solution Approach 1:
The patent applies different quality levels of analysis to different diagnostic results. Critical diagnostic results receive full baseline computation and threshold analysis for high precision detection, while less critical results use simplified evaluation. This local quality approach maintains high measurement precision for important metrics while preserving productivity through selective detailed analysis.
4Reliability
If traditional systems compute comprehensive baselines, then reliability improves, but loss of time in computation worsens
Solution Approach 1:
The patent extracts only the essential diagnostic results required for reliable regression detection. By computing baselines only for these extracted critical metrics rather than all available data, the system maintains detection reliability while significantly reducing the time lost to baseline computation.
Data Source
AI summary
One or more computer processors identify one or more similar, historical regression tests and historical builds utilizing a computed similarity measure between a regressed build and one or more historical builds conducted on a same release cycle, wherein the identified one or more similar historical regression tests and historical builds are K closest neighbors to the regressed build; predict an elapsed time of the one or more profiled regression tests utilizing a KNN algorithm comprising the K closest neighbors each weighted by a corresponding average distance from a test point and the elapsed time as a target variable; responsive to the predicted elapsed time exceeding an actual elapsed time associated with the regressed build, determine that the regressed build is an actual regression; responsive to determining that the regressed build is not due to variability, apply one or more mitigation actions to the regressed build.