Software Regression Detection Through Payload Impact Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Determining the impact of individual payloads on the performance metrics of complex software products is difficult due to the multiplicity of payloads in each build, which are often propagated from lower levels with additional payloads from other development teams, masking the true contributors to software regression.
Innovation Solution
A regression detector analyzes performance metrics using a multiple linear model to estimate coefficients for each payload, identifying those with statistically significant impacts by applying techniques like least squares estimation and LASSO, allowing for focused remedial actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple payloads are included in each build to reflect complex software product development, then the software product functionality and completeness is improved, but the difficulty of detecting and measuring the impact of individual payloads on performance regression increases
Solution Approach 1:
The patent segments the complex software build into individual traceable payloads, each with unique identifiers. By tracking which payloads are present in each build and comparing performance metrics across builds, the system isolates the impact of individual payloads even within multi-payload builds. This segmentation approach allows the software to maintain full functionality while enabling precise measurement of each payload's contribution to performance regression.
2Reliability
If payloads are propagated from lower levels in the hierarchy, then the software product completeness is improved, but the ability to identify true contributors to software regression deteriorates due to masking effects
Solution Approach 1:
The patent implements a feedback mechanism that tracks performance metrics across multiple builds and compares them against baseline values. When performance regression is detected, the system provides feedback about which payloads are present in the problematic build, enabling identification of contributing factors. This feedback loop maintains complete software functionality while precisely identifying regression sources through systematic comparison and analysis of payload-performance relationships.
Data Source
Figure 1A
Figure 1B
Figure 2~3
AI summary
Techniques for detecting software regression in computing systems are disclosed. One example technique includes fitting data in a dataset into a multiple variable model to obtain a set of estimated coefficients. The dataset having data representing multiple entries each containing an identification of a combination of multiple payloads included in a build of a software product and a corresponding value of a performance metric of executing the build at a computing device. The payloads individually represent a source code change, a feature enablement, or a configuration modification of the software product. The estimated coefficients individually correspond to one of the payloads. The method further includes in response to determining that a corresponding estimated coefficient of one of the payloads has an absolute value that is greater than a preset threshold, indicating that a software defect is likely present in the corresponding payload.