Automated Software Regression Source Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software regression identification in complex software systems is time-consuming and cumbersome due to the manual search of log files and the difficulty in predicting second and higher-order effects of multiple code changes, especially across various platforms.
Innovation Solution
A method using QTrace and DTrace tools to analyze code coverage and versioning information to quickly identify potential sources of software regressions by comparing test results across different versions of the code, isolating the specific portion of source code changed in intermediate versions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual search of log files is used to identify software regressions, then debugging can be performed, but the process becomes time-consuming and cumbersome
Solution Approach 1:
The patent replaces manual mechanical searching of log files with automated computer-based analysis systems. The system automatically compares test results across multiple builds, identifies regressions, and locates affected source code portions, substituting human effort with computational automation to reduce time while maintaining accuracy
Solution Approach 2:
The patent introduces an intermediary regression analysis system that acts as a mediator between test execution and debugging processes. This system processes test results, compares them across builds, and provides structured regression information, serving as an intermediate layer that automates the analysis workflow and eliminates manual log file searching
2Measurement precision
If brute force methods like binary search are used to identify regression sources, then the cause can be located, but the process requires numerous man-hours and is extremely costly
Solution Approach 1:
The patent performs preliminary actions by automatically comparing test results across multiple builds before debugging begins. The system pre-identifies regressions and pre-locates the portions of source code that differ between builds, preparing structured information in advance that guides the debugging process and eliminates the need for time-consuming binary search methods
Solution Approach 2:
The patent extracts only the relevant information needed for regression identification by automatically comparing test results and isolating the specific portions of source code that differ between builds. This extraction process filters out unnecessary data and focuses directly on the regression-causing changes, eliminating the need for exhaustive brute force searching through all code
3Ease of manufacture
If software is tested after a threshold of change is reached, then testing can be performed, but it is difficult to predict second and higher order effects of multiple code changes
Solution Approach 1:
The patent segments the analysis by comparing each build against a baseline build and identifying specific portions of source code that differ. This segmentation allows the system to track changes individually and collectively, capturing both direct and indirect effects of multiple code changes while maintaining the ability to test after threshold changes are reached
4Measurement precision
If software is run on multiple platforms to identify hardware or software problems, then platform-specific issues can be detected, but engineers are overwhelmed by the numerous factors that must be considered
Solution Approach 1:
The patent creates a universal regression analysis system that functions across multiple platforms. The system automatically adapts to different platforms by comparing test results and identifying regressions regardless of the underlying hardware or software environment, providing a single multi-functional solution that eliminates the need for engineers to manually manage platform-specific testing complexities
Data Source
AI summary
Techniques for isolating software regressions are provided. In one approach, QTrace is used to generate DTrace scripts to gather timing information of one or more portions of source code during a test of particular version of a program. Based on results of one or more tests of multiple versions of a program, a regression is identified with respect to a particular portion of source code. It is determined that the particular portion of source code was changed during an intermediate version of the program (i.e., between two tests). The intermediate version and/or the particular portion of source code are identified as being a potential source of the regression.