Regression Tracking via Equidistant Revision Binary Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for identifying software regressions in automated testing are time and resource intensive, especially when dealing with a large number of revisions, due to the need for extensive binary searches and resource management in complex software environments.
Innovation Solution
An automated system employing binary searches with equidistant revisions to minimize search depth, and optimizing build selection and prefetching during the binary search process to reduce the number of builds required and shorten the triaging cycle.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional approaches (reading change descriptions, dump analysis, statistics analysis, log file inspection) are used for regression identification, then comprehensive analysis can be performed, but the process becomes time and resource intensive when a large number of revisions are involved
Solution Approach 1:
The patent segments the large set of revisions into smaller subsets using binary search methodology. Instead of analyzing all revisions sequentially, the system divides the revision range in half at each step, testing intermediate revisions to narrow down the location of the regression-causing change. This segmentation dramatically reduces the number of revisions that need to be analyzed in detail, resolving the contradiction between comprehensive analysis and time consumption.
Solution Approach 2:
The patent performs preliminary actions by pre-fetching and preparing build artifacts, test environments, and analysis data before they are actually needed for regression identification. By anticipating future analysis needs and preparing resources in advance, the system reduces waiting time during the triaging process while maintaining the ability to perform comprehensive analysis when needed.
2Measurement precision
If conventional approaches are used for regression identification, then thorough analysis is possible, but resource consumption increases significantly with large numbers of revisions
Solution Approach 1:
The binary search approach segments the revision analysis into logarithmic steps rather than linear processing of all revisions. This reduces the computational workload from O(n) to O(log n), where n is the number of revisions. Each segmentation step requires minimal computational resources compared to analyzing all revisions thoroughly, thus resolving the contradiction between analysis thoroughness and resource consumption.
Solution Approach 2:
The patent extracts only the critical intermediate revisions needed for binary search testing, rather than processing all revisions. By taking out only the essential samples at each binary search step, the system performs sufficient analysis to identify the regression-causing revision with minimal computational resource expenditure.
3Productivity
If multiple parallel tracking processes are used to speed up regression identification, then processing throughput increases, but the number of builds required increases
Solution Approach 1:
The patent merges multiple parallel tracking processes by coordinating their binary search operations to share common builds and test results. When multiple processes are running simultaneously, the system combines their build requirements and eliminates duplicates, so that builds created for one process can be reused by other processes. This merging maintains high throughput while significantly reducing the total number of builds required compared to completely independent parallel processes.
Data Source
AI summary
An example system configured to perform regression tracking and triaging includes a processor and memory coupled with the processor. The memory is configured to provide the processor with instructions to schedule the testing of a first list of equidistant revisions associated with a software application. The memory is configured to provide the processor with instructions to schedule the testing of a second list of equidistant revisions associated with the software application. The memory is further configured to provide the processor with instructions to, for a first revision selected from the first list of equidistant revisions and a second revision selected from the second list of equidistant revisions, upon a determination that an equidistance between the first revision and the second revision is below a predetermined threshold, test the second revision using a first build generated based on the first revision, wherein the first build is executable by the processor.