Commit Prioritization for Faster Regression Source Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current software development processes, particularly in continuous integration, face significant challenges in identifying software bugs efficiently, as existing methods like bisection require substantial build and test time due to their O(log n) worst-case time complexity, making it difficult to isolate culprit commit change sets quickly.
Innovation Solution
A method that utilizes source code coverage information, historical test case failure and fix records, and source code dependency graph information to rank commits in a priority queue, prioritizing builds and tests based on weight assignments, thereby reducing build and test time by filtering irrelevant commits and optimizing the order of relevant ones.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If bisection method is used to identify culprit commit change sets, then identification accuracy is improved, but build and test time increases significantly
Solution Approach 1:
The system performs preliminary actions by pre-calculating source code coverage information, historical test case failure and fix records, and source code dependency graph information before the actual bug identification process. This allows commits to be pre-ranked and prioritized, so when a regression occurs, the system can quickly identify the culprit commit without performing exhaustive builds and tests, thus maintaining high identification accuracy while reducing time loss
Solution Approach 2:
The system segments the commit identification process by dividing commits into different priority levels based on coverage information, historical failure records, and dependency graphs. Instead of treating all commits equally in the bisection process, the system segments them into high-priority (likely culprit) and low-priority (less likely culprit) groups, allowing focused testing on segmented high-priority commits first, thereby reducing overall build and test time while maintaining identification accuracy
2Measurement precision
If all relevant commits are built and tested in order, then identification accuracy is improved, but productivity decreases
Solution Approach 1:
The system performs preliminary ranking of commits using source code coverage information, historical test case failure and fix records, and source code dependency graph information before the actual bug identification process. This preliminary action creates a prioritized list of commits, allowing the system to test high-priority commits first and identify bugs faster without sacrificing identification accuracy
Solution Approach 2:
The system applies local quality by treating different commits differently based on their individual characteristics. Commits are assigned different priority levels based on their coverage information, historical failure records, and dependency relationships. High-priority commits (those with higher likelihood of causing regression) are tested first with more resources, while low-priority commits are tested later or with fewer resources, thereby improving productivity while maintaining identification accuracy
Data Source
AI summary
Managing source code change set commits to decrease time to identify software bugs in versions of a software project is provided. An initial order of relevant commits corresponding to the software project is determined in a priority queue based on a weight of each respective commit of the relevant commits. The initial order of the relevant commits is adjusted in the priority queue based on a build distance between the relevant commits of affected source files. A particular commit having a highest priority ranking is selected in the priority queue for build and test. The build and the test of the particular commit having the highest priority ranking in the priority queue is executed.