Build System Prioritizes Test Functions by Code Reachability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large software projects, running comprehensive test suites can be time-consuming, and a single test failure often renders previous tests futile, highlighting the need for a more efficient prioritization of testing functions to identify issues early in the process.
Innovation Solution
A build system assigns priority levels to testing functions based on the code changes in a new commit, ensuring that functions likely to fail are executed first, utilizing a call graph to determine the reachability and importance of each function in the testing process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If comprehensive test suites are run to ensure code quality, then testing coverage is improved, but testing time increases significantly
Solution Approach 1:
The system performs preliminary analysis of code changes to identify which test functions are most likely to fail before executing the full test suite. By pre-calculating failure probabilities based on code reachability and historical data, the system prepares an optimized execution order in advance, running high-probability failure tests first to quickly identify issues and potentially short-circuit the testing process.
2Ease of manufacture
If tests are run in arbitrary order, then implementation is simple, but time is wasted when failing tests are not executed first
Solution Approach 1:
The system changes the execution parameter of test functions by assigning dynamic priority scores based on code change analysis. Instead of using a fixed or arbitrary order, the system calculates a priority parameter for each test function considering factors like code reachability, test failure history, and change impact, then executes tests in descending order of this parameter to maximize efficiency.
3Productivity
If test functions are prioritized based on code changes, then testing efficiency is improved, but system complexity increases
Solution Approach 1:
The system segments the test function selection and ordering process into distinct analytical components: code change detection, reachability analysis, failure probability calculation, and priority assignment. Each component handles a specific aspect of the prioritization task, making the overall complex system manageable through modular decomposition while maintaining high testing efficiency.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for assigning levels of priority to selected source code functions. One of the methods includes for each selected function, a respective associated first set of functions reachable from the selected function by at most N steps, and a respective associated second set of functions that are each reachable from the selected function by more than N steps and less than M steps are computed. A first partition having all selected functions whose respective associated first set of functions has at least one of the subject functions is computed. A second partition having selected functions not in the first partition and whose respective associated second set of functions has at least one of the subject functions is computed. Selected functions belonging to the first partition are assigned a higher priority than selected functions belonging to the second partition.