Software Build Testing via Fault Localization and Modified Bisection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In fast-paced software development environments, identifying error-inducing commit operations in large-scale software applications is time-consuming and resource-intensive, especially when complex test cases are executed on less than all builds, making it impractical to pinpoint the exact commit causing errors.
Innovation Solution
A modified bisection technique is employed, utilizing fault localization to identify suspect program elements and selecting a starting commit operation near these elements, reducing the number of iterations needed to isolate the error-inducing commit, thereby minimizing time and computing resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional error detection and debugging methods are used on all builds, then measurement precision is improved, but loss of time and use of energy increase significantly
Solution Approach 1:
The patent segments the testing process by dividing builds into different categories (e.g., builds with recent commits vs. stable builds) and applying different test case sets to each segment. This allows comprehensive testing where needed while reducing testing overhead on stable builds, thereby maintaining error detection accuracy while reducing overall testing time and resource consumption.
Solution Approach 2:
The patent applies partial action by selectively executing only a subset of test cases on certain builds rather than running all test cases on all builds. The system determines which test cases are necessary based on build characteristics, commit history, and risk assessment, thereby reducing time and energy consumption while maintaining sufficient error detection capability.
2Reliability
If comprehensive test cases are executed on all builds, then reliability is improved, but productivity decreases due to time consumption
Solution Approach 1:
The patent implements a dynamic testing strategy where the set of executed test cases is adjusted based on real-time build characteristics, commit history, and identified suspect program elements. The system dynamically determines the appropriate testing depth and scope for each build, enabling faster releases for low-risk changes while maintaining thorough testing for high-risk changes, thus balancing reliability and productivity.
Solution Approach 2:
The patent changes testing parameters (such as test case selection criteria, testing depth, and resource allocation) based on build characteristics and identified suspect elements. By adapting testing parameters to the specific context of each build, the system maintains high software quality while reducing unnecessary testing overhead, thereby improving release speed without compromising reliability.
3Loss of time
If fault localization is used to identify suspect program elements, then device complexity increases, but loss of time in error detection is reduced
Solution Approach 1:
The patent applies preliminary action by performing fault localization and identifying suspect program elements early in the testing process, before executing comprehensive test cases. By pre-identifying suspect elements based on commit history and code analysis, the system can focus testing efforts on relevant areas, significantly reducing error identification time while the added complexity is justified by the substantial time savings in the overall testing workflow.
Data Source
AI summary
Various examples are directed to systems and methods for testing software. A testing system may use test case data describing a first test case failed by a first build of the software application, the first build generated by a first commit operation of the plurality of commit operations to execute a fault localization operation, the fault localization operation to identify a first suspect program element of a first build. The testing system may use the first suspect program element and commit data to select a second commit operation from the plurality of commit operations. The testing system may determine an error-inducing commit operation from the plurality of commit operations and executing a corrective action.