Mutation Testing in CI Systems via Targeted Test Subsets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mutation testing in continuous integration and continuous deployment systems is inefficient due to the need for recompiling and rebuilding applications, difficulty in reconstructing source code from bytecode, and high computing resource requirements, leading to delays in the development process.
Innovation Solution
A mutation test system that determines modified source code portions, generates mutated applications, and executes targeted subsets of test suites based on source code changes, using mappings and machine-learned models to optimize testing and reduce resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If mutation testing is performed by recompiling and rebuilding the entire application, then test coverage can be verified, but the development process is significantly delayed due to time-consuming compilation and rebuilding
Solution Approach 1:
The patent segments the application into individual classes or modules, allowing mutation testing to be performed on specific segments rather than the entire application. This enables selective compilation and testing of only the mutated portions, dramatically reducing the time required while maintaining test coverage verification capability.
Solution Approach 2:
The patent performs preliminary compilation of the entire application before mutation testing, so that when mutations are introduced, only the affected segments need to be recompiled. This preliminary action eliminates the need to recompile the entire application for each mutation, reducing development process delay.
2Reliability
If comprehensive test suites are executed on mutated applications, then test efficacy can be verified, but computing resource requirements become prohibitively high
Solution Approach 1:
The patent applies local quality by selecting and executing only the specific test cases that are relevant to the mutated code segments, rather than running the entire test suite. This targeted approach verifies test efficacy for the mutated portions while significantly reducing computing resource consumption.
Solution Approach 2:
The patent performs partial action by executing a subset of the test suite that is sufficient to verify the effectiveness of unit tests for the mutated code, rather than running all tests. This partial execution achieves the necessary verification while reducing resource usage.
3Ease of manufacture
If source code is compiled into bytecode for mutation testing, then mutations can be applied, but reconstructing the original source code from bytecode becomes difficult or impossible
Solution Approach 1:
The patent performs preliminary actions by maintaining and utilizing the original source code throughout the mutation testing process. Instead of attempting to reconstruct source code from bytecode, the system directly applies mutations to the source code, compiles the mutated source code, and executes tests, thereby preserving complete source code information.
Solution Approach 2:
The patent creates copies of the source code for mutation testing while preserving the original. Multiple copies can be mutated independently, allowing comprehensive testing without losing access to the original source code or requiring reconstruction from bytecode.
4Reliability
If frequent rebuilds and comprehensive test runs are performed in CI systems, then build breaks and bugs can be quickly identified, but the technical and resource-intensive nature of these tasks delays product development
Solution Approach 1:
The patent segments both the build process and test execution, allowing CI systems to perform mutation testing on individual classes or modules rather than the entire application. This segmentation enables frequent builds and tests to occur with minimal impact on overall development productivity.
Solution Approach 2:
The patent applies partial action by executing only the necessary subset of tests related to the specific code changes, rather than running comprehensive test suites on every build. This approach maintains quick bug detection capability while reducing the resource intensity and development delay.
Data Source
AI summary
Techniques described herein relate to implementing mutation testing of software applications associated with continuous integration (CI) systems. A mutation test system may determine one or more portions of modified source code within an application codebase. Mutated applications may be generated based on the modified source code, and a mutation test system may determine subsets application test suites for execution based on the portions of the modified source code and/or other factors. In various examples, the mutation test system may use mappings between portions of source code and test subsets, and/or machine-learned models or heuristics-based techniques to determine subsets of test suites based on discreet source code modifications. Mutation testing can be performed by executing the determined test subsets on the mutated applications, and the results may be used by the CI system to control the integration of the code changes into the shared source code repository and/or automated testing of the application build.