Test Framework Selecting Tests via Code Delta
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional software testing frameworks require repetitive testing of all code lines even when only minor changes are made, leading to unnecessary time and resource consumption, as they lack efficient methods to identify and focus on newly added or modified code.
Innovation Solution
A test framework that utilizes test coverage data and version control systems to selectively execute test routines only on newly added or modified code lines, prioritizing test routines with the highest coverage to quickly assess code stability and reduce redundant testing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire set of test routines is run to ensure comprehensive testing, then testing completeness is improved, but testing time and resource consumption increase
Solution Approach 1:
The patent segments the test routine execution by dividing the code base into changed portions (identified through version control delta) and unchanged portions. Test routines are then selectively executed only on the changed portions, rather than running the entire test suite. This segmentation allows comprehensive testing of affected code while avoiding redundant execution on unchanged code, thus resolving the contradiction between testing completeness and testing time.
2Reliability
If all test routines are executed to ensure code stability, then testing thoroughness is improved, but resource consumption increases
Solution Approach 1:
The patent extracts and identifies the specific portions of code that have changed between versions using version control delta. By taking out only the changed code segments and running test routines exclusively on those segments, the system maintains testing thoroughness for affected areas while eliminating unnecessary resource consumption from testing unchanged code. This extraction approach directly addresses the contradiction between testing thoroughness and resource consumption.
3Reliability
If repetitive testing of all code lines is performed after each revision, then bug detection capability is improved, but productivity decreases
Solution Approach 1:
The patent performs preliminary identification of changed code portions using version control delta before executing test routines. This preliminary action allows the system to pre-determine which code segments require testing, enabling selective execution of test routines only on those segments. By performing this preliminary analysis, the system maintains bug detection capability for changed code while significantly improving productivity by avoiding repetitive testing of unchanged code.
Data Source
AI summary
A data processing system obtains a delta between a first version of a target program and a second version of the target program, wherein the target program has been tested by a plurality of test routines. The data processing system obtains test coverage data that has been generated for the plurality of test routines, the test coverage data for a test routine identifying lines of code of the target program that have been previously tested by said test routine. The data processing system ranks the plurality of test routines based on the test coverage data and based on the delta. The data processing system selects one of the plurality of test routines based on the ranking and executes the selected test routine on the second version of the target program to determine code stability of the second version of the target program.