Mutation Analysis for Selective Regression Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for testing computer program revisions are time-consuming and inefficient, often requiring unnecessary tests that do not effectively detect errors introduced by changes in the code, particularly in modern test environments like circuit validation.
Innovation Solution
A method that determines and compares mutations in the source code of a first and second version of a computer program to classify changes, using mutation analysis to identify which tests are necessary to detect errors, allowing for selective regression testing and reducing the testing effort by focusing on changed or new mutations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If complete testing is performed for every version revision, then reliability of error detection is improved, but testing time and effort increase significantly
Solution Approach 1:
The patent segments the testing process by identifying and isolating only the mutated code sections (changed units) from the complete program. Instead of testing the entire program, the system segments the test suite to execute only tests relevant to the mutated sections, thereby reducing testing time while maintaining reliability for detecting errors in changed areas.
Solution Approach 2:
The patent applies local quality by focusing testing resources specifically on the mutated sections of the code. The system determines which tests are relevant to the changed units and executes only those tests, rather than applying uniform testing to the entire program. This localized approach maintains high detection reliability for errors in modified areas while significantly reducing overall testing time.
2Loss of time
If selective testing based on coverage information is used, then testing effort is reduced, but the ability to detect errors in changed code is compromised
Solution Approach 1:
The patent employs feedback mechanisms where the system uses mutation analysis results to dynamically determine which tests should be executed. The feedback loop involves: (1) inserting mutations into the program, (2) analyzing which tests would detect these mutations, (3) selecting tests based on this analysis, and (4) executing only those selected tests. This feedback-driven approach ensures that testing effort is reduced while maintaining reliable error detection capability.
Solution Approach 2:
The patent applies preliminary action by performing mutation analysis and test selection before actually executing the tests. The system preliminarily determines which tests are relevant to the mutated sections by analyzing the program structure and mutation locations, then executes only those pre-selected tests. This preliminary filtering action reduces testing effort while ensuring that the selected tests maintain adequate error detection capability.
3Loss of time
If tests are selected based on coverage results from the first version, then testing time is reduced, but the tests may not effectively detect errors in the second version
Solution Approach 1:
The patent applies parameter changes by adapting the test selection criteria from the first version to the second version through mutation analysis. Instead of using static coverage results from the original version, the system dynamically determines test relevance based on the actual mutations introduced in the second version. This parameter change approach ensures that testing time is reduced while maintaining test effectiveness for detecting errors in the revised code.
Solution Approach 2:
The patent implements dynamics by making the test selection process adaptive and flexible rather than static. The system dynamically analyzes the program structure and mutation locations to determine which tests should be executed in each version. This dynamic approach allows the testing process to adapt to code changes while maintaining reduced testing time and effective error detection capability.
Data Source
AI summary
A method for testing a circuit specification after changing a first version of the circuit specification into a second version of the circuit specification due to a revision of the circuit specification includes receiving a first set of mutations that can be or have been inserted into the first version of the circuit specification and a second set of mutations that can be inserted into the second version of the circuit specification computer program. Changed and unchanged mutations are identified in the first set of mutations and in the second set of mutations based on a comparison between the second version of the circuit specification and against the first version of the circuit specification. Information configured to test the second version of the circuit specification is generated using at least a portion of the identified mutations classified as the changed mutations.