Mutation Analysis Test Evaluation Prioritization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mutation analysis for evaluating computer program tests is time-consuming and resource-intensive, often requiring a large number of mutations to be tested, which impairs the quality of the test evaluation.
Innovation Solution
Selecting a suite of relevant tests for each mutation and determining a ranking system based on simplicity criteria to prioritize the execution of mutated programs, using a programmable circuit with logic sets that can be activated or deactivated to simulate both non-mutated and mutated programs, allowing for efficient identification of detected mutations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If mutation analysis is performed with a large number of mutations to improve test evaluation quality, then the quality of test evaluation is improved, but the time required for evaluation increases significantly
Solution Approach 1:
The patent applies preliminary action by pre-selecting a subset of relevant mutations before performing full mutation analysis. The system identifies and prioritizes mutations that are most likely to reveal test deficiencies, allowing evaluation to focus on the most critical cases first. This preliminary selection process reduces the overall evaluation time while maintaining high test evaluation quality by concentrating resources on the most informative mutations.
Solution Approach 2:
The patent implements skipping by allowing the evaluation process to quickly pass through mutations that are unlikely to provide valuable information. The system uses heuristics and preliminary analysis to identify and skip redundant or low-value mutations, rushing through the evaluation of obvious cases while dedicating more time to complex, informative mutations. This approach significantly reduces total evaluation time without compromising the quality of test assessment.
2Loss of time
If the number of mutations is reduced to decrease evaluation time, then the evaluation time is reduced, but the quality of test evaluation is impaired
Solution Approach 1:
The patent applies local quality by treating different mutations differently based on their characteristics and potential value. Instead of uniformly reducing the number of mutations, the system identifies specific mutations with high evaluation value and focuses computational resources on those. Each mutation is assessed individually for its potential to reveal test deficiencies, and the evaluation depth is adjusted locally for each mutation based on its importance, complexity, and likelihood of exposing test weaknesses.
Solution Approach 2:
The patent uses parameter changes by dynamically adjusting evaluation parameters such as the number of mutations tested, the depth of analysis for each mutation, and the selection criteria based on intermediate results. The system monitors evaluation progress and adapts parameters in real-time, increasing focus on promising mutation cases while reducing effort on less informative ones. This dynamic parameter adjustment maintains high evaluation quality while optimizing the time investment across the mutation set.
3Loss of time
If a predetermined number of mutations are tested to reduce evaluation time, then the evaluation time is reduced, but the detection of subtle mutations is impaired
Solution Approach 1:
The patent applies dynamics by making the mutation testing process adaptive rather than static. The system dynamically adjusts which mutations are tested and how deeply they are analyzed based on feedback from intermediate evaluation results. As the evaluation progresses, the system learns from detected patterns and modifies its testing strategy, prioritizing mutations that are more likely to be subtle or previously undetected. This dynamic approach ensures that limited testing resources are directed toward mutations that will most improve detection reliability.
Solution Approach 2:
The patent implements feedback mechanisms where results from initial mutation tests inform subsequent testing decisions. The system analyzes patterns in detected versus non-detected mutations and uses this feedback to refine its selection criteria for remaining mutations. Feedback loops allow the system to identify subtle mutation patterns that initial testing revealed, then specifically target similar mutations in subsequent evaluation phases. This iterative feedback process maintains high mutation detection capability even with reduced overall testing scope.
Data Source
AI summary
Methods and systems for evaluating computer program tests by mutation analysis, including the execution of mutated programs with the insertion of mutations and the identification of mutated programs which, with a pre-determined test, provide a result identical to a pre-determined result. A series of relevant tests is selected for each mutation from a plurality of tests. A mutation ranking system can be determined with the application of at least a first simplicity criterion to the selected series of tests. The mutated programs are subsequently executed according to the mutation ranking order. Each mutated program is executed with the tests from the associated series of tests. In addition, the method can comprise the programming and activation of a programmable circuit. A ranking system for the tests from a series of tests can be determined with the application of a second simplicity criterion to the tests from said series of tests.


