Mutation Analysis Test Evaluation Prioritization

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Mutation analysis for evaluating computer program tests is time-consuming and resource-intensive, often requiring a large number of mutations to be tested, which impairs the quality of the test evaluation.

Innovation Solution

Selecting a suite of relevant tests for each mutation and determining a ranking system based on simplicity criteria to prioritize the execution of mutated programs, using a programmable circuit with logic sets that can be activated or deactivated to simulate both non-mutated and mutated programs, allowing for efficient identification of detected mutations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If mutation analysis is performed with a large number of mutations to improve test evaluation quality, then the quality of test evaluation is improved, but the time required for evaluation increases significantly

Engineering Contradiction:
Improvetest evaluation qualityVSAvoidevaluation time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-selecting a subset of relevant mutations before performing full mutation analysis. The system identifies and prioritizes mutations that are most likely to reveal test deficiencies, allowing evaluation to focus on the most critical cases first. This preliminary selection process reduces the overall evaluation time while maintaining high test evaluation quality by concentrating resources on the most informative mutations.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent implements skipping by allowing the evaluation process to quickly pass through mutations that are unlikely to provide valuable information. The system uses heuristics and preliminary analysis to identify and skip redundant or low-value mutations, rushing through the evaluation of obvious cases while dedicating more time to complex, informative mutations. This approach significantly reduces total evaluation time without compromising the quality of test assessment.

Inventive Principle:
Principle #21Skipping (Rushing through)

2Loss of time

If the number of mutations is reduced to decrease evaluation time, then the evaluation time is reduced, but the quality of test evaluation is impaired

Engineering Contradiction:
Improveevaluation timeVSAvoidtest evaluation quality
Core Design Contradiction:
Loss of timeVSMeasurement precision

Solution Approach 1:

The patent applies local quality by treating different mutations differently based on their characteristics and potential value. Instead of uniformly reducing the number of mutations, the system identifies specific mutations with high evaluation value and focuses computational resources on those. Each mutation is assessed individually for its potential to reveal test deficiencies, and the evaluation depth is adjusted locally for each mutation based on its importance, complexity, and likelihood of exposing test weaknesses.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent uses parameter changes by dynamically adjusting evaluation parameters such as the number of mutations tested, the depth of analysis for each mutation, and the selection criteria based on intermediate results. The system monitors evaluation progress and adapts parameters in real-time, increasing focus on promising mutation cases while reducing effort on less informative ones. This dynamic parameter adjustment maintains high evaluation quality while optimizing the time investment across the mutation set.

Inventive Principle:
Principle #35Parameter changes

3Loss of time

If a predetermined number of mutations are tested to reduce evaluation time, then the evaluation time is reduced, but the detection of subtle mutations is impaired

Engineering Contradiction:
Improveevaluation timeVSAvoidmutation detection capability
Core Design Contradiction:
Loss of timeVSReliability

Solution Approach 1:

The patent applies dynamics by making the mutation testing process adaptive rather than static. The system dynamically adjusts which mutations are tested and how deeply they are analyzed based on feedback from intermediate evaluation results. As the evaluation progresses, the system learns from detected patterns and modifies its testing strategy, prioritizing mutations that are more likely to be subtle or previously undetected. This dynamic approach ensures that limited testing resources are directed toward mutations that will most improve detection reliability.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent implements feedback mechanisms where results from initial mutation tests inform subsequent testing decisions. The system analyzes patterns in detected versus non-detected mutations and uses this feedback to refine its selection criteria for remaining mutations. Feedback loops allow the system to identify subtle mutation patterns that initial testing revealed, then specifically target similar mutations in subsequent evaluation phases. This iterative feedback process maintains high mutation detection capability even with reduced overall testing scope.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS7574681B2Method and system for evaluating computer program tests by means of mutation analysis
Publication Date: 2009.08.11 SYNOPSYS INC
  • US7574681B2 patent drawing
  • US7574681B2 patent drawing
  • US7574681B2 patent drawing

AI summary

Methods and systems for evaluating computer program tests by mutation analysis, including the execution of mutated programs with the insertion of mutations and the identification of mutated programs which, with a pre-determined test, provide a result identical to a pre-determined result. A series of relevant tests is selected for each mutation from a plurality of tests. A mutation ranking system can be determined with the application of at least a first simplicity criterion to the selected series of tests. The mutated programs are subsequently executed according to the mutation ranking order. Each mutated program is executed with the tests from the associated series of tests. In addition, the method can comprise the programming and activation of a programmable circuit. A ranking system for the tests from a series of tests can be determined with the application of a second simplicity criterion to the tests from said series of tests.