Test Case Selection Device for Software Change Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for selecting test cases after a program change are inefficient, as they often include refactoring changes that do not affect functionality, leading to unnecessary test cases and potential oversight of necessary test cases due to unchanged output ranges.
Innovation Solution
A test case selection apparatus that extracts non-equivalence sets by comparing before-change and after-change functions, selecting test cases where the output differs, thereby identifying important test cases for re-execution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all test cases from before-change program are performed on after-change program, then reliability of defect detection is improved, but test person-hours and workload are increased
Solution Approach 1:
The patent extracts only the necessary test cases from the complete test suite by analyzing program differences. It identifies test cases that specifically cover changed instructions and selects only those test cases for re-execution, rather than running all test cases. This extraction approach maintains defect detection reliability while significantly reducing test person-hours.
Solution Approach 2:
The patent segments the test case selection process into multiple stages: (1) detecting program differences between before-change and after-change versions, (2) identifying changed instructions, (3) determining affected test cases, and (4) selecting final test cases for execution. This segmentation allows systematic identification of essential test cases without executing the entire test suite.
2Extent of automation
If test cases are selected based on source code difference detection, then test case selection automation is improved, but refactoring changes are incorrectly identified as functional changes leading to unnecessary test cases
Solution Approach 1:
The patent applies local quality analysis by examining the specific impact of each changed instruction on program behavior. Instead of treating all source code differences equally, it analyzes whether each difference actually affects functional output. This allows automated distinction between refactoring changes (no functional impact) and functional changes (affect output), reducing unnecessary test cases while maintaining automation.
Solution Approach 2:
The patent introduces dynamic analysis by executing test cases with modified input data and comparing outputs between before-change and after-change programs. This dynamic verification determines whether a changed instruction actually produces different output, allowing automated filtering of refactoring changes from functional changes, thus reducing the number of unnecessary test cases.
3Measurement precision
If symbolic execution is performed to detect output range changes, then measurement precision of output impact is improved, but test cases with unchanged output ranges are overlooked even when functionality changes
Solution Approach 1:
The patent merges multiple analysis approaches: (1) static source code difference detection to identify changed instructions, (2) symbolic execution to analyze output range changes, and (3) dynamic test execution with output comparison. This combination ensures that both output-range-changing changes and output-range-preserving functional changes are detected, preventing overlooked test cases while maintaining measurement precision.
Solution Approach 2:
The patent implements feedback mechanisms where test execution results are compared between before-change and after-change programs. When output differences are detected, the corresponding test cases are identified for selection. This feedback loop ensures that functional changes not detectable through output range analysis alone are still captured, improving test case selection accuracy.
4Manufacturing precision
If manual resetting of expected output values is performed in each test case, then test accuracy is improved, but test preparation time and workload are increased
Solution Approach 1:
The patent performs preliminary automated analysis to pre-determine which test cases require re-execution and what their expected outputs should be. By analyzing program differences and tracing their impact on test cases before actual test execution, the system prepares expected output values in advance, eliminating the need for manual resetting during test preparation while maintaining test accuracy.
Solution Approach 2:
The patent implements self-service automation where the system automatically determines expected output values for selected test cases by comparing before-change and after-change program executions. This automated self-determination of expected values eliminates manual intervention, reducing test preparation time while maintaining or improving test accuracy through consistent automated comparison.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A non-equivalence set extraction unit (120) extracts, from a target set list, a non-equivalence set being a target set that does not satisfy a condition that an output of an after-change target function is coincident with an output of a before-change target function when a same input is provided to both of the before-change target function and the after-change target function. A test case selection unit (140) selects, from a plurality of test cases, a test case wherein the before-change target function included in the non-equivalence set is called, as an important test case.