Test Case Validation via Production Code Usage Comparison
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software test cases often exercise more functional code units than are actually used in a productive environment, leading to longer test runs and inefficiency.
Innovation Solution
A method to evaluate test cases by comparing the code units covered in a design environment with those used in a productive environment, using usage information or a bill of materials to identify necessary and unnecessary tests, and adjusting test cases to focus on frequently used logical code units.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If test cases are designed to cover all logical code units in the design environment, then test coverage is improved, but test duration increases
Solution Approach 1:
The system performs preliminary analysis by comparing design environment code units with productive environment usage information before executing tests. This pre-comparison identifies which code units are actually used in production, allowing the test suite to be pre-filtered to only include relevant tests, thus reducing test duration while maintaining necessary coverage
Solution Approach 2:
Instead of executing all possible test cases that cover every logical code unit in the design environment, the system applies partial action by selectively executing only those test cases that correspond to code units actually used in the productive environment. This eliminates excessive testing of unused code units while maintaining adequate coverage of critical paths
2Reliability
If test cases exercise more functional code units than actually used in productive environment, then test completeness is improved, but testing efficiency deteriorates
Solution Approach 1:
The system applies local quality by differentiating between code units based on their actual usage in the productive environment. Test cases are selectively applied only to code units that are locally relevant (actually used in production), rather than uniformly testing all code units. This creates a customized test strategy that maintains completeness for critical areas while improving efficiency by excluding irrelevant tests
3Reliability
If all logical code units are tested in design environment, then code unit coverage is improved, but relevance to productive environment decreases
Solution Approach 1:
The system changes the parameter of test selection from static (all code units in design environment) to dynamic (code units filtered by productive environment usage). By using usage information from the productive environment as a filtering parameter, the test suite adapts to reflect actual production conditions, improving relevance while maintaining necessary coverage through the comparison process
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
A system and method includes obtaining and storing, on a computer readable storage device, a list of a set of test cases designed to test logical code units in multiple objects of a software program in a design environment, identifying a first set logical code units covered by the test cases, identifying a second set of logical code units corresponding to a productive environment, and comparing, via a computer, the first and second sets of logical code units to generate an evaluation of the test cases for the software program in the productive environment.