Iterative Test Double Refinement for Software Coverage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Test doubles in software testing can reduce coverage of the function under test and create false paths, making it difficult to pinpoint errors and slowing down the testing process.
Innovation Solution
A method to generate test double proxies with initial abstract stubs for callee functions, refining them into more specific stubs based on improving test coverage, allowing for better isolation and verification of the function under test without modifying the source code.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If test doubles are used in place of callee functions, then the complexity of testing the particular function is reduced, but the coverage of the particular function is reduced and false paths are created
Solution Approach 1:
The patent segments the test double generation process into multiple refinement stages. Instead of using a single abstract test double, the system iteratively refines test doubles through multiple generations, where each generation adds more specific behavior. This segmentation allows the system to maintain simplicity in the testing framework while progressively improving test coverage through controlled refinement of test double behavior.
Solution Approach 2:
The patent implements dynamic test double refinement where test doubles evolve from abstract to concrete behavior based on test results. The system dynamically adjusts the level of abstraction in test doubles during the testing process, starting with highly abstract test doubles that reduce complexity, then progressively refining them to improve coverage without permanently increasing testing complexity.
2Ease of operation
If abstract test stubs are used for all callee functions, then the testing process is simplified, but the test coverage is reduced
Solution Approach 1:
The patent applies preliminary action by generating initial abstract test stubs for all callee functions before actual testing begins. These abstract stubs provide a simplified starting point that makes the testing process easier to set up and execute. The system then performs preliminary refinement iterations to progressively improve coverage while maintaining the simplified testing approach.
Solution Approach 2:
The patent implements feedback mechanisms where test results from abstract test stubs are analyzed to determine which stubs need refinement. The system uses coverage information as feedback to iteratively refine specific test stubs from abstract to more concrete implementations, thereby improving test coverage while maintaining simplicity for stubs that don't require refinement.
3Reliability
If test doubles are used to isolate the function under test, then the isolation is improved, but the ability to pinpoint errors is reduced due to false paths
Solution Approach 1:
The patent applies local quality by making different test doubles have different levels of abstraction based on their specific purpose and the requirements of the function under test. Rather than uniformly abstracting all test doubles, the system refines individual test doubles locally to maintain isolation where needed while preserving error detection capability in critical paths. This localized approach to test double quality allows simultaneous achievement of isolation and error detection.
Data Source
AI summary
A method to generate test double proxies for callee functions of a function under test may include generating an initial set of test double proxies with abstract test stubs for all callee functions called by the function under test. Each of the test double proxies in the initial set of test double proxies may correspond to a different one of the callee functions. The method may also include generating a first refined set of test double proxies that includes a first refined test stub instead of a first one of the abstract test stubs for a first test double proxy in the initial set of test double proxies in response to determining that refining the first one of the abstract test stubs improves a test coverage of the function under test.