Automated Error Injection for Autonomous Software Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing software testing techniques for error handling in complex software programs are inefficient, often requiring manual stub creation and injection at each function call, which becomes labor-intensive as code complexity increases.
Innovation Solution
A software testing tool that performs information gathering to identify function calls and return errors, followed by error-injection tests to simulate return errors and assess error handling capabilities, allowing for comprehensive testing with reduced manual effort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual stub creation and injection is performed at each function call, then error handling testing can be conducted, but labor intensity increases significantly as code complexity increases
Solution Approach 1:
The testing tool automatically performs information gathering to identify function calls and their return errors, then generates test cases autonomously without requiring manual stub creation. The system serves itself by extracting test information directly from the codebase through automated analysis of function signatures and error return types.
Solution Approach 2:
The manual mechanical process of creating and injecting stubs at each function call is replaced by an automated software system that uses information gathering techniques to extract function call patterns and generates corresponding test cases programmatically, eliminating manual code modification.
2Reliability
If stubs are injected at each function call, then error handling can be tested, but maintaining stubs becomes difficult when code is modified
Solution Approach 1:
Instead of modifying actual stubs in the codebase, the system creates virtual copies or representations of function calls through automated information gathering. The test cases are generated as independent artifacts that capture the essential error handling scenarios without being physically embedded in the production code, making them immune to code changes.
Solution Approach 2:
The system performs preliminary information gathering to extract function call patterns and error return types before generating test cases. This upfront extraction creates a snapshot of the codebase's error handling structure that can be used to generate comprehensive tests without requiring subsequent maintenance when the code evolves.
3Productivity
If automated test generation is used, then productivity increases, but comprehensive testing is not achieved without systematic identification of function calls and return errors
Solution Approach 1:
The system performs preliminary information gathering that systematically analyzes the codebase to identify all function calls and their potential return errors before test generation. This preliminary extraction ensures that the automated test generation has complete and accurate information about the codebase's error handling patterns, enabling comprehensive testing coverage.
Solution Approach 2:
The information gathering process continuously analyzes the codebase structure, function signatures, and error return types to feed back into test case generation. This feedback loop ensures that the generated tests accurately reflect the actual error handling patterns in the code, maintaining comprehensive coverage while improving productivity.
Data Source
AI summary
One or more embodiments relate to executing a software testing tool to identify function calls—internal and/or external—of software code and their corresponding errors. Once identified—such as during an information gathering operation—the error codes may be returned in place of actual outputs of the function during testing, and the downstream processing of the software as a result of the errors may be evaluated. As such, an automatic software testing tool may be implemented that not only identifies functions calls and corresponding errors, but also evaluates performance of the software in view of the various different error types associated with the function calls.