Embedded Fault Injection Instruction Sets for Software Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Software testing, particularly exception path testing, is challenging due to complications in reproducing exception paths, which often require code modification and manual intervention, leading to difficult test script creation and maintenance, with testers lacking intimate knowledge of the technology and code, and developers lacking tools for automation.
Innovation Solution
A software manipulation framework that allows developers to embed instruction sets for simulating faults directly into software products, enabling test engineers to extract and compile these instructions into scripts, which are executed at runtime to induce specific error paths, reducing the need for manual script writing and maintaining fault injection routines.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual fault injection is used to test exception paths, then testing coverage is improved, but test script creation and maintenance complexity increases
Solution Approach 1:
The system enables self-service fault injection by embedding instruction sets directly in the software code that can be automatically extracted and executed. Testers don't need to manually create complex injection scripts - the system automatically locates and executes the embedded instructions at specified runtime points, eliminating the need for complicated test script maintenance while maintaining comprehensive exception path coverage
Solution Approach 2:
Embedded instruction sets act as intermediaries between the testing framework and the software code. These instructions are embedded within the software during compilation but remain invisible during normal execution. When testing is needed, the framework extracts and executes these embedded instructions to inject faults, serving as a mediator that simplifies the testing process while maintaining precise control over exception path execution
2Measurement precision
If code modification is performed to induce faults, then fault simulation accuracy is improved, but software recompilation is required
Solution Approach 1:
Fault injection instructions are embedded in the software code during the original compilation process, before testing begins. The instructions are prepared in advance and stored within the binary, so when testing is needed, the system simply extracts and executes these pre-prepared instructions without requiring any code modification or recompilation, thus maintaining high fault simulation accuracy while eliminating time loss
Solution Approach 2:
The system creates a copy of the fault injection functionality by embedding instruction sets that replicate the effect of code modification. Instead of actually modifying the source code and recompiling, the embedded instructions create a virtual copy of the modified state at runtime, achieving the same fault simulation accuracy without the overhead of recompilation
3Productivity
If embedded instruction sets are executed at runtime, then fault simulation efficiency is improved, but performance impact may occur
Solution Approach 1:
The system implements partial action by only extracting and executing embedded fault injection instructions when explicitly requested for testing purposes. During normal operation, these embedded instructions remain dormant and are not executed, minimizing performance impact. When testing is needed, only the specific instructions relevant to the test case are extracted and executed, not the entire instruction set, thus improving fault simulation efficiency while controlling performance overhead
Solution Approach 2:
The embedded instruction sets are designed with local quality by making them context-specific and location-aware. Each instruction is embedded at a specific location in the code and only activated when the execution flow reaches that location during testing. This localized approach ensures that fault injection only occurs where and when needed, minimizing unnecessary performance impact while maintaining high simulation efficiency at the targeted locations
Data Source
AI summary
Systems and methods for allowing software developers and the like to embed various different types of code and/or data manipulations into a software product. The embedded code manipulation instructions may be accessed and executed by test engineers or other users to induce fault paths and the like in the software product at run-time for analyzing the software's response to the induced fault paths. For instance, a user application of a software manipulation framework may expose a list of all of the instruction sets to a test engineer or the like in conjunction with one or more testing procedures to be performed on the software. Unless accessed by the test engineer, all of the testing probes may be represented by noops or placeholders at runtime to limit performance impact on the software.