Predictive Fault Injection System for Distributed Software
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing Software Implemented Fault Injection (SWIFI) systems face challenges in precisely identifying the state of a distributed system for fault injection, resulting in significant latency and a 'blind spot' where faults cannot be injected, particularly in large-scale cloud platforms.
Innovation Solution
A fault injection system that uses a predictive time model to estimate when states and transitions occur in a distributed system, allowing for precise fault injection without the latency issues of traditional event-based approaches, utilizing distributed tracing technology and temporal estimation to identify optimal injection times.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional event-based SWIFI systems are used to inject faults, then fault injection can be performed, but significant latency occurs between state identification and actual fault injection
Solution Approach 1:
The system pre-calculates and stores the expected duration of each state in a lookup table during system initialization or offline analysis. When a state is identified for fault injection, the pre-computed duration is immediately retrieved and used to schedule the fault injection timing, eliminating the need for real-time state monitoring and calculation.
Solution Approach 2:
The patent introduces a timing buffer mechanism that accounts for the latency between state identification and fault injection. By predicting state durations in advance and building in timing margins, the system ensures faults are injected at the correct moment despite the inherent delay in the injection process.
2Adaptability or versatility
If event-based approaches are used for fault injection, then faults can be injected into distributed systems, but blind spots are created during which no faults can be injected
Solution Approach 1:
The system pre-determines all possible state durations and stores them in advance. This allows the fault injection mechanism to operate continuously by scheduling injections based on pre-computed timing information, eliminating gaps where no faults can be injected.
Solution Approach 2:
By using pre-calculated state durations to continuously schedule fault injections, the system maintains uninterrupted fault injection capability. The timing buffer mechanism ensures that injections are always scheduled at appropriate moments, eliminating blind spots and maintaining continuous test coverage.
3Measurement precision
If real-time event monitoring is implemented to identify states for fault injection, then precise state identification is achieved, but system complexity increases
Solution Approach 1:
The system performs state duration analysis offline or during initialization, storing results in lookup tables. At runtime, only simple table lookups are required based on current state identification, dramatically reducing the computational complexity while maintaining precise state identification.
Solution Approach 2:
The patent creates a simplified temporal model that mirrors the actual system states with pre-computed durations. This model serves as a lightweight proxy that captures essential timing information without requiring complex real-time analysis of the actual distributed system.
Data Source
AI summary
A fault injection system for a software system, wherein the software system includes executable commands, events generated by commands and states representing a state of a state machine between two events, wherein a fault injection rule database is configured to provide a rule for injecting a selected fault, wherein the rule provides a fault injection decision in the occurrence of a selected command, a selected state during which fault injection is to occur and a condition under which fault injection is to occur; an estimator module is configured to determine when the selected state will occur related to the execution of the selected command and to provide a duration of a state as a time interval for fault injection, wherein the determination is based on a predictive time model; and an injector module is configured to execute the selected command and to inject the selected fault during the time interval.