Automated Failure Testing via Domain-Specific Language
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current failure testing methods for computing systems are time-intensive and require manual effort, often failing to identify potential failure scenarios or verify recovery from simulated failures, especially in distributed environments, due to the need for multi-team collaboration and specific technical knowledge.
Innovation Solution
A computer-implemented method and system that uses natural language inputs to define simulated failure scenarios, translating these inputs into executable commands to simulate failures in distributed computing systems, monitor outcomes, and compare them to expected results, generating alerts for discrepancies and enabling automated verification of application recovery.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual failure testing is performed using FMEA methodology with multi-team collaboration, then comprehensive failure scenarios can be identified, but the testing process becomes time-intensive and requires up to five days to complete
Solution Approach 1:
The system pre-generates failure scenarios using automated analysis of system architecture, dependencies, and historical data before actual testing begins. This preliminary generation of test cases eliminates the need for time-consuming manual scenario creation by multiple teams, while still ensuring comprehensive coverage of potential failure modes.
Solution Approach 2:
An automated intermediary system acts as a mediator between system components and testers, generating and executing failure scenarios automatically. This intermediary translates system specifications into comprehensive test cases without requiring direct human collaboration for each scenario, significantly reducing testing time while maintaining reliability.
2Adaptability or versatility
If manual failure testing is performed by personnel from multiple teams, then diverse expertise can be utilized, but the process requires extensive coordination and manual effort
Solution Approach 1:
The automated testing system incorporates multiple functions within a single platform: scenario generation, test execution, result analysis, and reporting. This universal system replaces the need for coordination between multiple specialized teams, as the automated system performs all these functions independently while still leveraging diverse system knowledge through automated analysis.
Solution Approach 2:
The system performs self-service by automatically generating failure scenarios based on system architecture analysis, executing tests, and producing reports without requiring manual intervention from multiple teams. The system serves itself by translating system specifications into comprehensive test cases and autonomously managing the entire testing process.
3Ease of operation
If developers perform FMEA testing independently, then testing can be distributed, but individual developers lack knowledge of conducting complete FMEA tests
Solution Approach 1:
An automated intermediary system provides developers with independent testing capability by handling the complex FMEA methodology execution. The system mediates between the developer's simple test initiation and the complex automated scenario generation and execution, eliminating the need for developers to possess specialized FMEA knowledge while enabling distributed independent testing.
Solution Approach 2:
The system replaces the mechanical process of manual FMEA execution with automated computational processes. Instead of requiring developers to manually perform complex failure mode analysis, the system uses automated algorithms to generate and execute comprehensive test scenarios, substituting human expert knowledge with automated intelligence that developers can easily invoke.
Data Source
AI summary
Certain aspects of the present disclosure provide techniques for executing simulated failure scenarios on a computing system using natural language inputs in a domain-specific language. Embodiments includes receiving a failure scenario definition, which may be a natural language input identifying target systems in a distributed computing system, properties, and an expected outcome of a simulated system failure. Commands directed to the identified target systems to simulate a system failure on the identified systems according to the identified properties are generated and transmitted to the identified target systems for execution. The outcome of the simulated system failure is monitored, recorded, and compared to the expected outcome of the simulated system failure. Upon determining that the recorded outcome and expected outcome of the simulated system failure differ, an alert identifying a difference between the recorded outcome and the expected outcome is generated.