Software Replanning for Failure Recovery in Distributed Processing Graphs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In compositional applications, especially those involving distributed web services, components can fail due to network failures, denial of service attacks, or software failures, leading to complex event handling challenges as no single component has access to the necessary interfaces or information to handle events correctly, and existing automated planning systems struggle to recover from such failures without disrupting the application's semantics.
Innovation Solution
A method is introduced that involves receiving failure information to modify the planning domain, composing a new processing graph excluding the failing component, generating a recovery plan, and deploying the new graph to ensure continuous operation, using Stream Processing Planning Language (SPPL) to express planning tasks and handle exceptions by replanning and reconfiguring the application.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If automated planning systems are used to compose processing graphs, then application development time is reduced, but the systems struggle to recover from component failures without disrupting application semantics
Solution Approach 1:
The system performs preliminary actions by pre-defining alternative processing graphs and recovery plans during the planning phase. When a component failure occurs, the system can immediately switch to pre-planned alternatives without disrupting application semantics, thus resolving the contradiction between rapid development and reliable failure recovery.
Solution Approach 2:
The planning system acts as an intermediary that monitors component failures and orchestrates the transition to alternative processing graphs. This intermediary layer handles the complexity of failure recovery, allowing the application to maintain semantics while automatically recovering from failures, thus improving reliability without sacrificing development efficiency.
2Ease of operation
If dynamic binding of components is used to enable shorter application development cycles, then ease of operation is improved, but complexity of event handling increases when components fail
Solution Approach 1:
The system segments event handling by creating separate, modular processing graphs for different failure scenarios. Each graph handles specific failure conditions independently, reducing the overall complexity of event handling while maintaining dynamic binding capabilities that enable short development cycles.
Solution Approach 2:
The system changes parameters by dynamically selecting different processing graphs based on failure conditions. This parameter-based approach allows the system to maintain simple dynamic binding for normal operation while automatically switching to specialized recovery graphs when failures occur, thus managing event handling complexity without sacrificing ease of operation.
3Adaptability or versatility
If no single component has access to necessary interfaces to handle events correctly, then system modularity is improved, but ability to handle failures correctly deteriorates
Solution Approach 1:
The planning system provides a universal, multi-functional solution by maintaining multiple alternative processing graphs that can handle different failure scenarios. This universal approach allows any component failure to be handled correctly through the appropriate alternative graph, preserving system modularity while improving event handling correctness through centralized planning capabilities.
Data Source
AI summary
A method for recovering from software failures, includes: receiving failure information that identifies a failing component of a first processing graph; modifying a planning domain that includes a plurality of component descriptions according to the failure information; and composing a second processing graph by using the modified planning domain so that the second processing graph does not include the failing component.