Dynamic Process Recovery for Distributed Event-Driven Services
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional fault tolerance mechanisms, such as heartbeat communication, are insufficient for recovering event-driven services as they fail to automatically restart services from a last known good state after process failure or controlled shutdown in distributed environments.
Innovation Solution
A method for dynamic process recovery that detects process failures, releases resources, and restarts services from a last known good state in a secondary process, including storing heartbeat information, scheduling database agent jobs, and providing restart messages for services, while handling incorrect failure detection and controlled shutdown scenarios.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional heartbeat mechanism is used for process failure detection, then process failure detection capability is improved, but service recovery capability deteriorates
Solution Approach 1:
The system segments the monitoring and recovery functions into distinct components: heartbeat mechanism for failure detection, service registry for service information storage, and automatic recovery mechanism for service restoration. This segmentation allows each component to specialize in its function while working together to achieve comprehensive fault tolerance.
Solution Approach 2:
The patent introduces a service registry as an intermediary component that stores service information and coordinates between the heartbeat detection mechanism and the recovery mechanism. The registry acts as a mediator that enables the system to not only detect failures but also automatically recover services by providing the necessary service configuration information.
2Duration of action of stationary object
If process instance is restarted after failure, then process continuity is improved, but service state consistency deteriorates
Solution Approach 1:
The system performs preliminary actions by maintaining service state information in the service registry before failures occur. When a process fails and is restarted, the recovery mechanism uses the pre-stored service information from the registry to restore services to their correct state, ensuring consistency without requiring complex state synchronization during recovery.
3Reliability
If services are automatically restarted in secondary process, then fault tolerance is improved, but system complexity deteriorates
Solution Approach 1:
The service registry serves multiple functions: storing service information, coordinating failure detection, enabling automatic recovery, and providing service configuration to secondary processes. This multi-functionality reduces the need for separate specialized components, thereby limiting the increase in system complexity while achieving comprehensive fault tolerance.
Data Source
AI summary
Methods for providing dynamic process recovery that supports recovery of event driven services. The present invention addresses situations where a service-hosting process fails as well as all of the services which were running in that process. To recover from this scenario, the present invention restarts the services from a last known good state. The present invention also provides for situations where a process is shut down in a controlled manner, but the services running within it are still able to drive. Here again, the services are restarted in a secondary process. The recovery modal described here, therefore, advantageously addresses a problem in the design of a distributed event-driven environment.