Distributed Worker Coordinator Selection for Fault-Tolerant Task Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed computing systems face challenges in maintaining fault-tolerance and scalability, particularly in environments where resources are dynamic and prone to failure or changes in load demand, leading to potential data corruption or the need to restart computational projects.
Innovation Solution
A distributed computing system design that utilizes worker coordinators and workers, with self-organization capabilities, strongly-consistent data stores, and redundancy mechanisms to ensure fault-tolerance and scalability, allowing tasks to be reassigned and completed even if worker or coordinator hosts fail, and accommodating varying resource availability and demand.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If distributed computing systems use dynamic resources and parallel task execution, then productivity and scalability are improved, but fault-tolerance deteriorates due to potential data corruption and system failures
Solution Approach 1:
The system performs preliminary actions by creating checkpoint records and task assignment logs before computational tasks are executed. These preliminary records are stored in the data store, enabling recovery operations to be performed if failures occur during task execution, thus maintaining fault-tolerance while allowing dynamic resource allocation and parallel execution.
Solution Approach 2:
The system creates copies of task assignment information and computational state in the data store. When failures occur, the system can retrieve these copies and reassign tasks to different worker processes or hosts, ensuring that computational productivity is maintained without data loss or corruption.
2Stability of the object's composition
If the system maintains strong consistency across distributed hosts, then data integrity is improved, but device complexity increases due to coordination overhead
Solution Approach 1:
The patent introduces a data store as an intermediary component that mediates between worker processes and coordinators. This data store maintains task assignment records and computational state in a centralized manner, allowing multiple hosts to access consistent data without complex peer-to-peer coordination protocols, thus achieving strong consistency while managing system complexity.
Solution Approach 2:
The system extracts the complexity of maintaining consistency by separating it into a dedicated data store component. This extraction allows the computational workers and coordinators to focus on their primary functions while the data store handles consistency maintenance through its checkpoint and recovery mechanisms.
3Reliability
If the system implements fault-tolerance mechanisms and redundancy, then reliability is improved, but loss of time increases due to coordination and recovery overhead
Solution Approach 1:
The system performs preliminary actions by continuously maintaining checkpoint records and task assignment logs in the data store during normal operation. When failures occur, these pre-existing records enable immediate recovery operations without requiring time-consuming data reconstruction or task re-planning, thus reducing recovery time while maintaining fault-tolerance.
Data Source
AI summary
A distributed work processing system for processing computational tasks is scalable and fault-tolerant without requiring centralized control. Worker processes running on worker hosts are organized into a logical group and worker coordinators running on worker coordinator hosts coordinate tasks assigned to worker processes. A task store might hold a collection of tasks to be performed by the logical group. A lock database can be used for locking the logical group for coordination by one worker coordinator process at a time. A membership store contains mappings of worker processes to logical groups, and an assignment store indicates which tasks are assigned to which workers. The worker coordinator process has a scanner process to deal with unassigned tasks and deduplicating duplicate assignments. If a worker coordinator does not see enough worker processes, it can instantiate more. If a worker process does not see a worker coordinator, it can instantiate one.