Cluster Task Allocation Using Node Performance Feedback
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing cluster computing systems struggle to efficiently distribute tasks and manage resources in heterogeneous environments, leading to reduced performance and instability.
Innovation Solution
A cluster computing system that includes a host with a task allocator to monitor node performance and distribute tasks based on availability, utilizing a network to connect computing nodes with storage devices, and employing buffer memories for efficient data processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If tasks are distributed in a heterogeneous cluster environment without monitoring node performance, then task distribution can be simplified, but system performance and stability deteriorate
Solution Approach 1:
The host monitors the performance state of each computing node and uses this feedback information to dynamically adjust task distribution decisions, ensuring optimal system performance and stability in heterogeneous environments
2Device complexity
If the host processes all background tasks itself, then task management is simplified, but host performance and responsiveness deteriorate
Solution Approach 1:
The system divides task management responsibilities by separating foreground tasks (processed by the host) from background tasks (offloaded to computing nodes), allowing the host to maintain high responsiveness while distributing workload
Solution Approach 2:
Background tasks are extracted from the host and transferred to computing nodes for execution, relieving the host of non-critical workload while maintaining centralized control
3Loss of time
If computing nodes with poor performance are selected for background tasks, then task allocation is faster, but overall system performance deteriorates
Solution Approach 1:
The system continuously monitors computing node performance states and uses this real-time feedback to make informed task allocation decisions, selecting appropriate nodes based on current capacity and performance metrics
4Speed
If data is stored only in buffer memory without distributed storage, then access speed is improved, but system capacity and reliability deteriorate
Solution Approach 1:
The storage system is segmented into buffer memory for fast access and distributed storage devices for reliable persistence, with each layer serving its optimal function
Solution Approach 2:
The system uses computing nodes as intermediaries between buffer memory and storage devices, coordinating data transfer and ensuring data integrity across the storage hierarchy
Data Source
AI summary
A cluster computing system is provided. The cluster computing system includes: a host including a first processor and a first buffer memory; computing nodes, each of which includes a second processor and a second buffer memory configured to store data received from the host; a network configured to connect the host and the computing nodes; and storage devices respectively corresponding to the computing nodes. The first processor is configured to control a task allocator to monitor a task performance state of each of the computing nodes, select at least one of the computing nodes as a task node, and distribute a background task to the task node, and the second processor of the task node is configured to perform the background task on sorted files stored in the second buffer memory, the sorted files being received by the second buffer memory from the first buffer memory.