Task Assignment Coordination with Volatile Storage Partitioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing grid systems face inefficiencies in assigning tasks to node devices, leading to suboptimal use of processing, storage, and network bandwidth resources, particularly when multiple analysis routines are shared and performed concurrently.
Innovation Solution
An apparatus and method that derive task assignments for node devices by partitioning data sets and utilizing volatile storage to minimize data retrieval, where tasks are assigned based on node availability and completion times, with a coordinating device managing the assignment of tasks across multiple nodes to optimize resource utilization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If tasks are assigned to the next available node device using the classic approach, then node device processing resources are utilized, but storage and network bandwidth resources are used inefficiently
Solution Approach 1:
The system performs preliminary actions by having node devices store data partitions in volatile storage before tasks are assigned. When a node completes a task, the data partition remains in volatile storage, allowing the same node to immediately process the next task without requiring data retrieval from non-volatile storage or network transfer, thus resolving the contradiction between processing resource utilization and storage/network efficiency
Solution Approach 2:
The system maintains continuity of useful action by keeping data partitions in volatile storage of node devices after task completion. This allows for continuous processing of subsequent tasks on the same node without interruption for data retrieval, maximizing processing resource utilization while minimizing storage and network bandwidth consumption
2Adaptability or versatility
If multiple analysis routines share grid resources concurrently, then resource versatility is improved, but task assignment complexity increases
Solution Approach 1:
The system implements self-service by allowing node devices to autonomously manage their own volatile storage resources. Each node maintains its own data partitions and can immediately process assigned tasks without complex coordination with other nodes, simplifying the task assignment management while maintaining the ability to share grid resources concurrently across multiple analysis routines
3Reliability
If data is retrieved from non-volatile storage for each task, then data availability is ensured, but task completion time increases
Solution Approach 1:
The system performs preliminary action by pre-loading data partitions into volatile storage of node devices before tasks are executed. This preliminary placement of data in fast volatile storage ensures data availability for subsequent tasks while eliminating the time-consuming retrieval process from non-volatile storage, directly resolving the contradiction between data availability and task completion time
Data Source
AI summary
An apparatus includes a processor and storage to store instructions that cause the processor to perform operations including: receive an indication of completion of a first task with a first partition such that the first node device is available to assign to perform another task; delay assignment of performance of a second task on a second partition to the first node device for up to a predetermined period of time, in spite of readiness of the second task to be performed on the second partition and availability of the first node device; determine whether an indication of completion of the first task with the second partition such that the second node device is available to assign to perform another task is received within the predetermined period of time; and assign performance of the second task on the second partition to the second node device based on the determination.