Dynamic I/O Job Redistribution via Job Bins
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage systems face challenges in redistributing I/O jobs among active OS threads efficiently, leading to timing out of I/O operations and resource consumption when threads become inactive or are reactivated, which impacts performance and requires new I/O requests.
Innovation Solution
The method involves creating job bins and bin groups on a processing node, distributing I/O jobs among these groups, and re-associating them with active threads upon changes in the number of active threads, allowing for dynamic redistribution of I/O jobs to maintain efficient execution without gaps in thread associations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If I/O jobs are statically assigned to OS threads, then thread association is simple to maintain, but I/O operations time out when threads become inactive and require new I/O requests
Solution Approach 1:
The patent introduces job bins as an intermediary layer between I/O jobs and OS threads. Instead of directly assigning jobs to threads, jobs are grouped into bins that can be dynamically reassigned. This mediator allows the system to maintain job associations even when threads become inactive, preventing I/O timeouts while managing complexity through the bin abstraction layer.
Solution Approach 2:
The patent implements dynamic reassociation of job bins to OS threads based on thread availability. When threads become inactive or are reactivated, the system dynamically updates the associations between job bins and threads, ensuring that I/O jobs are always assigned to active threads. This dynamic approach prevents timeouts while adapting to changing thread states.
2Adaptability or versatility
If the number of active threads changes, then system can adapt to load variations, but existing I/O job associations become invalid requiring redistribution
Solution Approach 1:
The patent segments I/O jobs into discrete job bins that can be independently managed and reassigned. When the number of active threads changes, only the affected job bins need to be redistributed rather than all jobs. This segmentation reduces the time loss associated with redistribution by limiting the scope of changes needed.
Solution Approach 2:
The patent maintains job bin structures in advance that can be quickly reassigned when thread quantities change. By having job bins pre-formed and organized, the system can perform rapid redistribution when threads are added or removed, minimizing the time loss during adaptation without requiring complete job reassignment.
3Productivity
If threads are frequently activated and deactivated, then resource utilization is optimized, but I/O job associations become unstable causing timeouts
Solution Approach 1:
The job bin intermediary layer absorbs the instability caused by frequent thread activation and deactivation. When threads change state, the job bins remain as stable containers that can be reassigned without affecting the underlying I/O job structures. This mediator protects I/O operation reliability while allowing flexible thread management for optimized resource utilization.
Data Source
AI summary
A thread may be de-activated (terminated or hibernated) or activated (e.g., re-activated or create anew if allowed) on a processing node, in response to which it may be desirable to redistribute the I/O jobs among the now active threads. Redistributing the I/O jobs may involve re-associating one or more active threads resulting from the activation or de-activation with one or more of the bin groups and/or re-assigning one or more job bins with one or more bin groups, for example, as will now be described. The bin groups may be re-associated with remaining active threads. I/O jobs may be redistributed among the active threads re-assigning job bins to bin groups. One or more queued I/O jobs may be moved to the thread that now owns the I/O job.