Resynchronization Job Scheduling in Distributed Object Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed object-based storage systems, the arbitrary execution of resynchronization jobs leads to prolonged downtime for objects with fault tolerance requirements and inefficient storage space recycling due to the time-consuming nature of completing all resynchronization jobs across components.
Innovation Solution
Implementing a two-level queue structure for resynchronization jobs, where higher-priority objects are scheduled first, and using per-object and per-job priority queues to ensure that all jobs for a given object are completed within a shorter time frame, thereby reducing downtime and enabling earlier recycling of storage space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If resynchronization jobs are executed in arbitrary order with per-component basis, then the storage system can process multiple components concurrently, but the average time to complete data resynchronization for all components of an object becomes very long
Solution Approach 1:
The patent segments resynchronization jobs into per-object units rather than per-component units. By grouping all components of an object into a single resynchronization job, the system ensures that once a job starts, all components of that object are resynchronized together, reducing the total time objects remain non-compliant while maintaining concurrent processing capability through multiple object-level jobs
Solution Approach 2:
The patent merges multiple component-level resynchronization operations into a single object-level job. This consolidation ensures that all components of an object are processed together in one coordinated effort, preventing the scenario where some components complete while others remain pending, thus reducing the overall time the object remains in a non-compliant state
2Device complexity
If resynchronization jobs are executed in arbitrary order, then the storage system can maintain simplicity in job management, but slack space on source storage nodes cannot be recycled until all component resynchronization jobs complete
Solution Approach 1:
The patent segments the resynchronization process into object-level units, allowing the system to track and manage completion at the object level rather than waiting for all individual component jobs across all objects to complete. This enables earlier identification of completed objects and subsequent recycling of their associated slack space
Solution Approach 2:
The patent implements feedback mechanisms that monitor the completion status of object-level resynchronization jobs. When an object's resynchronization job completes, the system immediately receives feedback and can proceed to recycle the associated slack space, rather than waiting for all component jobs system-wide to finish
3Ease of operation
If resynchronization jobs are executed in arbitrary order, then the system can avoid complex scheduling logic, but the time window during which objects are not in-compliance with fault tolerance requirements is prolonged
Solution Approach 1:
The patent segments resynchronization into object-level jobs that are tracked and managed as discrete units. This segmentation allows the system to monitor compliance status at the object level and quickly restore fault tolerance compliance when object-level jobs complete, without requiring complex scheduling to prioritize specific objects
Solution Approach 2:
The patent performs preliminary grouping of components into object-level jobs before execution. This preliminary organization ensures that when a resynchronization job for an object starts, all necessary component operations are prepared to complete together, thereby reducing the duration of non-compliance without requiring complex real-time scheduling decisions
Data Source
AI summary
Techniques for intelligently scheduling resynchronization jobs in a distributed object-based storage system are provided. In one set of embodiments, a storage node of the system can create a resynchronization job for a component of an object maintained by the system, where the resynchronization job defines one or more input/output (I/O) operations to be carried out with respect to the component. If a number of currently running resynchronization jobs on the storage node has reached a threshold, the storage node can further determine a priority level associated with the object; add the resynchronization job to an object queue for the object; and if the added resynchronization job is a first job in the object queue, add the object queue as a new queue entry to a global priority queue corresponding to the priority level associated with the object.