Archival Task Offloading to Secondary Storage Clusters
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data archival operations in primary clusters of data storage systems consume resources, impacting performance and client I/O requests, as they handle file selection, transfer, and storage, while secondary clusters remain idle for failover, indicating a need to offload archival tasks to minimize resource usage and enhance system efficiency.
Innovation Solution
Implementing a data storage system with a file designation component, cluster selection component, and archival management component that selects files for archival from primary clusters, locates matching copies in secondary clusters, and directs them to remote storage, thereby offloading archival operations to idle secondary clusters, reducing resource consumption and improving primary cluster performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If primary clusters perform archival operations, then data archival is achieved, but resource consumption increases and performance deteriorates
Solution Approach 1:
The patent extracts the archival operation from the primary cluster and transfers it to the secondary cluster. The primary cluster designates files for archival and receives confirmation, while the secondary cluster actually performs the archival execution, thereby separating the archival function from the primary cluster's resource-intensive operations.
Solution Approach 2:
The patent introduces an archival management component as an intermediary that coordinates between the primary cluster and secondary cluster. This mediator manages the archival process by receiving file designations from the primary cluster, selecting appropriate secondary clusters, and coordinating the actual archival execution, thereby optimizing resource utilization across the distributed system.
2Productivity
If primary clusters handle file selection and transfer, then archival tasks are completed, but resource usage increases
Solution Approach 1:
The patent segments the archival task into distinct phases: file designation by the primary cluster, secondary cluster selection by the archival management component, and archival execution by the secondary cluster. This segmentation distributes computational workload across multiple clusters, reducing resource consumption at any single location while maintaining overall archival productivity.
3Reliability
If secondary clusters remain idle for failover, then system reliability is maintained, but resource utilization decreases
Solution Approach 1:
The patent makes secondary clusters multi-functional by enabling them to perform both their traditional failover role and new archival execution role. The secondary clusters maintain their standby capability for failover while simultaneously being utilized for archival operations, thereby improving resource utilization without compromising system reliability.
Data Source
AI summary
Archival task processing in a data storage system is described herein. A method as described herein can include designating, by a device operatively coupled to a processor, a file stored by a primary cluster of a data storage system for archival to a remote storage system; locating, by the device, a secondary cluster of the data storage system having stored thereon a copy of the file, the secondary cluster being distinct from the primary cluster; and, in response to determining that the file stored by the primary cluster matches the copy of the file stored by the secondary cluster, causing, by the device, the secondary cluster to archive the copy of the file to the remote storage system.