Distributed Media Agent Coordination for Auxiliary Copy Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional centralized approaches to managing auxiliary-copy jobs in storage management systems lead to bottlenecks, as many processes executing simultaneously can overwhelm the storage manager, impacting performance and delaying other operations.
Innovation Solution
A scalable approach is implemented where media agents take on control and coordination functions, offloading tasks from the storage manager, and prioritizing jobs using a job-priority queue to distribute resources efficiently and prevent bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If a centralized approach is used where the storage manager executes one auxiliary-copy manager process for every auxiliary-copy job, then comprehensive control and coordination of auxiliary-copy jobs is achieved, but bottlenecks occur in the storage manager causing performance problems and delays
Solution Approach 1:
The patent segments the centralized auxiliary-copy management function into distributed media agent processes. Instead of one central storage manager handling all auxiliary-copy jobs, multiple media agents are deployed across the network, each capable of independently managing auxiliary-copy jobs for specific storage devices or data sets. This segmentation distributes the processing load and eliminates the bottleneck at the storage manager while maintaining comprehensive control through coordinated media agent operations.
2Reliability
If multiple auxiliary-copy manager processes execute simultaneously on the same processing platform, then complete coverage of auxiliary-copy jobs is achieved, but resource contention and bottlenecks increase
Solution Approach 1:
The patent transitions from a single-dimension centralized model (one storage manager handling all jobs) to a multi-dimensional distributed model where media agents are spread across multiple networked processing platforms. Each media agent operates on its own processing platform, utilizing available computational resources across the network. This dimensional shift from centralized to distributed architecture allows simultaneous execution of multiple auxiliary-copy jobs without overloading a single processing platform, thereby maintaining reliable job execution coverage while reducing resource contention.
3Ease of operation
If the storage manager launches new job-specific auxiliary-copy processes for each auxiliary-copy job, then dedicated process control is achieved, but system resources are consumed and bottlenecks are created
Solution Approach 1:
The patent implements universal media agent processes that can handle multiple auxiliary-copy jobs simultaneously, replacing the previous model of dedicated job-specific processes. Each media agent is designed with multi-functional capabilities to manage various auxiliary-copy jobs across different storage devices and data sets. This universality reduces the total number of processes required in the system, decreasing resource consumption while maintaining ease of operation through standardized media agent interfaces and job management protocols.
Data Source
AI summary
A scalable approach is disclosed for processing auxiliary-copy jobs in a storage management system by using distributed media agent resources instead of a centralized storage manager. Enhanced media agents coordinate and control auxiliary-copy jobs and tap the storage manager to reserve data streams and provide job-specific metadata on demand. An enhanced storage manager may initially select a media agent as “coordinator” to coordinate auxiliary-copy jobs with any number of other media agents, which act as “controllers.” A coordinator media agent is generally responsible for obtaining data stream reservation information from the storage manager and assigning auxiliary-copy jobs to respective controller media agents, based on the components involved in the respective reserved data streams.