Distributed Media Agent Coordination for Scalable Auxiliary Copy Jobs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional centralized approaches to managing auxiliary-copy jobs in storage management systems lead to bottlenecks, as many processes executing on the same platform can cause performance issues and impact other operations, particularly in larger systems.
Innovation Solution
A scalable approach is implemented where media agents perform control and coordination functions, offloading tasks from the storage manager and prioritizing jobs using a job-priority queue, allowing for on-demand data stream reservations and resource allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a centralized approach is used where the storage manager executes one auxiliary-copy manager process for every auxiliary-copy job, then job management control is improved, but system performance deteriorates due to bottlenecks in the storage manager
Solution Approach 1:
The patent segments the centralized auxiliary-copy job management into distributed components: auxiliary-copy managers are distributed across multiple media agents rather than centralized in the storage manager. Each media agent independently manages auxiliary-copy jobs for its associated storage devices, eliminating the bottleneck where the storage manager had to execute one process per job.
Solution Approach 2:
The patent introduces an intermediary mechanism where media agents act as intermediaries between the storage manager and auxiliary-copy jobs. The storage manager communicates job priorities and metadata to media agents, which then handle the actual auxiliary-copy processing locally, reducing direct processing burden on the storage manager.
2Productivity
If multiple auxiliary-copy processes execute on the same processing platform, then job processing capability is improved, but resource contention increases causing bottlenecks
Solution Approach 1:
The patent segments the processing workload by distributing auxiliary-copy processes across multiple media agents rather than concentrating them on a single processing platform. Each media agent handles auxiliary-copy jobs for its local storage devices, spreading the computational and I/O resource demands across the system infrastructure.
Solution Approach 2:
The patent adds a spatial dimension to process distribution by organizing auxiliary-copy management hierarchically: the storage manager handles high-level coordination and prioritization, while media agents handle local execution. This dimensional separation allows parallel processing without direct resource contention on the same platform.
3Stability of the object's composition
If the storage manager manages all auxiliary-copy jobs centrally, then system coordination is improved, but scalability deteriorates in larger systems with hundreds or thousands of jobs
Solution Approach 1:
The patent segments the system into hierarchical layers: the storage manager maintains system-wide coordination through job priority queues and metadata management, while media agents independently manage local auxiliary-copy execution. This segmentation allows the system to scale horizontally by adding more media agents without proportionally increasing the storage manager's processing burden.
Solution Approach 2:
The patent enables media agents to perform self-service for auxiliary-copy job execution. Each media agent autonomously processes auxiliary-copy jobs for its local storage devices using locally available resources, reducing the coordination overhead on the storage manager and enabling independent scaling of storage capacity and processing capability.
Data Source
AI summary
A scalable approach is disclosed for processing auxiliary-copy jobs in a storage management system by using distributed media agent resources instead of a centralized storage manager. Enhanced media agents coordinate and control auxiliary-copy jobs and tap the storage manager to reserve data streams and provide job-specific metadata on demand. An enhanced storage manager may initially select a media agent as “coordinator” to coordinate auxiliary-copy jobs with any number of other media agents, which act as “controllers.” A coordinator media agent is generally responsible for obtaining data stream reservation information from the storage manager and assigning auxiliary-copy jobs to respective controller media agents, based on the components involved in the respective reserved data streams.