Client Managed Backup Metadata Caching for Storage Manager Bottleneck
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information management systems face bottlenecks in data processing due to communication constraints, particularly during backup and restore operations, which limit the scalability and availability of computing resources as the number of client devices increases.
Innovation Solution
Implementing a distributed backup management system where clients store job metadata locally and communicate directly with cloud or network storage, reducing the burden on the storage manager and media agent, and enabling clients to manage backup operations independently to alleviate communication bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a centralized storage manager coordinates all backup operations for multiple clients, then backup management control is improved, but communication bottlenecks worsen as the number of clients increases
Solution Approach 1:
The patent segments the centralized storage manager's coordination function by introducing distributed media agents that operate autonomously at client locations. Each media agent handles backup operations locally without requiring continuous communication with the storage manager, thereby segmenting the communication load and eliminating the bottleneck while preserving backup management control through periodic status reporting and policy enforcement.
2Adaptability or versatility
If the number of client devices supported by the information management system increases, then system coverage is improved, but resource availability deteriorates due to communication bottlenecks
Solution Approach 1:
The patent implements self-service by enabling media agents to autonomously perform backup operations without continuous storage manager intervention. Media agents locally manage data transfer, caching, and coordination with cloud storage, allowing the system to scale to numerous clients without proportionally increasing storage manager computing resources.
3Reliability
If clients communicate frequently with the storage manager during backup operations, then backup coordination is improved, but communication overhead increases
Solution Approach 1:
The patent applies preliminary action by having media agents pre-fetch backup data from client devices and store it in local caches before cloud storage operations are initiated. This preliminary local processing reduces the frequency and volume of communications required during actual backup execution, minimizing communication overhead while maintaining coordination through initial setup and periodic status updates.
Data Source
AI summary
Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, a client obtains information for enabling a secondary storage job (e.g., a backup or restore) from a storage manager and stores the information (which may be referred to as job metadata) in a local cache. The client may then reuse the job metadata for multiple storage jobs reducing the frequency of communication with the storage manager. When a configuration of the information management system changes, or the availability of resources changes, the storage manager can push updates to the job metadata to the clients. Further, a client can periodically request updated job metadata from the storage manager ensuring that the client does not rely on out-of-date job metadata.