Client-Managed Backup Metadata Caching for Scalable Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information management systems experience communication bottlenecks due to increased client devices, leading to reduced hardware and storage availability, particularly during backup and restore processes, which limits the scalability of the system.
Innovation Solution
Clients obtain job metadata from a storage manager and store it locally, reducing the frequency of communication with the storage manager, and distribute backup management, enabling direct communication with cloud or network storage systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If clients communicate frequently with the storage manager during backup operations, then backup management and control are improved, but communication bottlenecks worsen and system scalability deteriorates
Solution Approach 1:
The patent segments the backup management functionality by introducing local agents on client devices that can independently manage backup operations. These agents obtain job metadata from the storage manager but then operate autonomously to execute backups, reducing the need for continuous communication with the central storage manager and thereby resolving the contradiction between centralized control and system scalability.
2Quantity of substance
If the number of client devices is increased, then system capacity is improved, but communication overhead increases and resource availability decreases
Solution Approach 1:
The patent implements self-service mechanisms where local agents on client devices autonomously manage their own backup operations. Agents obtain initial job metadata from the storage manager but then independently execute backups, monitor their own status, and manage local resources without requiring continuous communication with the storage manager, thereby reducing communication overhead as the system scales.
3Ease of operation
If centralized backup management is maintained, then coordination and control are improved, but communication bottlenecks increase and backup speed decreases
Solution Approach 1:
The patent divides backup management into centralized coordination (storage manager providing job metadata) and decentralized execution (local agents performing actual backups). This segmentation allows the storage manager to maintain coordination control while local agents execute backups independently at full speed without communication bottlenecks, resolving the contradiction between centralized control and backup speed.
Solution Approach 2:
The patent introduces local agents as intermediaries between the storage manager and the actual backup processes. These agents receive job metadata from the storage manager and then mediate the execution of backups locally, reducing direct communication requirements between the storage manager and backup operations, thereby improving backup speed while maintaining coordination.
Data Source
AI summary
Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, a client obtains information for enabling a secondary storage job (e.g., a backup or restore) from a storage manager and stores the information (which may be referred to as job metadata) in a local cache. The client may then reuse the job metadata for multiple storage jobs reducing the frequency of communication with the storage manager. When a configuration of the information management system changes, or the availability of resources changes, the storage manager can push updates to the job metadata to the clients. Further, a client can periodically request updated job metadata from the storage manager ensuring that the client does not rely on out-of-date job metadata.