Specialized Metadata for Cloud Backup Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing and searching incremental backup data stored on cloud-based object storage systems is inefficient due to the difficulty in tracking data relationships and the resource-intensive nature of existing search tools, which do not account for data stored as part of an incremental backup infrastructure.
Innovation Solution
A system that utilizes a specialized metadata database to efficiently manage and search backup data by creating and storing metadata separately from the objects, allowing for efficient reconstruction, search, and allocation of backup data, including email properties, without accessing the stored objects, and leveraging this metadata to reallocate data to different storage tiers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If incremental backups are used to limit data transfer, then data transfer volume is reduced, but tracking data relationships becomes increasingly difficult
Solution Approach 1:
The patent introduces a metadata database as an intermediary layer between the object storage and the backup management system. This metadata database stores structured information about backup data relationships, including parent-child relationships between incremental backups and their base backups. The metadata acts as a mediator that simplifies the tracking of data relationships without requiring direct analysis of the stored backup objects, thus resolving the contradiction between reduced data transfer and increased tracking complexity.
2Ease of operation
If native search tools are used on object storage, then search functionality is provided, but searching becomes inefficient and resource intensive
Solution Approach 1:
The patent implements preliminary action by pre-processing and indexing backup data metadata during the backup operation itself. The metadata database is populated and organized in advance with searchable attributes and relationships. When a search operation is needed, the system queries the pre-organized metadata database rather than scanning through raw backup objects, dramatically improving search efficiency and reducing resource consumption while maintaining ease of operation.
3Adaptability or versatility
If backup data is stored as objects on cloud storage, then scalability and cost efficiency are improved, but managing and tracking data relationships becomes difficult
Solution Approach 1:
The patent applies segmentation by separating the management of backup data into two distinct components: the actual backup objects stored in the cloud object storage system, and the metadata about these objects stored in a separate metadata database. This segmentation allows the scalable object storage to handle large volumes of backup data while the separate metadata database handles the complex relationship tracking. The two components work together through references, enabling both scalability and manageable complexity.
Data Source
AI summary
Described is a system (and method) for managing specialized metadata that may be used to manage and search incremental backup data stored on a cloud-based object storage. The system may create and store such metadata as part of a specialized metadata database that includes a data catalog and a backup catalog. The system may leverage the metadata database to initiate operations to efficiently manage incremental backup data stored on the object storage. For example, the metadata may be relied upon to efficiently reconstruct (e.g. synthetically) the client data to a point-in-time of any incremental backup. In addition, the metadata may include properties of the backed-up data, which are maintained separately from the backup data stored as objects. Accordingly, these properties may be searched to identify and locate backup data without having to retrieve the stored objects.