Sparse Metadata Segment Trees for Backup Storage Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face high overhead and metadata churn issues when handling incremental backups, leading to reduced storage device lifespan and inefficiency, particularly due to the need to update entire metadata structures for small data changes.
Innovation Solution
The implementation of sparse metadata segment trees (SMSTs) that only store changes since the last backup, allowing for efficient access, reading, and updating of files by linking incremental changes to a base segment tree using unique identifiers, reducing the need for frequent updates across the entire metadata structure.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If the pointing technique using small metadata to reference large data is used, then storage overhead is minimized, but metadata churn increases and storage device lifespan decreases
Solution Approach 1:
The patent segments the metadata structure into a hierarchical tree format where the entire file system metadata is divided into smaller manageable segments organized in trees. Each tree represents a portion of the file system, allowing incremental updates to affect only specific segments rather than requiring updates to the entire metadata structure. This segmentation reduces metadata churn while maintaining efficient access patterns.
Solution Approach 2:
The patent introduces a hierarchical dimension to the metadata structure by organizing segments into multi-level trees with root nodes, intermediate nodes, and leaf nodes. This dimensional organization allows the system to traverse only the necessary path from root to leaf when accessing or updating specific files, rather than scanning or updating the entire flat metadata structure, thereby reducing the frequency of metadata writes.
2Productivity
If incremental backups storing only changed data are implemented, then backup size and time are reduced, but the frequency of metadata updates increases
Solution Approach 1:
The hierarchical segment tree structure divides metadata into discrete segments that can be independently updated. When incremental backups occur, only the specific segments containing changed file metadata are updated and written to storage, rather than updating the entire metadata structure. This segmentation enables fast incremental backups while minimizing metadata update frequency.
Solution Approach 2:
The patent pre-organizes metadata into a hierarchical tree structure with predetermined segment boundaries and traversal paths. This preliminary organization allows the backup system to quickly identify and update only the necessary segments during incremental backups, avoiding the need to scan or update unrelated metadata portions, thereby reducing backup time and update frequency.
3Reliability
If small metadata changes are updated frequently to reflect file modifications, then data access accuracy is maintained, but storage device wear increases
Solution Approach 1:
By segmenting metadata into hierarchical trees, the patent ensures that updates to file metadata affect only the specific segments containing the changed information and their ancestral nodes in the tree hierarchy. This localized update approach maintains data access accuracy for modified files while preventing unnecessary writes to unrelated segments, thereby reducing storage device wear from frequent update operations.
Solution Approach 2:
The patent employs a segment tree structure where individual segments can be independently created, updated, or discarded. When metadata changes occur, the system creates new versioned segments rather than repeatedly rewriting the same segments, allowing the storage system to tolerate higher update frequencies on disposable segment objects while protecting the overall device from excessive wear through selective updating strategies.
Data Source
AI summary
Methods, systems, and apparatus for optimizing a cache memory device of a storage system are described. In one embodiment, a first base segment tree representing a first full backup including data and metadata describing the data is cached in a cache memory device. Subsequently, a plurality of incremental segment trees representing incremental backups to the first full backup are cached in the cache memory device. Each of incremental segment trees corresponding to the changes to the first full backup, without modifying the first base segment tree in response to the changes. At least two of the incremental segment trees are merged into an updated incremental segment tree to reduce a storage space of the cache memory device to store the incremental segment trees. The updated incremental segment tree comprises data and metadata represented by two or more incremental segment trees.