Tiering Metadata Between Deduplication Devices
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems, particularly primary storage solutions, face challenges in supporting content-based storage with ultra-high performance due to intensive computational load, block fragmentation, loss of sequential data location, and algorithmic difficulties in distributing tasks across processing and storage elements.
Innovation Solution
The implementation of a Content Addressable Storage (CAS) system with separate control and data planes, using cryptographic hashing for content addressing, and distributing metadata and data across multiple deduplication devices to optimize storage efficiency and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If content-based storage is implemented, then storage capacity efficiency is improved, but computational load increases
Solution Approach 1:
The patent divides the storage system into multiple deduplication devices, each handling a portion of the content-based storage tasks. This segmentation distributes the computational load across multiple devices while maintaining content-based storage benefits, resolving the contradiction between improved storage efficiency and increased computational burden.
2Manufacturing precision
If large blocks are broken into smaller blocks for content addressing, then content addressing granularity is improved, but performance degrades due to block fragmentation
Solution Approach 1:
The patent introduces a hierarchical block structure where data is organized across multiple levels of granularity. Instead of simply breaking blocks into smaller pieces, the system creates a multi-dimensional organization with parent-child block relationships, allowing content addressing at fine granularity while maintaining performance through the hierarchical structure.
3Speed
If address-based mapping is used, then sequential data location is maintained, but storage capacity efficiency decreases due to duplicate data
Solution Approach 1:
The patent introduces metadata as an intermediary layer between the address-based interface and content-based storage. The metadata contains content hashes that enable deduplication and content-based addressing, while the actual data blocks maintain their physical locations. This intermediary allows the system to provide address-based access while achieving content-based storage efficiency.
4Adaptability or versatility
If data is distributed across multiple devices, then scalability is improved, but system complexity increases
Solution Approach 1:
The patent creates a universal metadata format and protocol that works across multiple deduplication devices. The metadata structure and content-addressing mechanism are designed to be device-agnostic, allowing the system to scale by simply adding more devices with the same universal interface, thereby improving scalability without proportionally increasing system complexity.
Data Source
AI summary
In one aspect, a method includes moving metadata from a memory of a first deduplication device to a disk of the first deduplication device, moving the data related to the metadata or portions of the data to a second deduplication device, deleting the data from the first deduplication device and adding a pointer to the metadata at the first deduplication device indicating where the data is in the second deduplication device.