Content Addressable Storage Array for Block-Level Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage systems face inefficiencies due to the duplication of data during backup operations, leading to wasteful consumption of storage resources, and content addressable storage systems are hindered by slow write operations caused by computationally intensive hash computations at the file level.
Innovation Solution
A content addressable storage array element (CASAE) performs content addressable storage computations on each data block to prevent duplicate storage, using a hashing function to determine if data already exists, and provides a block pointer instead of duplicating data, thereby optimizing storage by eliminating duplicates and enhancing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If content addressable storage computation is performed at the file level to eliminate duplicate data, then storage efficiency is improved, but write operation speed deteriorates due to computationally intensive hash computations
Solution Approach 1:
The system segments the content addressable storage computation from the file level to the block level. Each data block is independently hashed and checked for duplicates, allowing parallel processing of multiple blocks simultaneously. This segmentation reduces the computational overhead per write operation while maintaining duplicate elimination effectiveness across the entire file.
Solution Approach 2:
The patent introduces a cache memory as an intermediary between the hash computation and the storage operations. The cache stores recently computed hash values and their corresponding block locations, acting as a mediator that avoids redundant hash computations for duplicate blocks. This intermediary layer significantly reduces the computational intensity of write operations while preserving the ability to eliminate duplicates.
2Reliability
If data is duplicated during backup operations to ensure data availability, then reliability is improved, but storage resource consumption increases
Solution Approach 1:
Instead of creating full physical copies of data for backup and redundancy, the system uses content addressable storage with hash-based deduplication. When data is written, its hash is computed and checked against existing data; if a duplicate is found, only a reference is stored rather than another copy. This copying principle maintains data availability through references while dramatically reducing storage resource consumption.
Solution Approach 2:
The system changes the parameter of data representation from physical copies to hash-based references. By transforming the backup mechanism from storing actual data copies to storing computational fingerprints (hashes) of data, the system achieves the same reliability goal with minimal storage overhead. The parameter change from 'copy data' to 'hash data' enables efficient duplicate elimination.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A content addressable storage array element (CASAE) of a storage system is configured to eliminate duplicate data stored on its storage resources. The CASAE independently determines whether data associated with a write operation has already been written to a location on its storage resources. To that end, the CASAE performs a content addressable storage computation on each data block written to those resources in order to prevent storage of two or more blocks with the same data. If data of a block has been previously stored on the resources, the CASAE cooperates with a file system executing on the system to provide a reference (block pointer) to the same data block rather than duplicate the stored data. Otherwise, the CASAE stores the data block at a new location on the resources and provides a block pointer to that location.