Tiered Data Storage Deduplication via Content-Addressable Hashing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

The ongoing storage of multiple versions of data in tiered systems incurs significant economic and computing costs due to the large amount of data and the need to maintain version and location information, despite declining storage costs.

Innovation Solution

A three-tiered data storage system is implemented, where agents at the customer location create and store checkpoints, appliances store snapshots, and cloud storage archives data, with policies defining when and how data is stored, utilizing differencing to reduce redundancy by storing only changed portions of data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If multiple versions of data are stored in tiered systems, then data resiliency and accessibility are improved, but storage costs and computational overhead increase

Engineering Contradiction:
Improvedata resiliencyVSAvoidstorage costs
Core Design Contradiction:
ReliabilityVSQuantity of substance

Solution Approach 1:

The patent merges multiple versions of the same data into a single stored copy by implementing a content-addressable storage system. When data is written, the system checks if identical data already exists using hashing; if found, it stores only a reference to the existing copy rather than duplicating the data. This combining approach maintains data resiliency through versioning metadata while eliminating redundant storage of identical data blocks.

Inventive Principle:
Principle #5Merging (Combining)

Solution Approach 2:

The patent uses content-addressable copying where data is copied only when actually modified. The system creates snapshots and versions by copying only the changed data blocks rather than entire datasets. This selective copying mechanism, combined with deduplication, reduces storage requirements while maintaining the ability to restore any previous version of data.

Inventive Principle:
Principle #26Copying

2Reliability

If multiple versions of data are stored in tiered systems, then data resiliency and accessibility are improved, but computational overhead for maintaining version and location information increases

Engineering Contradiction:
Improvedata accessibilityVSAvoidcomputational overhead
Core Design Contradiction:
ReliabilityVSDevice complexity

Solution Approach 1:

The patent implements self-service mechanisms where the storage system automatically manages versioning and deduplication without requiring complex external management. The content-addressable storage automatically hashes incoming data, checks for existing copies, and manages version metadata. This automation reduces computational overhead by eliminating manual version tracking and location management while maintaining data accessibility.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The patent changes the fundamental parameter of data identification from location-based to content-based addressing. By using hashing of data content as the address key, the system transforms how data is located and versioned. This parameter change simplifies the management of multiple versions since each unique content automatically gets a unique identifier, eliminating the need for complex version tracking infrastructure.

Inventive Principle:
Principle #35Parameter changes

3Reliability

If data is stored locally and in cloud-based systems, then data resiliency is improved, but the amount of redundant data and storage costs increase

Engineering Contradiction:
Improvedata resiliencyVSAvoidredundant data
Core Design Contradiction:
ReliabilityVSLoss of substance

Solution Approach 1:

The patent segments data into discrete blocks that can be independently addressed, stored, and managed across different locations (local and cloud). Each data block is hashed and stored as a separate entity, allowing the system to distribute segments across tiered storage locations while eliminating redundancy. This segmentation enables efficient deduplication where identical blocks anywhere in the system are recognized and deduplicated.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent creates a universal content-addressable storage layer that operates consistently across local and cloud-based systems. The same deduplication and versioning mechanisms work uniformly regardless of whether data resides locally or in the cloud. This universal approach eliminates redundant data across the entire distributed system while maintaining resiliency through consistent data protection policies across all storage locations.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Data Source

PatentUS11275522B2Efficient data storage management system
Publication Date: 2022.03.15 APARAVI SOFTWARE AG
  • US11275522B2 patent drawing
  • US11275522B2 patent drawing
  • US11275522B2 patent drawing

AI summary

An efficient data storage system is described. An agent software application on computing devices in a first tier processes snapshot backups and pushes them to an appliance software application on a server in a second tier. The appliance software application processes archive backups and pushes them to cloud storage in a third tier. A cloud application on a management server receives storage policy specifications from customers and promulgates the policies to the agent software application and the appliance software application. The policy specifications include a snapshot specification including a snapshot time period for backups in the second tier and an archive specification including an archive time period for backups in the third tier. The backups are created efficiently such that if a file has not changed, a reference to a file is included in a storage set rather than the data file itself. This reduces the size of storage sets.