Deduplication System Small Object Direct Reference Storage

Solution Overview

Problem

Existing deduplication systems are inefficient in handling both large and small objects, as they often require similar data structures and processes, leading to increased storage size and latency for small objects due to unnecessary intermediate-level data structures, which are not optimal for compact storage and quick access.

Innovation Solution

Implementing a deduplication system that differentiates between large and small objects by using a hierarchy of data structures for large objects and direct references for small objects, omitting intermediate-level data structures to reduce storage size and memory usage, thereby enhancing efficiency and reducing I/O latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a hierarchy of data structures with intermediate-level structures is used for all objects, then large objects can be stored and managed, but small objects experience increased storage size and I/O latency due to unnecessary intermediate structures

Engineering Contradiction:

Improveability to handle both large and small objectsVSAvoidstorage size overhead for small objects

Core Design Contradiction:

Adaptability or versatilityVSQuantity of substance

Solution Approach 1:

The patent applies local quality by differentiating the data structure representation based on object size. Small objects use a simplified direct reference structure, while large objects use the full hierarchy with intermediate-level structures. This localized differentiation optimizes storage for small objects by eliminating unnecessary intermediate structures, while preserving the ability to handle large objects with the complete hierarchy.

Inventive Principle:

Principle #3Local quality

2Adaptability or versatility

If a hierarchy of data structures with intermediate-level structures is used for all objects, then large objects can be stored and managed, but I/O operations on small objects experience increased latency

Engineering Contradiction:

Improveability to handle both large and small objectsVSAvoidI/O latency for small objects

Core Design Contradiction:

Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent applies local quality by differentiating the data structure representation based on object size. Small objects use a simplified direct reference structure, while large objects use the full hierarchy with intermediate-level structures. This localized differentiation optimizes storage for small objects by eliminating unnecessary intermediate structures, while preserving the ability to handle large objects with the complete hierarchy.

Inventive Principle:

Principle #3Local quality

3Device complexity

If a unified data structure is used for all object sizes, then system complexity is reduced, but storage efficiency deteriorates for small objects

Engineering Contradiction:

Improvedata structure complexityVSAvoidstorage size efficiency

Core Design Contradiction:

Device complexityVSQuantity of substance

Solution Approach 1:

The patent applies local quality by differentiating the data structure representation based on object size. Small objects use a simplified direct reference structure, while large objects use the full hierarchy with intermediate-level structures. This localized differentiation optimizes storage for small objects by eliminating unnecessary intermediate structures, while preserving the ability to handle large objects with the complete hierarchy.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS11663234B2Storage of a small object representation in a deduplication system

Publication Date: 2023.05.30 HEWLETT PACKARD ENTERPRISE DEV LP

AI summary

Examples may include storage of a small object representation in a deduplication system. Examples may store the small object representation of an object in the deduplication system based on a determination that the object is smaller than a threshold size. In examples, the small object representation may include a direct reference from a top-level data structure to small object metadata in a bottom-level data structure of the small object representation.