Decoupled Object Storage Metadata Replication

Solution Overview

Problem

Traditional distributed object storage systems face inefficiencies due to the replication of both data and metadata, leading to increased complexity, time delays, and bandwidth consumption, as they apply the same storage rules to both, resulting in unnecessary storage of digital content in multiple locations.

Innovation Solution

An intelligent object storage system that uses user-definable rules to decouple digital content and metadata, allowing only metadata to be stored in remote locations while ensuring data protection and disaster recovery, with the option to retrieve digital content as needed, thereby optimizing storage capacity and reducing replication overhead.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If both digital content and metadata are replicated to remote locations, then data protection and disaster recovery are improved, but storage capacity utilization deteriorates and bandwidth consumption increases

Engineering Contradiction:

Improvedata protectionVSAvoidstorage capacity utilization

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent segments the object storage into two distinct components: metadata and digital content. Remote locations replicate and store only the metadata portion of objects, while the full digital content remains stored at the source location. This segmentation allows remote sites to maintain data protection and accessibility for metadata operations without duplicating the entire digital content, thereby improving storage capacity utilization while maintaining reliability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the metadata from the digital content object and separates it for independent replication to remote locations. By taking out only the essential metadata component and leaving the bulk digital content at the source, the system achieves disaster recovery capability for metadata while avoiding the bandwidth consumption and storage overhead of replicating complete objects.

Inventive Principle:

Principle #2Taking out (Extraction)

2Reliability

If both digital content and metadata are replicated to remote locations, then data protection and disaster recovery are improved, but bandwidth consumption increases

Engineering Contradiction:

Improvedisaster recoveryVSAvoidbandwidth consumption

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent segments the object storage into two distinct components: metadata and digital content. Remote locations replicate and store only the metadata portion of objects, while the full digital content remains stored at the source location. This segmentation allows remote sites to maintain data protection and accessibility for metadata operations without duplicating the entire digital content, thereby improving storage capacity utilization while maintaining reliability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the metadata from the digital content object and separates it for independent replication to remote locations. By taking out only the essential metadata component and leaving the bulk digital content at the source, the system achieves disaster recovery capability for metadata while avoiding the bandwidth consumption and storage overhead of replicating complete objects.

Inventive Principle:

Principle #2Taking out (Extraction)

3Reliability

If replication is applied to both data and metadata, then data protection is improved, but replication time and processing delays increase

Engineering Contradiction:

Improvedata protectionVSAvoidreplication time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the object storage into two distinct components: metadata and digital content. Remote locations replicate and store only the metadata portion of objects, while the full digital content remains stored at the source location. This segmentation allows remote sites to maintain data protection and accessibility for metadata operations without duplicating the entire digital content, thereby improving storage capacity utilization while maintaining reliability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent extracts the metadata from the digital content object and separates it for independent replication to remote locations. By taking out only the essential metadata component and leaving the bulk digital content at the source, the system achieves disaster recovery capability for metadata while avoiding the bandwidth consumption and storage overhead of replicating complete objects.

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentEP2962218B1Decoupled content and metadata in a distributed object storage ecosystem

Publication Date: 2021.04.07 HITACHI VANTARA LLC

Figure 1
Figure 2
Figure 3

AI summary

A storage system comprises: a controller; a memory; one or more objects, each object having content data and metadata which includes system metadata; and a policy which governs writing of the content data and metadata of the one or more objects from the storage system to an external storage. The policy includes a configurable criterion for decoupling of the content data and the metadata of a given object and a rule for determining storage locations for the content data and the metadata between the storage system and the external storage in a decoupled manner if the content data and the metadata of the object are decoupled. The configurable criterion is based on an attribute of the metadata of the object. The controller is operable to execute the policy against the objects and to apply the rule to determine storage locations for the content data and metadata of a decoupled object.