Deduplication Server Reduces Cloud Storage Network Traffic

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud storage systems face performance issues due to network latencies and increased data transfer times when clients are far from the storage location, leading to reduced efficiency in data transfer operations.

Innovation Solution

A server is introduced to facilitate efficient data transfer by processing requests locally, dividing data into segments, generating fingerprints for deduplication, and communicating with the cloud storage system to identify and transfer only missing segments, thereby reducing the amount of data sent over the network.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Loss of time

If data is transferred directly between client and cloud storage system, then data transfer can be performed, but network latency and transfer time increase when client is far from storage location

Engineering Contradiction:

Improvedata transfer timeVSAvoidnetwork accessibility

Core Design Contradiction:

Loss of timeVSEase of operation

Solution Approach 1:

A server is introduced as an intermediary component between the client and cloud storage system. The server receives data from the client, processes it locally (dividing into segments, generating fingerprints), and then communicates with the cloud storage system. This intermediary approach reduces network latency by performing computation-intensive tasks locally rather than requiring direct client-to-cloud communication for all operations.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If all data segments are transferred to cloud storage system, then complete data can be stored, but network traffic increases

Engineering Contradiction:

Improvedata completenessVSAvoidnetwork traffic

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The server performs preliminary actions by dividing data into segments and generating fingerprints before transferring to the cloud storage system. By pre-processing the data and identifying which segments are new or modified through fingerprint comparison, the system only transfers necessary data segments rather than all data, thereby reducing network traffic while ensuring data completeness.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts only the necessary data segments for transfer by comparing fingerprints of existing and new data. Instead of transferring all data segments, the server identifies and extracts only the missing or modified segments, significantly reducing network traffic while maintaining complete data storage.

Inventive Principle:

Principle #2Taking out (Extraction)

3Productivity

If computation-intensive tasks are performed at cloud storage system, then data processing can be done, but network latency increases

Engineering Contradiction:

Improvedata processing capabilityVSAvoidprocessing latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The server segments computation-intensive tasks from the cloud storage system and performs them locally. By dividing the overall data processing workflow into segments (data reception, segmentation, fingerprint generation, comparison) that can be executed locally at the server, the system reduces processing latency while maintaining data processing capability.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11677826B2Efficient transfer to and from a deduplicated cloud storage system

Publication Date: 2023.06.13 EMC IP HLDG CO LLC

AI summary

Efficient data transfer is disclosed. A server provides an endpoint for a client. The client can communicate with a cloud storage system through the server rather than connect directly to the cloud storage system. The server is configured to perform requests in a manner that reduces the amount of data transferred to and from the cloud storage system.