Dynamic Data Compression for Network Replication

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing data replication methods face challenges in efficiently replicating data across remote sites with low bandwidth, as constant compression consumes CPU time, requires hardware resources, and may not significantly reduce data size, leading to latency and resource wastage.

Innovation Solution

Implementing dynamic or on-demand compression that determines whether to compress data based on bandwidth throughput, using policies to decide when compression is beneficial, and employing staggered compression to optimize data transmission.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If data is always compressed prior to transmission, then bandwidth usage is reduced, but CPU time is consumed and latency increases

Engineering Contradiction:

Improvedata sizeVSAvoidcompression time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The system dynamically adjusts compression settings based on real-time network conditions. When bandwidth is充足, compression is reduced or skipped; when bandwidth is limited, compression is increased. This dynamic adaptation resolves the contradiction by making compression optional rather than constant, reducing unnecessary CPU time consumption while maintaining bandwidth efficiency when needed.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes compression parameters (compression ratio, algorithm selection) based on network conditions and data characteristics. By adjusting these parameters dynamically, the system optimizes the balance between compression time and bandwidth usage, avoiding excessive CPU consumption for data that doesn't benefit much from compression while still achieving bandwidth savings when necessary.

Inventive Principle:

Principle #35Parameter changes

2Quantity of substance

If data is always compressed prior to transmission, then bandwidth usage is reduced, but hardware resources are consumed

Engineering Contradiction:

Improvedata sizeVSAvoidCPU time

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

Inventive Principle:

Principle #35Parameter changes

3Quantity of substance

If data is compressed, then data size is reduced, but replication latency increases

Engineering Contradiction:

Improvedata sizeVSAvoidreplication speed

Core Design Contradiction:

Quantity of substanceVSSpeed

Solution Approach 1:

The system dynamically adjusts compression settings based on real-time network conditions. When bandwidth is充足, compression is reduced or skipped to maintain fast replication; when bandwidth is limited, compression is increased to ensure data transmission completes within acceptable timeframes. This dynamic adaptation resolves the contradiction by adjusting compression based on actual network performance.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system performs preliminary assessment of network conditions and data characteristics before initiating compression. By evaluating bandwidth availability, data compressibility, and replication urgency in advance, the system determines the optimal compression strategy, avoiding unnecessary compression that would delay replication while ensuring compression is applied when it will actually improve replication performance.

Inventive Principle:

Principle #10Preliminary action

4Quantity of substance

If compression is applied to all data, then bandwidth usage is reduced, but some data does not achieve significant size reduction

Engineering Contradiction:

Improvedata sizeVSAvoidcompression efficiency

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The system applies different compression strategies to different data blocks or data types based on their compressibility characteristics. Highly compressible data receives aggressive compression, while already-compressed or random data is transmitted uncompressed. This selective approach resolves the contradiction by applying compression locally where it is effective rather than uniformly to all data, improving overall compression efficiency.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentEP3958541B1Data replication with dynamic compression

Publication Date: 2025.01.01 COMPELLENT TECH

Figure 1
Figure 2
Figure 3

AI summary

A method for replicating data between two or more network connected data storage devices, the method including dynamically determining whether to compress data prior to transmitting across the network based, at least in part, on bandwidth throughput between the network connected data storage devices. If it has been determined to compress the data, the method involves compressing the data and transmitting the compressed data over the network. If it has been determined not to compress the data, the method involves transmitting the data, uncompressed, over the network. Dynamically determining whether to compress data may include comparing bandwidth measurements with a predetermined policy defining when compression should be utilized. In some embodiments, the policy may define that compression should be utilized when an estimated time for compressing the data and transmitting the compressed data is less than an estimated time for transmitting the data uncompressed.