Dynamic Compression Dictionary Sizing for Replica Update Coalescing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods in storage area networks with replicated storage objects are inefficient due to static compression dictionary sizes, which do not adapt to varying update sizes and rates, leading to suboptimal compression efficiency.
Innovation Solution
A method and apparatus that dynamically select a compression dictionary size based on the aggregate size of replica updates enqueued for transmission, coalesce updates, and compress them using the selected dictionary size to improve compression efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a static compression dictionary size is used, then the system is simple to operate, but compression efficiency deteriorates when update sizes and rates vary
Solution Approach 1:
The patent implements dynamic adjustment of compression dictionary size based on the aggregate size of enqueued replica updates. The system monitors the total size of updates waiting in the buffer and automatically selects an appropriate dictionary size from available options, allowing the compression system to adapt to varying update patterns without manual intervention while maintaining high compression efficiency.
2Productivity
If a larger compression dictionary size is used, then compression ratio improves, but memory usage and processing overhead increase
Solution Approach 1:
The patent changes the parameter of dictionary size dynamically based on the workload characteristics. Instead of using a fixed large dictionary that consumes excessive memory, the system selects from multiple predefined dictionary sizes (e.g., 256KB, 1MB, 4MB) based on the aggregate size of enqueued updates, optimizing the balance between compression ratio and memory consumption for each specific scenario.
3Loss of energy
If replica updates are coalesced before compression, then network bandwidth usage reduces, but latency increases
Solution Approach 1:
The patent applies partial coalescing by combining updates only when the aggregate size reaches a threshold that justifies the coalescing operation. The system monitors enqueued updates and performs coalescing selectively based on current buffer conditions, achieving bandwidth reduction without excessive coalescing that would introduce unacceptable latency.
Data Source
AI summary
Storage arrays in a mirror relationship use dynamic compression dictionary size adjustment and data buffer merging to more efficiently compress data associated with maintenance of consistency of a replicated storage object. A compression dictionary size selector selects a supported compression dictionary size based on the aggregate size of the updates enqueued for transmission in the data buffers, e.g., a compression dictionary size that is greater than or equal to, and as close as possible to, the aggregate size of the enqueued updates. A combiner selects enqueued updates based on the selected compression dictionary size, e.g., aggregate enqueued update size less than or equal to, and as close as possible to, the selected compression dictionary size. The selected updates are coalesced and compressed using the selected compression dictionary size.