Compressed Data Replication Using Scatter-Gather IO Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data replication methods in storage arrays are limited by slowest disk performance, leading to inefficient I/O operations and increased overheads due to decompression-recompression cycles when handling compressed data.
Innovation Solution
A method that utilizes a state bitmap to identify snapshot differences and allocation states of data, generating scatter-gather information based on target node capabilities to optimize I/O alignment, avoiding partial reading and decompression-recompression cycles.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data replication methods are used in storage arrays, then data can be copied from source to target, but I/O performance is limited by the slowest disk and decompression-recompression cycles increase overhead
Solution Approach 1:
The patent segments the data replication process by using scatter-gather lists to divide data into multiple non-contiguous segments that can be processed independently. This allows the system to replicate only the necessary portions of compressed data without requiring full decompression and recompression cycles, thereby improving replication efficiency while reducing I/O operation time
Solution Approach 2:
The patent performs preliminary actions by pre-calculating scatter-gather information and identifying allocation states before actual data replication occurs. The system prepares the data structure and determines which segments need replication in advance, avoiding unnecessary decompression-recompression operations and reducing overall I/O time
2Speed
If data is scattered and stored on multiple hard drives to improve access speed, then capacity and performance are improved, but I/O performance is limited by the slowest disk
Solution Approach 1:
The patent introduces scatter-gather information structures as intermediaries between the storage array and the data replication process. These structures enable the system to efficiently manage data across multiple disks by organizing scatter-gather lists that map logical data segments to physical disk locations, allowing parallel I/O operations without being bottlenecked by the slowest disk
Solution Approach 2:
The patent implements dynamic I/O optimization by adapting the scatter-gather list generation based on the specific IO capability of the target node. The system dynamically adjusts replication strategies according to real-time disk performance characteristics, enabling flexible data distribution that maximizes throughput while avoiding the slowest disk bottleneck
3Adaptability or versatility
If decompression-recompression cycles are performed on compressed data, then data can be handled flexibly, but overheads are increased
Solution Approach 1:
The patent extracts only the necessary portions of compressed data that need to be replicated, using scatter-gather information to identify and isolate specific data segments. This allows the system to handle compressed data flexibly by selecting and replicating only the required segments without performing full decompression-recompression cycles, thereby reducing computational overhead while maintaining data handling flexibility
Data Source
AI summary
Techniques to replicate data involve obtaining, based on a state bitmap of the data, a first identifier indicating a snapshot difference of the data and a second identifier indicating an allocation state of the data, where the first identifier indicates that data in which pages in the data needs to be replicated from a source node to a target node, and the second identifier indicates which pages in the data are compressed together as a single compression group. Such techniques further involve generating scatter-gather information for replicating the data based on an input output (IO) capability of the target node, the first identifier, and the second identifier. Such techniques further involve replicating the data corresponding to the scatter-gather information from the source node to the target node. Accordingly, data transmission overheads during data replication can be reduced, and IO performance can be improved.