Selective Data Replication via Block Checksums
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data replication methods in computer networks are inefficient, requiring significant processing power and storage space, especially for large databases, and can impact system performance.
Innovation Solution
A method that assesses file attributes and sizes to determine if replication is necessary, using checksum comparisons for larger files and synchronizing metadata for identical checksums, with a threshold-based approach to optimize resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data shadowing and mirroring are used to protect data, then data reliability is improved, but processing power and storage space are significantly consumed
Solution Approach 1:
The patent segments the data replication process by dividing files into blocks and selectively replicating only those blocks that have changed. This is achieved by comparing checksums of file blocks between source and destination, thereby avoiding replication of unchanged data and reducing processing power consumption while maintaining data reliability.
Solution Approach 2:
The patent applies partial action by performing replication only on necessary portions of data. Instead of replicating entire files or volumes, the system identifies and replicates only the specific file blocks that contain changes, thus consuming less processing power and storage space while still ensuring data protection.
2Reliability
If data shadowing and mirroring are used to protect data, then data reliability is improved, but storage space is significantly consumed
Solution Approach 1:
The patent segments data into file blocks and maintains replication information selectively. By dividing files into manageable blocks and tracking changes at the block level, the system reduces the amount of storage space required for replication metadata and actual replicated data, while still ensuring complete data protection.
Solution Approach 2:
The patent changes the parameter of replication granularity from file-level or volume-level to block-level. This parameter change enables more efficient use of storage space by allowing selective replication of only the necessary data blocks, rather than replicating entire files or volumes regardless of actual changes.
3Productivity
If differential replication operations are used to copy changed files, then replication efficiency is improved, but processing power is significantly consumed
Solution Approach 1:
The patent segments the differential replication process into block-level operations. By dividing files into blocks and computing checksums for individual blocks, the system can efficiently identify changed portions without processing entire files, thereby improving replication efficiency while reducing processing power consumption.
Solution Approach 2:
The patent replaces traditional file-level differential replication mechanisms with a block-level checksum-based system. This substitution uses cryptographic hash functions (checksums) to detect changes, which is computationally more efficient than traditional file comparison methods, thereby improving replication efficiency while consuming less processing power.
4Productivity
If incremental replication operations are used to copy changed files, then replication efficiency is improved, but network bandwidth is significantly consumed
Solution Approach 1:
The patent segments replication data into blocks and transmits only those blocks that have changed. By identifying changed blocks through checksum comparison and replicating only those specific blocks over the network, the system improves replication efficiency while minimizing network bandwidth consumption.
Solution Approach 2:
The patent extracts and replicates only the necessary data blocks that contain changes. By removing unchanged blocks from the replication stream through checksum-based identification, the system reduces network bandwidth consumption while maintaining replication efficiency.
Data Source
AI summary
Systems and methods for performing data replication are disclosed. Determining whether to update replicated data typically involves comparison of readily obtainable attributes of a given source file and its corresponding replicated file. Such attributes can be obtained from, for example, metadata. In certain situations, an additional assessment of the source and replicated files can be beneficial. For example, if integrity of an existing replicated file's content is maintained, one may not want to re-replicate the corresponding source file. For large source files, such a decision can provide substantial reductions in expenditures of available computing and network resources. In certain embodiments, a threshold for identifying such large files can be based on one or more operating parameters such as network type and available bandwidth. In certain embodiments, replication file's integrity can be checked by calculating and comparing checksums for the replication file and its corresponding source file.