Replicated Dataset Hash Validation for Storage Integrity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in data replication and management, particularly in flash storage systems, where lower-level processes often perform redundant operations, leading to increased latency and reduced reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where higher-level processes manage data operations, including direct addressing of data blocks and managing allocation units without lower-level controller intervention, and utilizing non-volatile RAM for quick data buffering and proactive data rebuilding across storage nodes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If lower-level controllers manage data operations in flash storage systems, then data management is automated, but latency increases and reliability decreases due to redundant operations
Solution Approach 1:
The patent extracts the data management functions from lower-level controllers and relocates them to higher-level processes. This removes the redundant automation layer that caused latency, allowing direct mapping between host I/O requests and flash memory operations without controller intervention.
Solution Approach 2:
The system segments data management responsibilities by separating host I/O processes from storage controller functions. Higher-level host processes handle data operations directly on flash memory devices, while lower-level controllers focus only on physical layer operations, eliminating redundant processing layers.
2Extent of automation
If lower-level controllers manage data operations, then automation is provided, but reliability decreases due to redundant operations
Solution Approach 1:
The patent extracts data management functions from lower-level controllers to higher-level host processes, removing the redundant automation layer that compromised reliability. This direct management approach eliminates the risk of controller-level errors affecting data integrity.
Solution Approach 2:
The system implements proactive data rebuilding mechanisms where higher-level processes perform data verification and reconstruction before failures occur. This preliminary action ensures data integrity without relying on potentially flawed lower-level controller operations.
3Productivity
If non-volatile RAM is used for data buffering, then write operations are reduced, but additional hardware is required
Solution Approach 1:
The patent utilizes non-volatile RAM devices that serve dual functions: as high-speed buffering memory and as persistent storage. This multi-functionality reduces the need for separate hardware components while improving write operation efficiency through the RAM's fast write capabilities.
Solution Approach 2:
The system uses non-volatile RAM to create intermediate copies of data during write operations. Data is first written to the fast non-volatile RAM buffer, then asynchronously transferred to flash storage, reducing the impact of slow flash write operations on system productivity.
Data Source
AI summary
Verifying that data has been correctly replicated to a replication target, including: replicating a dataset stored at a first computing system to a second computing system; and determining, based at least on a comparison of a first hash and a second hash, validity of the dataset stored at the second computing system, wherein the first hash is generated by applying a hash function to a copy of the dataset that is stored at the first computing system and the second hash is generated by applying the hash function to a copy of the dataset that is stored at the second computing system.