Storage Tiering After Disaster Recovery With Valid Data Staging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face inefficiencies in data management and reliability, particularly in handling power failures and data integrity during recovery operations, due to reliance on lower-level storage controllers and redundant write operations.
Innovation Solution
Implementing a direct-mapped flash storage system where higher-level operating systems manage data operations, including direct block addressing and proactive data relocation, with non-volatile RAM buffers and dual storage controllers for failover and data integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If redundant write operations are performed to ensure data integrity, then reliability is improved, but productivity deteriorates due to increased write overhead
Solution Approach 1:
The patent implements a log-structured file system that pre-organizes data into sequential write units (logs) before writing to flash memory. This preliminary organization allows the system to write data in efficient sequential patterns rather than random writes, reducing write amplification while maintaining data integrity through the log structure's inherent ordering and checkpoint mechanisms.
Solution Approach 2:
The patent uses copy-on-write techniques where data is first written to a log buffer or temporary location, then copied to the final destination only when integrity is confirmed. This copying approach allows the system to maintain reliable data storage while reducing the number of actual write operations to the flash memory, as unsuccessful writes can be discarded without affecting the source data.
2Device complexity
If lower-level storage controllers are used to manage data operations, then device complexity is reduced, but reliability deteriorates due to limited error handling capabilities
Solution Approach 1:
The patent introduces a log-structured file system layer as an intermediary between the host and flash memory storage. This intermediary layer provides sophisticated error handling, power failure recovery, and data integrity checks that go beyond basic controller capabilities. The log structure acts as a buffer and recovery mechanism, allowing the system to maintain reliability without requiring complex hardware controllers.
Solution Approach 2:
The patent replaces reliance on hardware controller mechanisms with software-based error handling and recovery procedures. Instead of using complex storage controllers with built-in error correction, the system uses firmware or operating system-level log-structured file systems that provide equivalent or superior reliability through software-managed checkpoints, logs, and recovery protocols.
3Reliability
If data is written immediately to ensure data availability, then reliability is improved, but loss of substance increases due to unnecessary write amplification
Solution Approach 1:
The patent implements a log-structured file system that pre-organizes data into sequential write units (logs) before writing to flash memory. This preliminary organization allows the system to write data in efficient sequential patterns rather than random writes, reducing write amplification while maintaining data integrity through the log structure's inherent ordering and checkpoint mechanisms.
Solution Approach 2:
The patent maintains continuous write operations to the log buffer in memory, allowing data to be accumulated and organized continuously rather than in discrete interrupt-driven writes. This continuous operation reduces the overhead of write operations and minimizes write amplification, as data flows continuously through the log structure to flash memory in efficient sequential batches.
Data Source
AI summary
Staging data on a storage element integrating fast durable storage and bulk durable storage, including: receiving, at a storage element integrating fast durable storage and bulk durable storage, a data storage operation from a host computer; storing data corresponding to the data storage operation within fast durable storage in accordance with a first data resiliency technique; and responsive to detecting a condition for transferring data between fast durable storage and bulk durable storage, transferring the data from fast durable storage to bulk durable storage in accordance with a second data resiliency technique.


