External Log-Structured Engine for Flash Read Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional network storage systems face challenges in achieving consistent read latency due to the overhead of internal data layout engines in flash memory devices, variability in read access times, and inefficient write operations in flash solid-state drives (SSDs), which hinder predictable system behavior and performance.
Innovation Solution
Incorporating 'raw' flash memory into network storage systems controlled by an external log-structured data layout engine that employs a write-anywhere policy, avoiding internal data layout engines and using parity-based RAID techniques to manage data access and scheduling, thereby reducing wear and variability in read latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If flash memory with internal data layout engine is used, then read access speed is improved, but overhead is added which offsets performance gains
Solution Approach 1:
The patent removes the internal data layout engine from the flash memory device, extracting this complex functionality to the host system. This allows the flash memory to operate as a simpler raw storage device while the host's data layout engine handles all data placement and retrieval operations, eliminating the overhead associated with dual controllers and enabling direct flash access without internal translation layers.
Solution Approach 2:
The patent introduces an external data layout engine as an intermediary between the host system and the raw flash memory. This mediator handles the translation between logical storage addresses and physical flash locations, as well as managing the erase-before-write requirement, thereby simplifying the flash device itself while maintaining efficient data access through coordinated host-device operation.
2Speed
If conventional flash memory is used, then read performance is superior to disk drives, but write performance is not as good due to erase requirement
Solution Approach 1:
The patent implements preliminary actions by pre-flushing and pre-erasing flash blocks before writing new data. The host system maintains a queue of write operations and performs erase operations in advance on blocks that will be overwritten, ensuring that the flash memory is in the correct state before actual data writing begins. This eliminates the latency penalty of sequential erase-write operations.
Solution Approach 2:
The patent enables continuous useful action by allowing multiple write operations to be queued and processed in parallel. The host system can issue write commands to different flash blocks simultaneously, and the data layout engine coordinates these operations to minimize idle time. Erase operations are performed in the background on blocks ready for overwriting, maintaining continuous productive operation rather than sequential blocking.
3Reliability
If flash memory blocks are erased before writing, then write integrity is maintained, but latency is added to write operations
Solution Approach 1:
The patent applies preliminary action by pre-flushing and pre-erasing flash blocks before they are needed for writing. The host system maintains a queue of write operations and performs erase operations in advance on blocks that will be overwritten, ensuring that the flash memory is in the correct state before actual data writing begins. This eliminates the latency penalty of sequential erase-write operations.
Solution Approach 2:
The patent enables continuous useful action by allowing multiple write operations to be queued and processed in parallel. The host system can issue write commands to different flash blocks simultaneously, and the data layout engine coordinates these operations to minimize idle time. Erase operations are performed in the background on blocks ready for overwriting, maintaining continuous productive operation rather than sequential blocking.
4Reliability
If read operations wait for erase operations to complete, then data consistency is ensured, but read latency increases
Solution Approach 1:
The patent implements dynamics by making the read operation queue adaptable to the state of flash blocks. When a block is in the process of being erased or is marked as unavailable, the read queue dynamically redirects requests to other available blocks in the same RAID group. This dynamic redistribution ensures that read operations continue without waiting for erase operations to complete, maintaining both data consistency through RAID reconstruction and low latency through continuous operation.
Solution Approach 2:
The patent introduces an external data layout engine as an intermediary between the host system and the raw flash memory. This mediator handles the translation between logical storage addresses and physical flash locations, as well as managing the erase-before-write requirement, thereby simplifying the flash device itself while maintaining efficient data access through coordinated host-device operation.
Data Source
AI summary
A technique for achieving consistent read latency from an array of non-volatile solid-state memories involves an external entity determining the “busy” or “not busy” status of non-volatile solid-state memory elements in a RAID group. An external data layout engine then uses parity based RAID data reconstruction to avoid having to read from any memory element that is busy in a RAID group, along with careful scheduling of writes and erasures.