Padded Compressed Data Blocks for Block-Level Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional storage systems face inefficiencies in data management and reliability due to unnecessary write operations and lack of direct control over storage processes, leading to potential data loss and reduced system reliability.
Innovation Solution
Implementing a direct-mapped flash storage system where the operating system initiates and controls data management processes, such as data allocation and garbage collection, without relying on lower-level storage controllers, and using non-volatile RAM as a buffer to improve write latency and data persistence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional storage controllers manage data allocation and garbage collection, then storage operations are automated, but write latency increases and data reliability decreases due to unnecessary write operations
Solution Approach 1:
The patent extracts the storage controller from the data path, eliminating it as an intermediary component. The host system directly manages flash memory devices, removing the controller's garbage collection and wear leveling functions. This extraction eliminates unnecessary write operations performed by the controller, reducing write latency and improving data reliability by ensuring data is written only when actually needed by the host.
Solution Approach 2:
The host system performs storage management functions that were traditionally handled by the storage controller. The host directly allocates data blocks, manages wear leveling, and controls garbage collection on flash memory devices. This self-service approach eliminates redundant processing steps and gives the host direct control over when writes occur, reducing unnecessary writes and improving both reliability and latency.
2Duration of action of stationary object
If storage controllers perform garbage collection and wear leveling, then flash memory lifespan is extended, but data integrity may be compromised due to redundant write operations
Solution Approach 1:
The patent removes the storage controller's garbage collection and wear leveling functions from the system. These functions are transferred to the host system, which performs them more efficiently by having direct visibility into data lifecycle requirements. This eliminates the controller's redundant write operations that could corrupt data, while still extending flash memory lifespan through proper host-managed wear leveling.
Solution Approach 2:
The patent introduces a buffer memory as an intermediary between the host and flash memory devices. This buffer temporarily holds data during write operations, allowing the host to manage wear leveling and garbage collection without performing redundant writes to the flash memory. The buffer mediates between host data requirements and flash memory constraints, protecting data integrity while extending device lifespan.
3Productivity
If non-volatile RAM is used as a buffer, then write performance improves, but system complexity increases
Solution Approach 1:
The patent utilizes the host system's existing RAM memory for buffer operations, giving it multiple functions: application data processing and storage buffering. By leveraging already-present hardware resources, the system achieves improved write performance without adding dedicated buffer hardware, thus avoiding increased system complexity while still providing the performance benefits of volatile buffering.
Data Source
AI summary
Preparing data for deduplication including: generating, by a storage system for a compressed data block, a padded compressed data block by padding the compressed data block to conform to a fixed block size, wherein the fixed block size is greater than a size of the compressed data block; storing, in the storage system, the padded compressed data block beginning at a block boundary of a storage device in the storage system; and performing block-based deduplication on the storage system, wherein the block-based deduplication determines whether the padded compressed data block matches one or more other padded compressed data blocks stored in the storage system.