Hybrid Memory Journaling With Parallel Compressed EC Writes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current storage systems face challenges in optimizing compatibility, efficiency, and collaboration due to long data paths and bottlenecks in bandwidth, particularly in journaling and compression operations across storage servers, which affect PCIe and memory bus performance.
Innovation Solution
A hybrid memory device and controller system that integrates a journal drive, data buffer, compression, and erasure code encoding/decoding, allowing parallel write operations and reducing latency by distributing compressed data across multiple channels, thereby enhancing throughput and reducing bandwidth consumption.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is written through long data paths involving multiple servers and network fabrics, then compression and erasure coding can be performed, but bandwidth bottlenecks occur on PCIe and memory buses
Solution Approach 1:
The patent segments the EC codeword into multiple parts and distributes them across different memory devices within the same server. This allows parallel write operations to multiple non-volatile memory devices simultaneously, eliminating the bandwidth bottleneck on PCIe and memory buses while maintaining data integrity through distributed storage of encoded data parts.
Solution Approach 2:
The patent merges the journaling function and data storage function into a single hybrid memory device that contains both volatile memory (for journal buffer) and non-volatile memory (for data storage). This integration eliminates the need for separate data paths to external journal drives, reducing bandwidth consumption on PCIe buses while maintaining reliable journaling and compression capabilities.
2Loss of time
If a fast storage medium is used as journal drive, then access latency is reduced, but device complexity increases with separate journal and data drives
Solution Approach 1:
The patent combines the journal drive and data drive into a single hybrid memory device that integrates both volatile memory (for journal buffer) and non-volatile memory (for data storage). This unified architecture reduces device complexity by eliminating separate journal and data drives while maintaining fast access latency through the volatile memory component for journaling operations.
Solution Approach 2:
The hybrid memory device performs multiple functions: it serves as both a journal buffer (using volatile memory) and a data storage device (using non-volatile memory). This multi-functional design eliminates the need for separate specialized drives, reducing system complexity while maintaining the performance benefits of fast journaling and reliable data storage.
3Ease of operation
If data is temporarily held in DIMM buffer, then data format conversion can be performed, but long data paths increase bandwidth consumption
Solution Approach 1:
The patent merges the data buffer function with the non-volatile memory device by integrating a buffer memory directly within the hybrid memory device. This allows data format conversion and processing to occur locally within the same device that stores the data, eliminating the need for long data paths across the memory bus and reducing bandwidth consumption while maintaining ease of operation for format conversion.
Data Source
AI summary
One embodiment provides a system which facilitates data management. During operation, the system receives, by a first memory device, data to be written to a first non-volatile memory of the first memory device and to a second non-volatile memory of a storage drive distinct from the first memory device. The system performs, by the first memory device on the received data, a compression operation and erasure code (EC)-encoding to obtain a compressed EC codeword. The system initiates a first write operation and a second write operation in parallel, wherein the first write operation comprises writing a first part of the compressed EC codeword to the first non-volatile memory, and wherein the second write operation comprises writing the first part of the compressed EC codeword to the second non-volatile memory.