Storage Node Memory Controller for Distributed Erasure Coding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed storage systems face inefficiencies in managing data chunks and coding chunks generated through erasure coding, leading to high CPU usage, frequent context switching, and network traffic, due to the need for frequent error correction and storage in shared memory blocks.
Innovation Solution
A memory controller is introduced that differentiates between data chunks and coding chunks, storing them in separate memory blocks with varying error correction codes, prioritizing coding chunks for higher reliability and data chunks for frequent read operations, and optimizing garbage collection and read reclaim operations based on chunk types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data chunks and coding chunks are stored in shared memory blocks, then storage space is optimized, but error correction overhead and CPU usage increase
Solution Approach 1:
The patent segments the storage system into distinct data chunk storage regions and coding chunk storage regions within the memory device. This segmentation allows different error correction strategies to be applied to each region, optimizing both storage efficiency and reliability by preventing error correction operations on coding chunks when only data chunk updates are needed.
2Reliability
If frequent error correction is performed on coding chunks, then data reliability is maintained, but system performance and productivity decrease
Solution Approach 1:
The patent extracts coding chunks from the error correction processing flow. When a write request contains only data chunks, the system performs error correction only on the data chunks and stores them in data chunk storage regions, completely bypassing error correction operations on coding chunks. This extraction principle eliminates unnecessary processing overhead while maintaining data reliability.
3Adaptability or versatility
If context switching is performed frequently to manage different chunk types, then data management flexibility improves, but CPU usage and operational complexity increase
Solution Approach 1:
The patent applies preliminary action by pre-classifying incoming write data into data chunks and coding chunks before storage operations. The memory controller identifies chunk types in advance and routes them to appropriate storage regions, eliminating the need for frequent context switching during write operations. This preliminary classification simplifies the management flow while maintaining flexibility.
4Productivity
If read reclaim operations are performed on all memory blocks, then storage efficiency improves, but operational overhead and time consumption increase
Solution Approach 1:
The patent applies local quality by performing read reclaim operations selectively based on the storage region. Instead of uniformly processing all memory blocks, the system identifies and processes only data chunk storage regions that require read reclaim operations, while leaving coding chunk storage regions untouched. This localized approach reduces operational overhead and time consumption while maintaining storage efficiency.
Data Source
AI summary
Provided herein is a storage node of a distributed storage system and a method of operating the same. A memory controller may include a data controller configured to receive a write request and write data corresponding to the write request from a host, and configured to determine a physical address of a memory block in which the write data is to be stored based on chunk type information included in the write request, a memory control component configured to provide a program command for instructing the memory block to store the write data, the physical address, and the write data to the memory device, wherein the chunk type information is information about whether the write data indicates a type of data chunks or a type of coding chunks, the data chunks and the coding chunks being generated by the host performing an erasure coding operation on original data.