Erasure Encoding for Shared Memory Fault Tolerance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for fault tolerance in large-scale systems, such as supercomputers and data centers, are impractical due to the increased number of components that can fail and the frequency of unrecoverable message errors, leading to performance degradation and difficulty in maintaining data coherence as the system size increases.
Innovation Solution
The implementation of erasure encoding and decoding of cache lines in blast zones, where each zone includes active and spare nodes for data and parity portions, allowing for efficient reconstruction of failed nodes and maintaining coherence through serialization of memory requests and tracking of cache line states.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checkpointing by periodically copying entire memory to storage device is used for fault tolerance, then data recovery capability is improved, but system operation is halted frequently and performance deteriorates
Solution Approach 1:
The patent divides the memory system into multiple independent memory devices, each capable of autonomous fault tolerance operations through erasure coding. This segmentation allows local recovery without halting entire system operations, resolving the contradiction between reliability and productivity by enabling continuous system operation while maintaining data recovery capability at the device level.
Solution Approach 2:
The patent implements erasure coding that encodes data with redundancy information in advance, before failures occur. This preliminary encoding allows immediate reconstruction of failed data from remaining redundant copies without requiring full memory checkpointing, thus maintaining both high reliability and continuous system performance.
2Reliability
If redundant copies of data are stored in memory and storage devices for handling physical failures, then fault tolerance is improved, but storage overhead increases
Solution Approach 1:
The patent changes the redundancy parameter from storing complete duplicate copies to storing erasure-coded portions where k data portions and m parity portions can reconstruct any failed portion. This parameter change achieves the required fault tolerance with reduced storage overhead, as only m/(k+m) of the total storage is redundant rather than full copies.
3Reliability
If memory coherence protocol is used to ensure data consistency across multiple processors, then data coherence is improved, but communication complexity and system difficulty increase
Solution Approach 1:
The patent segments the coherence management into independent memory device units, each maintaining its own coherence state through serialization of memory requests. This segmentation reduces overall communication complexity by allowing parallel coherence management at the device level rather than requiring centralized coordination across all processors and memory devices.
Solution Approach 2:
Each memory device autonomously manages its own coherence and fault tolerance through internal erasure coding and request serialization, without requiring complex inter-device communication protocols. This self-service approach simplifies the overall system by eliminating the need for sophisticated distributed coherence protocols.
Data Source
AI summary
A system includes at least one memory controller that partitions at least one memory into a plurality of nodes. Blast zones are formed that each include a predetermined number of nodes. Cache lines are erasure encoded to be stored in one or more blast zones with at least two nodes in a blast zone storing respective portions of a cache line and at least one node in the blast zone storing a parity portion. In one aspect, it is determined that data stored in one or more nodes of a blast zone needs to be reconstructed and stored in one or more spare nodes designated to replace the one or more nodes. Erasure decoding is performed using data from one or more other nodes in the blast zone to reconstruct the data for storage in the one or more spare nodes.