Clustered Memory Resiliency via Erasure Coding and Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Clustered memory systems face higher latency and lower bandwidth compared to local memory, and are susceptible to failures that can cause computing devices to crash, as they lack redundancy and efficient resource management.
Innovation Solution
Implementing redundancy through replication and erasure coding techniques, such as internal and external erasure coding, to ensure data availability and improve latency and bandwidth constraints, while also using management hosts to monitor and manage memory allocations across a network environment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in clustered memory servers, then memory capacity is improved, but latency increases and bandwidth decreases compared to local memory
Solution Approach 1:
The patent segments memory resources by dividing clustered memory into multiple chunks and distributing them across different memory servers. Each computing device is assigned specific chunks, allowing direct access to frequently used data while maintaining the ability to access additional capacity through the network. This segmentation enables the system to achieve both increased capacity and reduced latency by keeping hot data in local or nearby memory chunks.
Solution Approach 2:
The patent introduces a hierarchical memory architecture that adds a network dimension to traditional local memory. By organizing memory in layers (local memory, clustered memory, distributed memory across servers), the system provides multiple access paths and dimensions for data retrieval, allowing optimization between capacity and speed based on workload requirements.
2Quantity of substance
If clustered memory is used to increase memory capacity, then memory availability is improved, but reliability decreases due to susceptibility to server failures
Solution Approach 1:
The patent implements redundancy by maintaining multiple copies of memory chunks across different memory servers before failures occur. When a memory server fails, the system can immediately retrieve data from redundant copies without interruption to computing devices. This beforehand cushioning through replication ensures high availability and reliability while maintaining expanded memory capacity.
Solution Approach 2:
The patent creates and maintains multiple copies of memory data across different servers in the cluster. Each memory chunk is replicated to ensure that if one server fails, the data remains accessible from other servers. This copying strategy provides both capacity expansion and fault tolerance, resolving the contradiction between availability and reliability.
3Quantity of substance
If memory servers are added to expand memory capacity, then memory resource pool is improved, but system complexity increases
Solution Approach 1:
The patent creates a universal memory management system where memory servers can serve multiple computing devices simultaneously, and memory chunks can be dynamically allocated and reallocated based on demand. This multi-functionality allows the same infrastructure to handle various workloads and scaling scenarios without proportionally increasing management complexity, as the system adapts to different configurations through standardized protocols.
Data Source
AI summary
Disclosed are various embodiments for improving the resiliency and performance of clustered memory. A computing device can generate at least one parity page from at least a first local page and a second local page. The computing device can then submit a first write request for the first local page to a first one of a plurality of memory hosts. The computing device can also submit a second write request for the second local page to a second one of the plurality of memory hosts. Additionally, the computing device can submit a third write request for the parity page to a third one of the plurality of memory hosts.