Programmable Switch Memory Pooling for Cluster Swap Bottlenecks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commodity servers often face insufficient memory capacity, leading to inefficient swapping to local storage, which is slow and can cause thrashing issues, and over-provisioning local memory is costly and energy-inefficient.
Innovation Solution
Implement programmable networking devices such as switches, SmartNICs, IPUs, and DPUs to manage memory dynamically across a cluster, enabling transparent access to remote memory resources, balancing memory usage and improving performance and reliability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If local memory capacity is increased to avoid swapping, then memory performance is improved, but system cost and energy consumption increase
Solution Approach 1:
The patent extends memory access from the local dimension to the network dimension by enabling servers to access remote memory resources through the network infrastructure. This transforms the memory architecture from purely local to distributed across multiple nodes, allowing memory capacity expansion without proportionally increasing local hardware at each server.
Solution Approach 2:
The network switches are enhanced to perform multiple functions: traditional packet switching plus memory management operations. The switches can allocate, track, and manage memory resources across the cluster, enabling them to serve both networking and memory management roles, thereby reducing the need for dedicated memory hardware at each server.
2Reliability
If local memory capacity is increased to prevent swapping, then system reliability is improved, but energy consumption increases
Solution Approach 1:
The system moves memory resources from local physical dimension to distributed network dimension, allowing dynamic allocation of memory capacity across the cluster. This enables the system to maintain stability through remote memory access rather than relying on oversized local memory that would consume excessive energy.
3Quantity of substance
If swapping to local storage is used when memory is insufficient, then memory capacity limitation is bypassed, but system performance deteriorates due to slow access speed
Solution Approach 1:
The network switch acts as an intermediary between servers and remote memory resources, managing memory allocation and access requests. This intermediary enables fast memory access by directly routing requests to remote memory locations rather than forcing slow storage-based swapping, while still providing access to expanded memory capacity.
4Reliability
If customized systems with increased memory capacity are built, then memory performance is improved, but manufacturing cost increases
Solution Approach 1:
The patent merges memory resources across multiple servers into a unified distributed memory pool. By combining unused or underutilized memory from various nodes in the cluster, the system achieves increased effective memory capacity without requiring each server to be customized with expensive high-capacity memory, thereby reducing overall manufacturing costs.
Data Source
AI summary
Methods, apparatus, and systems for adaptive collaborative memory with the assistance of programmable networking devices. Under one example, the programmable networking device is a switch that is deployed in a system or cluster of servers comprising a plurality of nodes. The switch selects one or more nodes to be remote memory server nodes and allocate one or more portions of memory on those nodes to be used as remote memory for one or more remote memory client nodes. The switch receives memory access request messages originating from remote memory client nodes containing indicia identifying memory to be accessed, determines which remote memory server node is to be used for servicing a given memory access request, and sends a memory access request message containing indicia identifying memory to be accessed to the remote memory server node that is determined. The switch also facilitates return of messages containing remote memory access responses to the client nodes.