Software-defined coherent caching for disaggregated memory latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Disaggregated memory architectures face increased latency issues due to the remote access of memory resources, which limits performance and scalability in hyperscale computing environments.
Innovation Solution
Implementing software-defined coherent caching policies that allow for the pinning down of large data structures from remote disaggregated memory to a local cache within the same coherent domain as the processors, utilizing expanded Network Interface Controller (NIC) capabilities and programmable logic to manage cache eviction and caching decisions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If memory resources are disaggregated across multiple nodes, then memory capacity and manageability are improved, but access latency increases significantly
Solution Approach 1:
The patent segments memory resources into disaggregated memory pools while implementing software-defined caching layers that divide hot data access from cold data storage. This allows the system to maintain large disaggregated memory capacity while providing low-latency access to frequently accessed data through distributed caching mechanisms.
Solution Approach 2:
The patent introduces software-defined caching policies and cache management intermediaries between the processor and disaggregated memory resources. These intermediaries intercept memory access requests, determine whether data should be cached based on configurable policies, and manage cache coherence, thereby reducing access latency for hot data while maintaining access to the full disaggregated memory capacity.
2Speed
If CPU caches are used to cache data locally, then access speed is improved, but cache capacity is limited and scalability is restricted
Solution Approach 1:
The patent extends the caching hierarchy from traditional CPU-attached caches to a distributed software-defined caching layer that operates across multiple nodes and can leverage disaggregated memory resources. This dimensional extension allows the system to provide cache-like performance for large capacities by distributing cache functionality across the disaggregated memory infrastructure rather than being constrained by individual CPU cache sizes.
3Adaptability or versatility
If remote memory access is implemented, then memory scalability is improved, but performance degrades due to high latency
Solution Approach 1:
The patent implements preliminary action by proactively caching frequently accessed data (hot data) in software-defined caches before actual access occurs. The configurable caching policies predict which data will be needed and pre-position it in faster storage locations, thereby eliminating the latency penalty of remote memory access while maintaining the scalability benefits of disaggregated memory architecture.
Data Source
AI summary
Methods and apparatus for software-defined coherent caching of pooled memory. The pooled memory is implemented in an environment having a disaggregated architecture where compute resources such as compute platforms are connected to disaggregated memory via a network or fabric. Software-defined caching policies are implemented in hardware in a processor SoC or discrete device such as a Network Interface Controller (NIC) by programming logic in an FPGA or accelerator on the SoC or discrete device. The programmed logic is configured to implement software-defined caching policies in hardware for effecting disaggregated memory (DM) caching in an associated DM cache of at least a portion of an address space allocated for the software application in the disaggregated memory. In connection with DM cache operations, such as cache lines evicted from a CPU, logic implemented in hardware determines whether a cache line in a DM cache is to be evicted and implements the software-defined caching policy for the DM cache including associated memory coherency operations.