Distributed Cache Prefetch via Programmable Switch
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional memory access prediction and data prefetch techniques are not effective in distributed cache systems due to high network latencies, limiting the use of Storage Class Memories (SCMs) despite their lower power consumption and cost, as they have longer data access latencies compared to DRAM or SRAM.
Innovation Solution
Implementing a distributed cache system with a programmable switch and a centralized memory prefetch prediction host that uses cache miss data to predict and prefetch cache lines across multiple clients, reducing network-wide data access latency and enabling the use of less expensive and power-efficient SCMs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If Storage Class Memories (SCMs) are used in distributed caches, then power consumption and cost are reduced, but data access latency increases
Solution Approach 1:
The system performs preliminary actions by predicting future memory access patterns and prefetching data before it is actually needed. The memory access predictor analyzes historical access patterns and generates prefetch requests, so that when data is needed, it is already in the distributed cache, eliminating the latency penalty of SCM access.
Solution Approach 2:
The patent replaces conventional fixed-function networking with Software Defined Networking (SDN) to substitute software-based control for hardware-based fixed routing. This enables dynamic optimization of data paths and allows the network to adapt to memory access patterns, reducing network latency to levels comparable with traditional memory systems.
2Device complexity
If conventional fixed-function networking is used, then network infrastructure is simple, but data transfer latency is hundreds of microseconds
Solution Approach 1:
The patent replaces conventional fixed-function networking hardware with Software Defined Networking (SDN), substituting software-based control planes for hardwired routing logic. This enables dynamic path selection, quality of service management, and adaptation to workloads, reducing data transfer latency from hundreds of microseconds to levels comparable with traditional memory access.
3Productivity
If memory access prediction and data prefetch are implemented, then CPU performance bottleneck is reduced, but system complexity increases
Solution Approach 1:
The patent segments the memory hierarchy into multiple distributed cache layers across the network, with each node having its own cache. The memory access predictor is also segmented and distributed across multiple nodes rather than centralized, allowing parallel operation and reducing the complexity burden on any single component.
4Quantity of substance
If distributed cache is implemented over network, then storage capacity and cost-effectiveness improve, but network latency becomes the performance bottleneck
Solution Approach 1:
The system performs preliminary actions by predicting future memory access patterns and prefetching data before it is actually needed. The memory access predictor analyzes historical access patterns and generates prefetch requests, so that when data is needed, it is already in the distributed cache, eliminating the latency penalty of network access.
Data Source
AI summary
A programmable switch receives a cache line request from a client of a plurality of clients on a network to obtain a cache line. One or more additional cache lines are identified based on the received cache line request and prefetch information. The cache line and the one or more additional cache lines are requested from one or more memory devices on the network. The requested cache line and the one or more additional cache lines are received from the one or more memory devices, and are sent to the client.