Prefetch Request Routing for Predictable Storage Cache Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing storage systems face performance degradation due to unpredictable latencies in prefetching data from non-volatile memory, leading to increased traffic, reduced throughput, and memory bandwidth consumption, especially when cache buffers and queues become full.
Innovation Solution
A storage device system that includes a host metadata unit to tag data requests as prefetch or regular, and a device metadata unit to route these requests to appropriate processing units, returning data from volatile memory for prefetch hits and a no-data response for misses, thereby maintaining consistent latency and preventing buffer overflow.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If prefetch requests are processed from non-volatile memory, then data availability is improved, but latency becomes unpredictable and buffer queues fill up
Solution Approach 1:
A cache buffer is introduced as an intermediary between non-volatile memory and the processing units. Prefetch requests first populate this cache buffer, which then serves subsequent regular requests. This mediator absorbs the latency variability of non-volatile memory and prevents direct queue filling, resolving the contradiction between data availability and predictable latency.
Solution Approach 2:
Data is prefetched from non-volatile memory in advance and stored in the cache buffer before being requested by processing units. This preliminary action ensures data availability when needed while decoupling the prefetch operation from the critical request-response path, making latency predictable and preventing buffer overflow during actual data access.
2Productivity
If prefetch requests are handled with high priority, then data readiness is improved, but regular request processing is delayed
Solution Approach 1:
The prefetch functionality is extracted as a separate operation that populates a dedicated cache buffer, distinct from the regular request processing path. Prefetch requests are handled independently and do not compete for processing resources with regular requests, eliminating the trade-off between data readiness and regular request processing time.
3Productivity
If cache buffer size is increased to handle prefetch hits, then throughput is improved, but memory bandwidth consumption increases
Solution Approach 1:
The cache buffer is designed with specific local characteristics optimized for prefetch operations. Rather than uniformly increasing all memory resources, a dedicated buffer with appropriate size and allocation strategy is created specifically for prefetch hits. This localized optimization improves throughput for prefetch operations while minimizing impact on overall memory bandwidth consumption.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Systems and methods for generating and processing prefetch requests are disclosed. A processor is configured to: receive from a computing device a first request for first data, the first request for the first data including a first indicia; select a first mode of processing of the first request based on the first indicia; based on selecting the first mode of processing of the first request, transmit the first data to the computing device; receive from the computing device a second request for second data, the second request for second data including a second indicia; select a second mode of processing of the second request based on the second indicia; and based on selecting the second mode of processing of the second request, determine a response type for a second response based on a location of the second data.