Cache Arrangement for Reducing Interconnect Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In systems with limited or shared interconnects between cache memory and primary memory, data pre-fetch requests can increase latency, particularly when accessing memory on a different die within a package, leading to decreased CPU performance due to increased memory latency and 'head of line blocking' in traditional monolithic system-on-chip designs.
Innovation Solution
Implementing a cache arrangement where data associated with memory requests is allocated in one cache memory but not the other, with the second cache arrangement performing speculative pre-fetches and allocating memory space for pre-fetched data, thereby reducing interconnect usage for speculative requests and optimizing data allocation based on quality of service and spatial locality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data pre-fetch requests are implemented to increase memory access speed, then data retrieval speed is improved, but latency in the interconnect is increased
Solution Approach 1:
The cache system is divided into two separate cache memories (first cache memory and second cache memory) with distinct allocation policies. The second cache memory handles pre-fetch requests and allocates space for pre-fetched data, while the first cache memory handles regular memory requests. This segmentation allows pre-fetch operations to occur independently without blocking the interconnect for regular data access, thus improving overall data retrieval speed while minimizing interconnect latency.
2Speed
If pre-fetch requests are sent to increase memory access speed, then overall memory access speed is improved, but CPU performance is decreased due to head of line blocking
Solution Approach 1:
By segmenting the cache into two independent memories with separate controllers, pre-fetch operations in the second cache memory do not block the interconnect or affect CPU access to the first cache memory. This eliminates head-of-line blocking where pre-fetch requests would otherwise force the CPU to wait, thereby maintaining both fast memory access and high CPU performance.
Solution Approach 2:
The second cache memory acts as an intermediary that handles speculative pre-fetch requests independently. It receives pre-fetch requests, allocates space for pre-fetched data, and returns results without blocking the main memory interconnect. This intermediary structure allows the system to benefit from pre-fetching while preventing it from degrading CPU performance through blocking.
3Speed
If cache memory is used to store pre-fetched data, then data retrieval speed is improved, but interconnect resources are overutilized
Solution Approach 1:
The cache system is divided into two separate cache memories (first cache memory and second cache memory) with distinct allocation policies. The second cache memory handles pre-fetch requests and allocates space for pre-fetched data, while the first cache memory handles regular memory requests. This segmentation allows pre-fetch operations to occur independently without blocking the interconnect for regular data access, thus improving overall data retrieval speed while minimizing interconnect latency.
Data Source
AI summary
A first cache arrangement including an input configured to receive a memory request from a second cache arrangement; a first cache memory for storing data; an output configured to provide a response to the memory request for the second cache arrangement; and a first cache controller; the first cache controller configured such that for the response to the memory request output by the output, the cache memory includes no allocation for data associated with the memory request.