Memory-Side Cache Queues for Host-Preferred CXL Request Handling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Managing the complexity of resource contention between Compute Express Link (CXL) memory requests and accelerator requests in memory systems, where CXL requests must make forward progress independently of other device activities, is a challenge.
Innovation Solution
Implementing separate processing queues for CXL (external) and accelerator (internal) requests, ensuring priority to CXL requests by maintaining host-preferred memory operations through distinct host and internal request queues within the memory controller.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If separate processing queues are implemented for CXL and accelerator requests, then forward progress of CXL requests is ensured, but device complexity increases
Solution Approach 1:
The memory controller is segmented into separate processing queues: a first queue for CXL memory requests and a second queue for accelerator requests. This segmentation isolates CXL request handling from accelerator activities, ensuring that CXL requests make forward progress independently without being blocked by accelerator operations, thereby resolving the contradiction between reliability and complexity.
Solution Approach 2:
A queue management mechanism acts as an intermediary between CXL requests and accelerator requests. The memory controller uses this intermediary structure to prioritize and schedule CXL requests separately, ensuring their forward progress while still allowing accelerator operations to proceed without direct interference, thus managing the complexity of handling multiple request types.
2Reliability
If CXL requests are prioritized over accelerator requests, then CXL forward progress is maintained, but accelerator performance may be degraded
Solution Approach 1:
By segmenting request processing into separate queues, the system guarantees CXL protocol compliance through dedicated queue handling while simultaneously preserving accelerator throughput. The accelerator queue operates independently, allowing high-priority CXL requests to be serviced without completely blocking accelerator access to memory resources.
Solution Approach 2:
The system applies partial prioritization to CXL requests rather than complete preemption. While CXL requests receive priority treatment to ensure protocol compliance, the accelerator queue maintains sufficient bandwidth and access opportunities to preserve overall system productivity and accelerator performance.
3Device complexity
If unified request handling is used for CXL and accelerator requests, then hardware operations are simplified, but resource contention increases
Solution Approach 1:
The patent introduces segmentation into request handling by creating separate queues for CXL and accelerator requests. This segmentation reduces resource contention by preventing accelerator requests from blocking CXL requests, thereby improving reliability of CXL protocol compliance while maintaining relatively simple hardware operations through structured queue management.
Data Source
AI summary
System and techniques for host-preferred memory operation are described herein. At a memory-side cache of a memory device that includes accelerator hardware, a first memory operation can be received from a host. A determination that the first memory operation corresponds to a cache set based on an address of the first memory operation is made. A second memory operation can be received from the accelerator hardware. Another determination can be made that the second memory operation corresponds to the cache set. Here, the first memory operation can be enqueued in a host queue of the cache set and the second memory operation can be enqueued in an internal request queue of the cache set. The first memory operation and the second memory operation can be executed as each is dequeued.