Memory-Side Cache Queueing for CXL Request Priority
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing memory systems face complexity in managing prioritization and ordering of CXL (Compute Express Link) memory requests over other internal requests, leading to potential blocking and inefficiencies in resource contention.
Innovation Solution
Implementing separate processing queues for deferred requests, using linked lists and directory-based storage structures to maintain priority of CXL requests while ensuring in-order processing and avoiding internal request blocking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If separate processing queues are implemented for CXL and internal requests, then CXL request priority and system performance are improved, but device complexity increases
Solution Approach 1:
The patent divides the request processing system into separate queues: an external request queue for CXL requests and an internal request queue for internal requests. This segmentation allows independent management and prioritization of different request types, resolving the contradiction by organizing complexity into manageable segments while maintaining high productivity for external requests.
Solution Approach 2:
The patent introduces a request queue management mechanism as an intermediary between request reception and processing. This intermediary layer (the queue system with associated control logic) mediates between external and internal requests, ensuring CXL requests are prioritized while still allowing internal requests to be processed, thus improving external request efficiency without completely blocking internal operations.
2Loss of time
If CXL requests are prioritized over internal requests, then external request latency is reduced, but internal request processing may be delayed
Solution Approach 1:
The patent implements dynamic queue management where the processing priority can change based on queue states. When the external request queue is empty, internal requests can be processed; when external requests are present, they receive priority. This dynamic adjustment resolves the contradiction by allowing internal request processing time to vary based on external request conditions, reducing external latency while minimizing impact on internal requests.
Solution Approach 2:
The patent changes the processing parameter (priority level) based on the type of request and queue state. External requests are assigned higher priority parameters when present, while internal requests use lower priority parameters. This parameter change mechanism allows the system to optimize for external request latency while ensuring internal requests eventually get processed, balancing the trade-off between the two competing requirements.
3Reliability
If linked lists and directory-based storage structures are used, then request ordering and data integrity are maintained, but memory usage and device complexity increase
Solution Approach 1:
The patent uses linked lists where each node contains a copy of essential request information (pointer, tag, etc.). This copying approach maintains request ordering and data integrity through the linked list structure while using only essential information copies rather than full request duplications, thus achieving reliability with controlled memory overhead.
Solution Approach 2:
The patent organizes request storage in a multi-dimensional structure using directory-based indexing combined with linked lists. Instead of a simple linear structure, it adds dimensional organization through directory entries that point to linked list heads, enabling efficient ordering and retrieval while managing memory usage through hierarchical organization rather than flat storage.
Data Source
AI summary
System and techniques for memory side cache request handling are described herein. When a memory request is received, a cache set for the memory request is determined. Here, the cache set has multiple ways and each way corresponds to a cache line. It can be detected that a way of the multiple ways is not ready for the memory request. In this case, a representation of the memory request is stored in a queue of multiple queues based on an interface upon which the memory request was received and the present ways of the cache set. Entries from the multiple queues can be dequeued in a defined order to determine a next memory request to process. The defined order gives priority to memory requests for a present way and then for external over internal requests.