Fabric-Attached Memory Work Queues for Low-Latency Submission
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed computing systems using Fabric Attached Memory (FAM) devices experience higher latency due to the additional protocol steps involved in accessing data over a system-wide fabric interconnect, which is not addressed by existing technologies.
Innovation Solution
Implementing a mechanism within the FAM device that includes a memory controller with command manager (CM) logic circuitry to route work packets to dedicated computing resources, using CM work queues and destination work queues to minimize latency by allowing a single request submission and ensuring space availability before initiating work, thereby reducing the need for resubmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is accessed over a system-wide fabric interconnect in distributed computing systems, then memory capacity and system scalability are improved, but latency increases due to additional protocol steps
Solution Approach 1:
The patent segments the work submission process into two distinct queues: a first work queue for receiving work packets and a second work queue for processing. This segmentation allows the system to decouple the arrival of work packets from their processing, enabling efficient handling of fabric interconnect communications without blocking the processing pipeline, thereby reducing latency while maintaining scalability.
Solution Approach 2:
The patent implements preliminary action by checking whether the second work queue is full before transferring work packets from the first work queue. This proactive check prevents blocking and resubmission delays, allowing the system to anticipate queue status and manage work flow in advance, thereby reducing latency caused by protocol steps in fabric interconnect access.
2Productivity
If multiple work packets are processed through fabric interconnect, then system productivity increases, but latency accumulates due to repeated protocol steps
Solution Approach 1:
The patent ensures continuity of useful action by maintaining two concurrent work queues that operate independently. While the first work queue receives new work packets continuously, the second work queue processes packets without interruption. This continuous operation eliminates idle time and protocol re-handshaking, thereby maintaining high productivity while reducing cumulative latency effects.
Solution Approach 2:
By checking queue status in advance before transfer operations, the system performs preliminary actions that prevent work packet blocking. This ensures that productivity is maintained by avoiding resubmission delays, while latency is reduced through proactive queue management that eliminates waiting time for protocol completion.
3Speed
If work packets are submitted without checking queue space availability, then submission speed increases, but resubmission overhead increases due to full queues
Solution Approach 1:
The patent applies preliminary action by checking whether the second work queue has available space before transferring work packets from the first work queue. This advance check prevents the scenario where packets are submitted into full queues, eliminating the need for resubmission and associated overhead. The submission speed remains high because the check is performed efficiently, while resubmission overhead is eliminated by preventing full-queue conditions proactively.
Solution Approach 2:
The system implements feedback by monitoring the status of the second work queue and using this information to control transfers from the first work queue. This feedback mechanism ensures that submission speed is optimized by only transferring packets when space is available, thereby preventing resubmission overhead while maintaining high throughput and low latency.
Data Source
AI summary
A method performed by a distributed computing system includes receiving a work packet from a separate computing device via a fabric interconnect at a command manager (CM) of a memory controller of a fabric attached memory (FAM) device, wherein the work packet includes a memory access to be performed by a FAM computing resource local to the FAM device; determining a work class of the work packet; placing the work packet in a CM work queue local to the CM for the work class when space is available in the CM work queue for the work class; and when the CM work queue for the work class is full, placing the work packet in a destination work queue according to an address included in the work packet, wherein the destination queue is implemented in a memory array of the FAM device external to the memory controller.