Fabric QoS Throttling for Local Memory Request Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale computing environments, shared memory buffers are often occupied by high-latency or over-utilized memory requests, leading to starvation of local memory requests that could be served faster, due to limited buffer sizes and uneven contention within fabric architectures.
Innovation Solution
Implement a system where network nodes monitor quality of service levels and transmit throttling messages to other nodes to manage resource access, using extended throttling signals across the interconnect fabric to address contention beyond local resources, including features like throttling message generation, propagation, and response execution modules.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If shared buffers are used to handle memory requests from multiple network nodes, then resource utilization is improved, but local memory requests become starved due to buffer occupation by high-latency requests
Solution Approach 1:
The patent implements quality of service (QoS) based throttling that differentiates between local and remote memory requests. Network nodes monitor QoS levels and apply selective throttling to remote requests while prioritizing local requests, ensuring that local memory access performance is maintained while still utilizing shared buffers for overall resource efficiency.
Solution Approach 2:
The system employs QoS monitoring and feedback mechanisms where network nodes continuously monitor the status of shared buffer usage and memory request performance. Based on this feedback, nodes dynamically adjust throttling actions to prevent local request starvation while maintaining high resource utilization, creating a closed-loop control system that balances competing demands.
2Productivity
If shared buffer size is increased to reduce request starvation, then memory request throughput is improved, but buffer occupancy by high-latency requests increases
Solution Approach 1:
The patent implements dynamic QoS-based throttling that adapts buffer usage patterns in real-time. Instead of static buffer allocation, the system dynamically adjusts the injection rate of memory requests based on monitored QoS conditions, allowing the effective buffer occupancy to fluctuate optimally between throughput maximization and starvation prevention.
Solution Approach 2:
The system changes operational parameters by adjusting the memory request injection rate based on QoS monitoring. When high-latency requests occupy excessive buffer space, the system reduces the injection rate to maintain healthy buffer occupancy levels, thereby preventing throughput degradation while avoiding request starvation.
3Ease of operation
If QoS monitoring and throttling messages are implemented across network nodes, then resource access fairness is improved, but system complexity increases
Solution Approach 1:
The patent extends existing fabric interconnect functionality to carry QoS throttling messages alongside normal memory requests. The same network infrastructure and protocol mechanisms are utilized for both data transmission and control signaling, avoiding the need for separate dedicated throttling channels and reducing overall system complexity.
Solution Approach 2:
Network nodes autonomously monitor their own QoS levels and generate appropriate throttling messages without requiring centralized control. Each node independently makes decisions about when to throttle remote requests based on local buffer occupancy and performance metrics, distributing the control intelligence across the system and reducing coordination overhead.
Data Source
Figure 1~2
Figure 3
Figure 4
AI summary
Technologies for quality of service based throttling in a fabric architecture include a network node of a plurality of network nodes interconnected across the fabric architecture via an interconnect fabric. The network node includes a host fabric interface (HFI) configured to facilitate the transmission of data to/from the network node, monitor quality of service levels of resources of the network node used to process and transmit the data, and detect a throttling condition based on a result of the monitored quality of service levels. The HFI is further configured to generate and transmit a throttling message to one or more of the interconnected network nodes in response to having detected a throttling condition. The HFI is additionally configured to receive a throttling message from another of the network nodes and perform a throttling action on one or more of the resources based on the received throttling message. Other embodiments are described herein.