NTB Link Arbitration for Storage Software Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computing systems with non-transparent bridge (NTB) links, unbalanced usage between host IO devices and storage software leads to inefficient utilization of NTB resources, resulting in overloading of certain links while underutilizing others, which can impact performance and reliability.
Innovation Solution
Implementing an NTB arbitration system that monitors usage patterns and selects NTB links for storage software operations based on exponential moving average usage values to balance the load across available links, ensuring that host IO device operations are prioritized on specific links while distributing storage software operations across all links.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If storage software uses any available NTB link without arbitration, then storage software can access peer memory efficiently, but NTB link usage becomes unbalanced causing overloading of certain links while underutilizing others
Solution Approach 1:
The patent introduces an NTB arbitration system as an intermediary component that sits between the storage software and the NTB links. This arbitration system monitors usage patterns of all NTB links and dynamically selects appropriate links for storage software operations, preventing any single link from becoming overloaded while ensuring balanced utilization across the available NTB link resources.
Solution Approach 2:
The NTB arbitration system implements feedback mechanisms by monitoring usage patterns of NTB links in real-time. Based on this feedback information about current link utilization, the arbitration system dynamically adjusts its link selection decisions, directing storage software operations to underutilized links and avoiding overloaded ones, thereby maintaining balanced usage across all NTB links.
2Ease of operation
If host IO devices are restricted to a specific subset of NTB links, then host IO device operations are prioritized and simplified, but the subset of NTB links becomes a bottleneck limiting overall system throughput
Solution Approach 1:
The patent segments the NTB link resources into two distinct subsets: one subset dedicated to host IO devices and another subset available to storage software. This segmentation allows host IO devices to operate with simplified, deterministic access patterns on their designated links while the arbitration system manages storage software access to both subsets, thereby preventing host IO operations from becoming a bottleneck while maintaining operational simplicity.
Solution Approach 2:
The system implements dynamic resource allocation where the availability of NTB links to storage software changes based on current usage patterns. When host IO devices are utilizing their dedicated subset of links, storage software can access both subsets, maximizing throughput. The arbitration system dynamically adjusts link allocation decisions based on real-time monitoring of host IO device activity and storage software requests.
Data Source
AI summary
A storage engine has a pair of compute nodes, each compute node having a separate PCIe root complex and attached memory. The PCIe root complexes are interconnected by multiple Non-Transparent Bridge (NTB) links. The NTB resources are unequally shared, such that host IO devices are required to use a first subset of the NTB links to implement memory access operations on the memory of the peer compute node, whereas storage software memory access operations are able to be implemented on all of the NTB links. A NTB link arbitration system arbitrates usage of the first and second subsets of NTB links by the storage software, to distribute subsets of the storage software memory access operations on peer memory to the first and second subsets of NTB links, while causing all host IO device memory access operations on peer memory to be implemented on the first set of NTB links.