Fabric Manager Architecture for PCIe Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing PCIe-based architectures face challenges in scaling to support large numbers of processors and fabrics, with limitations in interrupt mechanisms and memory mapping, leading to inefficiencies in data throughput and management.
Innovation Solution
The introduction of a Fabric Manager architecture that includes a Master entity and Node/ Switch Minions for managing fabric address spaces, along with a route naming protocol and the use of Non-Transparent Bridge technology to export Message Signaled Interrupts, enabling efficient data routing and scaling across multiple hosts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If existing PCIe-based architectures are used, then current data bus operations are maintained, but scaling to support large numbers of processors and fabrics is limited
Solution Approach 1:
The patent segments the fabric management functionality into distinct components: Fabric Managers (FMs) that manage specific fabric address spaces, and Non-Transparent Bridges (NTBs) that provide controlled access between fabrics. This segmentation allows each component to handle specific tasks independently, enabling the system to scale to support large numbers of processors and fabrics without overwhelming complexity in the overall architecture.
2Productivity
If Non-Transparent Bridge technology is introduced, then efficient data routing and scaling across multiple hosts is enabled, but interrupt mechanisms and memory mapping face limitations
Solution Approach 1:
The patent introduces Fabric Managers as intermediary entities between NTBs and the interrupt/memory mapping mechanisms. The FMs handle the complex tasks of address space management and interrupt routing, allowing NTBs to focus on efficient data routing. This intermediary layer shields the interrupt mechanisms from the complexities of multi-fabric management while maintaining reliable interrupt delivery across fabric boundaries.
3Ease of operation
If Fabric Manager architecture with Master and Minions is implemented, then fabric address space management is improved, but system overhead increases
Solution Approach 1:
The Fabric Manager is segmented into a Master FM that performs high-level coordination and Minion FMs that handle specific fabric address space management tasks. This segmentation distributes the management overhead across multiple specialized components rather than concentrating it in a single entity, improving ease of operation for address space management while managing system overhead through functional distribution.
Solution Approach 2:
The Minion FMs are designed to autonomously manage their specific fabric address spaces with minimal intervention from the Master FM. Each Minion FM independently handles address translation, allocation, and management for its designated fabric, providing self-service capabilities that reduce the coordination overhead and simplify the overall management architecture.
4Speed
If route naming protocol is implemented, then data routing efficiency is improved, but protocol complexity increases
Solution Approach 1:
The route naming protocol uses simplified parameter-based routing where routes are identified by changing key parameters such as fabric identifiers and address space descriptors rather than complex hierarchical names. This parameter change approach maintains high routing speed by enabling quick route identification and switching while reducing protocol complexity through the use of straightforward, structured route descriptors that are easier to parse and manage.
Data Source
AI summary
Methods and apparatus for efficient scaling of fabric architectures such as those based on PCIe technology, including up to very large fabrics and numbers of hosts/devices for use in ultra-high performance applications such as for example data centers and computing clusters. In one aspect, methods and apparatus for using Non-Transparent Bridge (NTB) technology to export Message Signaled Interrupts (MSIs) to external hosts are described. In a further aspect, an IO Virtual Address (IOVA) space is created is used as a method of sharing an address space between hosts, including across the foregoing NTB(s). Additionally, a Fabric Manager (FM) entity is disclosed and utilized for programming e.g., PCIe switch hardware to effect a desired host/fabric configuration.