Shared Multicast Local Identifier Ranges for InfiniBand Scalability
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limitations of traditional network architectures in high-performance computing environments, such as InfiniBand, restrict the scalability and flexibility of virtual machine migration due to complex addressing and routing schemes, leading to performance bottlenecks and challenges in maintaining load-balancing and QoS constraints across multi-subnet fabrics.
Innovation Solution
The implementation of shared multicast local identifier (MLID) ranges in a high-performance computing environment, allowing for dynamic reconfiguration and efficient routing through a global fabric manager that configures a shared MLID range and uses a multicast forwarding table for packet routing, enabling flexible and scalable inter-subnet communication.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional unicast LID addressing is used in InfiniBand subnets, then each virtual HCA instance can be configured with an independent LID, but the 48K LID value space limits the subnet size and scalability
Solution Approach 1:
The patent segments the addressing space by introducing a 128-bit DGID (Destination Global Identifier) that combines a 64-bit subnet prefix with a 64-bit interface identifier. This segmentation allows the LID space to be divided into multiple subnets, each with its own prefix, thereby extending the overall addressable space beyond the 48K limitation of traditional LIDs while maintaining independent addressing within each subnet.
Solution Approach 2:
The patent adds a new dimension to the addressing scheme by introducing the subnet prefix component to the DGID. Instead of relying solely on the 16-bit LID dimension, the system now uses a two-dimensional addressing space: subnet prefix (64 bits) + interface identifier (64 bits). This dimensional expansion enables vastly larger network scales while preserving the original LID functionality within each subnet.
2Adaptability or versatility
If hierarchical mapping structure is used to scale DGID to LID, then more subnets and end nodes can be represented, but the ability to route individual destinations independently is severely limited
Solution Approach 1:
The patent implements dynamic routing by maintaining forwarding tables that can be independently configured for different DGIDs. Rather than using a static hierarchical mapping, the system dynamically routes packets based on the specific DGID in the packet header, allowing independent destination routing even within the hierarchical subnet structure. The forwarding tables can be updated to reflect changing network conditions and routing requirements.
Solution Approach 2:
The patent introduces router nodes as intermediaries between subnets that perform DGID to LID mapping. These routers maintain forwarding tables that map DGIDs to local LIDs, enabling independent routing decisions at each router based on the specific destination. This intermediary layer preserves individual destination routing capability while supporting hierarchical subnet organization.
3Speed
If complex CAM based lookup hardware is used for DGID to LID mapping, then wire speed packet forwarding is achieved, but hardware scalability and cost are constrained
Solution Approach 1:
The patent segments the lookup process by first examining the subnet prefix portion of the DGID to determine the target subnet, then using a simplified second-stage lookup within that subnet. This segmentation allows the use of smaller, more scalable forwarding tables rather than requiring a single large CAM structure to handle all possible DGIDs, reducing hardware complexity while maintaining wire-speed forwarding capability.
Solution Approach 2:
The patent employs dynamic forwarding tables that can be programmatically configured and updated, replacing static CAM hardware with flexible software-configurable lookup structures. This dynamic approach allows the forwarding behavior to be modified without hardware changes, improving scalability and reducing the fixed complexity of CAM-based systems while maintaining high-speed operation through optimized data structures.
Data Source
AI summary
System and method for supporting shared multicast local identifiers (MLIDs) a high performance computing environment. In accordance with an embodiment, a shared MLID range can be configured such that each subnet within a fabric can utilize an MLID within a shared MLID range without the need to utilize a TCAM, or other memory, lookup of a MGID to MLID mapping.