Partitioning Network Switches for Scalable InfiniBand Subnets
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The limitations of traditional InfiniBand networks in supporting large-scale cloud computing environments, particularly due to the 48K unicast LID value space constraint, which hinders the scalability and flexibility in routing and load-balancing across multiple subnets, leading to performance bottlenecks and administrative challenges.
Innovation Solution
The implementation of a system and method that partitions network switches into domains, utilizing prepopulated and dynamically assigned LIDs within a vSwitch architecture, allowing for flexible forwarding domain boundaries and efficient routing across subnets through Linear Forwarding Tables (LFTs) and Global Route Headers (GRHs), enabling scalable and deterministic packet forwarding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If hierarchical mapping structure is used to scale the number of subnets and end nodes, then the total number of representable subnets and end nodes increases, but the ability to route individual destinations independently is severely limited
Solution Approach 1:
The patent segments the forwarding domain by introducing forwarding domain boundaries that divide the network into multiple independent forwarding domains. Each domain can perform independent routing lookups using Linear Forwarding Tables (LFTs), allowing individual destination routing to be maintained even as the total number of subnets and end nodes scales across multiple domains. This segmentation resolves the contradiction by enabling both large-scale representation and independent routing capability simultaneously.
2Measurement precision
If CAM based lookup hardware is used for mapping 128 bit DGID values to 16 bit DLID values, then routing accuracy is improved, but hardware scalability is limited
Solution Approach 1:
The patent replaces the mechanical CAM-based lookup hardware with a software-based Linear Forwarding Table (LFT) implementation. The LFT uses simple linear indexing instead of complex CAM hardware, achieving the same routing accuracy through software algorithms. This substitution eliminates the hardware scalability limitations while maintaining precise DGID to DLID mapping capability.
3Ease of operation
If a single subnet is used to simplify network management, then administrative complexity is reduced, but the 48K LID value space constraint limits network scalability
Solution Approach 1:
The patent segments the large network into multiple smaller forwarding domains, each with its own LID value space. This allows the network to scale beyond the 48K LID constraint of a single subnet while maintaining simplified management within each domain. The forwarding domain boundaries create independent management zones that can be administered separately, resolving the contradiction between management simplicity and scalability.
Solution Approach 2:
The patent introduces forwarding domain boundaries as intermediaries between different LID value spaces. These boundaries enable communication and routing between multiple subnets while presenting a unified management interface. The intermediary layer handles the complexity of cross-domain routing, allowing each domain to maintain simple local management while supporting large-scale network connectivity.
Data Source
AI summary
System and method for supporting flexible forwarding domain boundaries in a high performance computing environment. In accordance with an embodiment, flexible forwarding domain boundaries can be supported by dividing/partitioning a physical switch into two or more logical switches, where each switch is logically in a different domain, and allowing a fabric to be decomposed into independent subnets with one two or more physical end ports at the physical switch. By doing so, the same hierarchical forwarding structure and management structure between subnets can be provided as when complete physical switches are used as building blocks.