GID Switching for InfiniBand Subnet Scalability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The current InfiniBand architecture is limited to approximately 48,000 nodes per subnet, restricting the scalability of high-performance computing clusters, which is insufficient for larger applications.

Innovation Solution

Implementing GID switching in an InfiniBand fabric, where a subnet manager divides the subnet into multiple LID domains and employs GID capable switches to translate between LIDs and GIDs, enabling interconnection of more than 48,000 nodes by using GID-based routing between LID domains.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If the current InfiniBand architecture specification is used, then the subnet can support up to 48,000 nodes, but the scalability is restricted and cannot support larger computing clusters

Engineering Contradiction:

Improvenumber of nodesVSAvoidscalability

Core Design Contradiction:

Quantity of substanceVSAdaptability or versatility

Solution Approach 1:

The patent divides the large-scale network into multiple subnets, each supporting up to 48,000 nodes. By segmenting the overall network architecture into manageable subnets that can be interconnected, the system achieves scalability beyond the single subnet limit while maintaining the performance characteristics of individual subnets.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces subnet border routers as intermediary devices that connect multiple InfiniBand subnets. These border routers enable communication between nodes in different subnets while preserving the appearance of a single unified network to applications, thus achieving scalability without requiring changes to existing InfiniBand software or hardware interfaces.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Quantity of substance

If the subnet size is increased beyond 48,000 nodes, then larger computing clusters can be supported, but compatibility with existing InfiniBand software and hardware is lost

Engineering Contradiction:

Improvenumber of nodesVSAvoidcompatibility

Core Design Contradiction:

Quantity of substanceVSEase of manufacture

Solution Approach 1:

Subnet border routers serve as transparent intermediaries that handle the complexity of multi-subnet routing while presenting a unified network interface to applications. This allows existing InfiniBand software and hardware to operate without modification, maintaining compatibility while enabling support for networks exceeding 48,000 nodes through the border router infrastructure.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

By segmenting the network into standard-compliant subnets of up to 48,000 nodes each, the patent ensures that every individual subnet maintains full compatibility with existing InfiniBand specifications. The segmentation approach allows larger overall networks to be built from compatible building blocks without requiring changes to existing software or hardware.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS9288160B2GID capable switching in an infiniband fabric

Publication Date: 2016.03.15 INTEL CORP

AI summary

Methods, systems, and apparatus for extending the size of Infiniband subnets using GID switching in an Infiniband fabric. An Infiniband subnet is defined to include multiple local identifier (LID) domains, each including multiple nodes interconnected via one or more LID switches. In turn, the LID domains are interconnected via one or more GID switches. Messages may be transferred between nodes in a given LID domain using LID switches in the domain. Messages may be transferred between nodes in separate LID domains by routing the messages via one or more GID switches. In various embodiments, GID switches may be implemented to also operate as LID switches and perform routing based on selected packet header fields.