Control plane bridge

Abstract

The present disclosure relates to a control plane bridge. The control plane bridge comprises one or more interfaces and a processor. The one or more interfaces are configured to: (i) communicate with a local network operating according to a first control plane protocol, and (ii) communicate with an interconnect network connecting the local network with one or more other local networks, the interconnect network operating according to a second control plane protocol different from the first control plane protocol. The processor is configured to share control plane information between the local network and the other local networks by: (i) exchanging first messages with the local network according to the first control plane protocol, and (ii) exchanging second messages with one or more peer control plane bridges over the interconnect network using control messages of the second control plane protocol.

Description

Technical Field

[0001] This disclosure relates generally to network communications, and particularly, but not exclusively, to methods and systems for control plane bridges in heterogeneous communication networks. Background Technology

[0002] Large-scale distributed computing environments are becoming increasingly prevalent across industries and applications. These environments typically consist of multiple interconnected networks, each potentially employing different network technologies and protocols optimized for specific use cases.

[0003] The local networks in these environments typically employ proprietary protocols, such as InfiniBand™, designed for high-performance computing clusters, which offers low latency and high throughput. These local networks are often interconnected via wide area networks (WANs), which use standard protocols such as Border Gateway Protocol (BGP) for routing and control plane operations.

[0004] The scale and complexity of modern distributed computing systems continue to grow, with some environments containing thousands or even millions of computing nodes distributed across multiple geographical locations. This growth has spurred the development of multi-layered network architectures, in which local high-performance networks are integrated into a broader network infrastructure.

[0005] As distributed systems evolve, the industry is continuously working to improve the communication efficiency, resource utilization, and management capabilities of heterogeneous network domains. This includes researching methods for integrating different network technologies and protocols to create more flexible and scalable computing environments. Summary of the Invention

[0006] The embodiments described herein provide a control plane bridge comprising: one or more interfaces for (i) communicating with a local network operating according to a first control plane protocol, and (ii) communicating with an interconnection network connecting the local network to one or more other local networks, the interconnection network operating according to a second control plane protocol different from the first control plane protocol; and a processor for sharing control plane information between the local network and the other local networks by: (i) exchanging a first message with the local network according to the first control plane protocol, and (ii) exchanging a second message with one or more peer control plane bridges via the interconnection network using control messages of the second control plane protocol.

[0007] In one embodiment, the processor is configured to: receive given control plane information from the local network; generate one or more messages of a second control plane protocol carrying the given control plane information; and send the one or more messages to the one or more peer control plane bridges via the interconnection network for sharing with the other local networks.

[0008] In one exemplary embodiment, the processor is configured to monitor the topology of the local network using the first control plane protocol, and to generate one or more messages of the second control plane protocol in response to detecting a change in the topology, for notifying the change. In one disclosed embodiment, the processor is configured to segment the given control plane information into multiple messages of the second control plane protocol.

[0009] In some embodiments, the processor is configured to: receive one or more messages of the second control plane protocol from a peer control plane bridge via the interconnection network, the one or more messages of the second control plane protocol carrying given control plane information related to one of the other local networks; generate one or more messages of the first control plane protocol carrying the given control plane information; and send the one or more messages to a component in the local network. In one disclosed embodiment, the processor is configured to reassemble the given control plane information from multiple messages of the second control plane protocol.

[0010] In some embodiments, the processor is configured to: receive one or more messages of the second control plane protocol from the interconnection network, the one or more messages of the second control plane protocol carrying given control plane information associated with the interconnection network; generate one or more messages of the first control plane protocol carrying the given control plane information; and send the one or more messages to one or more components of the local network.

[0011] In one embodiment, the control plane information includes one or more of the following: discovery information about one or more processing elements (PEs) in a particular local network; routing information about reaching one or more processing elements (PEs) in a particular local network; network discovery information about a particular local network; and network topology changes in a particular local network or about network topology changes in the interconnected network.

[0012] In some embodiments, the second control plane protocol is a Border Gateway Protocol (BGP) based protocol, and the first control plane protocol is a non-BGP non-BGP protocol. In one embodiment, a multi-layer communication system comprising the local network, the other local networks, and the interconnected network is divided into two or more planes; and the control plane information includes one or more performance metrics from one or more of the planes.

[0013] Furthermore, according to the embodiments described herein, a method for control plane bridging is also provided. The method includes: communicating with a local network operating according to a first control plane protocol; communicating with an interconnecting network connecting the local network to one or more other local networks, the interconnecting network operating according to a second control plane protocol different from the first control plane protocol; and sharing control plane information between the local network and the other local networks by: (i) exchanging a first message with the local network according to the first control plane protocol; and (ii) exchanging a second message with one or more peer control plane bridges via the interconnecting network using control messages of the second control plane protocol.

[0014] According to the embodiments described herein, a computer software product is also provided, the product comprising a tangible, non-transitory, computer-readable medium storing program instructions that, when read by a processor, cause the processor to: communicate with a local network operating according to a first control plane protocol; communicate with an interconnection network connecting the local network to one or more other local networks, the interconnection network operating according to a second control plane protocol different from the first control plane protocol; and share control plane information between the local network and the other local networks by: (i) exchanging a first message with the local network according to the first control plane protocol; and (ii) exchanging a second message with one or more peer control plane bridges via the interconnection network using control messages of the second control plane protocol.

[0015] A more complete understanding of this disclosure will be achieved through the following detailed description in conjunction with the accompanying drawings. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating a multi-layer communication system employing control plane bridging, according to embodiments described herein.

[0017] Figure 2 A schematic map illustrating the embodiments described herein is used for Figure 1 The diagram shows a block diagram of the control plane bridge in a multi-layer communication system.

[0018] Figure 3 and Figure 4 This is a schematic diagram illustrating a flowchart of a method for controlling planar bridging according to embodiments described herein.

[0019] Figure 5 This is a schematic diagram illustrating a multi-layer communication system with a planar network topology, based on embodiments described herein.

[0020] Figure 6 This is a schematic diagram illustrating a computing system using control plane bridging according to embodiments described herein. Detailed Implementation

[0021] Overview Large-scale distributed computing environments typically consist of multiple local networks connected by an interconnect network. In many cases, the local networks and the interconnect network use different control plane protocols. In a typical example, the interconnect network uses the Border Gateway Protocol (BGP) for routing and control plane operations. On the other hand, the local network may contain an InfiniBand™ (IB) network and therefore use the IB control plane.

[0022] This heterogeneity presents significant challenges to efficient inter-network communication and resource utilization. For example, the IB control plane is incompatible with BGP, which fundamentally hinders seamless integration between local high-performance networks and the broader interconnect infrastructure.

[0023] Furthermore, each local network typically maintains its own network topology view, but lacks visibility into the broader multi-layered network structure. This limited awareness hinders efficient routing and resource allocation across the entire system. As the number of interconnected local networks increases, maintaining consistent control plane information across all domains becomes increasingly complex and resource-intensive, thus limiting the overall system scalability.

[0024] Traditional heterogeneous network bridging methods, such as bridging between InfiniBand and BGP domains, typically introduce significant latency and processing overhead, thereby reducing overall system performance. Network administrators also face challenges in configuring, monitoring, and troubleshooting multiple network domains with different control plane protocols, which increases the management complexity of these environments.

[0025] In large-scale distributed computing environments, these problems can lead to low resource utilization, increased latency, reduced network flexibility, and higher operating costs.

[0026] The embodiments described herein provide an improved system and method for control plane bridging between local and interconnected networks. The disclosed techniques enable seamless communication and efficient resource management across heterogeneous network domains. The embodiments described herein are primarily illustrated by bridging the gap between the InfiniBand and BGP control planes. More generally, the disclosed techniques can be used with any other suitable network type and control plane protocol.

[0027] In some embodiments, a multilayer communication system includes two or more local networks interconnected by an interconnection network. The system also includes two or more control plane bridges (hereinafter referred to as "bridges" for simplicity). A given control plane bridge acts as an intermediary between the local network and the interconnection network, which connects the local network to its peer local networks. For example, the bridge can be implemented using software on a network device running within a network device on the local network. Logically, the control plane bridge is located at the boundary between the local network and the interconnection network.

[0028] The control plane bridge performs the following key functions: - Protocol Conversion: The bridge converts between local network control plane protocols (such as InfiniBand) and plane protocols used in interconnecting network control (such as BGP). This enables seamless sharing of control plane information between different network domains.

[0029] - Topology Discovery and Distribution: The bridge actively monitors the topology of its local network and distributes this information to other networks via the interconnection network. It can also receive and transform topology information from other networks (other local networks and interconnection networks), thus providing each local network with a comprehensive view of the entire multi-layer network.

[0030] - Efficient Information Encoding: Local network topology and control plane information are efficiently encoded into BGP messages for distribution across interconnected networks, thereby minimizing overhead and maximizing scalability.

[0031] - Scalable Architecture: This solution supports multiple local networks connected via an interconnected network switch mesh structure, enabling flexible and scalable network topologies to meet the growing needs of distributed computing environments.

[0032] The control plane bridge technology disclosed in this paper eliminates manual network configuration, which is highly unscalable, time-consuming, and error-prone. This distributed solution eliminates single points of failure, thereby enhancing the overall resilience and availability of the system. For example, in some implementations, the control plane bridge is co-located with its corresponding local network's border switches or routers, thus sharing their availability. The failure of a given control plane bridge will not affect other bridges or the entire system.

[0033] Furthermore, by interacting directly with native protocols in both local and interconnected networks, the control plane bridges disclosed herein achieve seamless cross-domain communication while maintaining domain-specific optimization capabilities. The solution is also scalable, with its size linearly proportional to network size. Each control plane bridge operates independently of the others, requiring no upper-layer coordinating entity. These characteristics make the techniques disclosed herein highly effective in large and growing networks.

[0034] This article describes various example implementations and use cases of the control plane bridging technique disclosed herein.

[0035] System Description Figure 1 A block diagram illustrating a multi-layer communication system 20 employing a control plane bridge according to embodiments described herein is provided. System 20 may include, for example, a data center, a high-performance computing (HPC) cluster, or any other suitable communication system.

[0036] System 20 includes two or more local networks 24 and an interconnecting network 28 connecting the networks 24. In this example, system 20 includes three local networks, but any other suitable number of local networks may be used in alternative embodiments.

[0037] Each local network 24 includes a set of processing elements (PEs) 32. These PEs 32 can represent various computing resources, such as central processing units (CPUs), graphics processing units (GPUs), dedicated accelerators, and / or any other suitable type of processing device. Within each local network 24, the PEs 32 are connected to a local network switch 36 via network link 44. The local network switch 36 manages traffic within its respective local network 24 and can operate using local network protocols such as InfiniBand.

[0038] At the boundary of each local network 24, a corresponding control plane bridge 48 is provided. The control plane bridge 48 acts as an intermediary between the local network 24 and the wider interconnected network 28. The control plane bridge 48 can be implemented as a software module running on dedicated hardware, or it can be implemented as part of the local network switch 36.

[0039] The interconnection network 28 includes multiple interconnection network switches 40, which are connected in this example to form a mesh topology. This mesh arrangement allows for flexible routing and greater fault tolerance in the multi-layer system 20. The interconnection network 28 uses a different control plane protocol than the local network 24, which is BGP in this example.

[0040] Network links 44 connect the various components in the multi-layer system 20. These links establish communication paths between PE 32 and local network switch 36, between switches 36, between local network switch 36 and control plane bridge 48, between control plane bridge 48 and interconnecting network switch 40, and between switches 40.

[0041] The following will describe in detail how the control plane bridge 48 performs the following key functions: - Convert control plane information between protocols used in local network 24 and interconnected network 28; - Monitor the topology of its associated local network 24 and the distribution of topology information (to other local networks 24 and / or interconnection networks 28) via interconnection network 28; - Receive and process topology information from other local networks 24 and / or interconnected networks 28 to provide a comprehensive view of the entire multi-layer system 20 for each local network.

[0042] Figure 2 The illustration is schematically based on the exemplary embodiments described herein. Figure 1 A block diagram of the internal structure of the control plane bridge 48 used in a multi-layer communication system.

[0043] In this example, the control plane bridge 48 includes two main interfaces: a local network interface 52 for communicating with the corresponding local network 24, and an interconnect network interface 56 for communicating with the interconnect network 28.

[0044] At the heart of the control plane bridge 48 is the processor 60, which executes various software modules and algorithms to facilitate control plane bridging. The processor 60 is connected to both the local network interface 52 and the interconnect network interface 56, enabling it to process and translate control plane information between the two network domains.

[0045] exist Figure 2 In the illustrated embodiment, the processor 60 includes a BGP protocol engine 68, a local network module 72, an information encoding engine 76, and a route advertisement manager 80.

[0046] BGP protocol engine 68 handles various BGP-related operations, including message parsing, routing, and BGP state management. For example, the specific specification of BGP can be found in the Internet Engineering Task Force (IETF) Request for Comments (RFC), entitled "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, January 2006. Engine 68 is responsible for interpreting and generating BGP messages exchanged with the Internet 28. BGP protocol engine 68 typically, for example, achieves full BGP compatibility with the Internet 28 according to RFC 4271, thus participating in communications as a standard BGP speaker, capable of establishing peer relationships, processing route advertisements, and managing large community attributes.

[0047] Local network module 72 interfaces with components of local network 24 (e.g., switch 36 and / or subnet manager (SM)) to manage communication using the local network's native control plane protocol (InfiniBand in this example). In one embodiment, module 72 communicates natively with control plane elements of local network 24 via vendor-specific APIs (e.g., IB API or proprietary APIs of a proprietary architecture). For example, when connected to an InfiniBand network interface, module 72 can interact directly with the InfiniBand SM to extract control plane information such as network topology, routing information, and performance metrics. Among other tasks, module 72 typically extracts relevant network information and converts it into a standardized internal format. Module 72 is typically configurable to support the control plane protocols used in the local network.

[0048] The information encoding engine 76 is responsible for efficiently encoding local network topology and other control plane information into BGP messages for distribution on the interconnection network 28. Engine 76 also decodes incoming BGP messages to extract relevant information for use by the local network 24. Among other tasks, engine 76 typically performs conversions between internal data formats and the BGP large community attribute format, and implements TLV, bit vector, or segmented transmission as needed.

[0049] The route advertisement manager 80 oversees the process of advertising routing and network topology information to both the local network 24 and the peer network 24 via the interconnection network. In one exemplary embodiment, the route advertisement manager 80 summarizes local network resources into IPv6 subnets and advertises them to the interconnection network 28 via the BGP protocol engine 68.

[0050] The control plane bridge 48 also includes a network state database 64. Database 64 stores comprehensive information about the current state of the local network 24, peer-to-peer local networks 24, interconnected networks 28, and the broader multi-layered system as a whole. The processor 60 continuously updates the database 64 based on information received from the local and interconnected networks, thereby ensuring that the control plane bridge 48 maintains an up-to-date view of the entire network topology.

[0051] The various components within the processor 60 work together to perform the key functions of the control plane bridge 48. They are responsible for the conversion between different control plane protocols, the monitoring and distribution of topology information, and ensuring the efficient encoding and transmission of control plane data in heterogeneous network environments. It should be noted that the internal structure of the processor 60 and the division of labor among its components are described as examples only and are not limiting examples.

[0052] Control plane information type and format In various embodiments, the processor 60 of bridge 48 may collect and advertise various types of control plane information related to its corresponding local network 24. In this context, the term "control plane information" refers to any information that facilitates network devices (e.g., switches) and PEs in establishing and performing communication with each other. Several non-limiting examples of control plane information include: - The existence of a local network, or the existence of a PE or PE group in the local network.

[0053] - Discovery information for PEs or PE groups within a local network or local network.

[0054] - Routing information to reach the local network or PE or PE group within the local network.

[0055] - Changes in network topology in the local network or interconnected network, such as faulty links or switch information, changes in network configuration, etc.

[0056] In some embodiments, the processor 60 of bridge 48 advertises local network resources as aggregated IPv6 subnets. In this way, a standardized addressing mechanism is created, enabling global resource discovery while preserving local addressing schemes.

[0057] In various embodiments, processor 60 can use a variety of formats and encoding schemes to efficiently encode control plane information within the 12-byte payload capacity of BGP large community attributes. Processor 60 can support multiple encoding schemes and can select one based on specific network requirements. Example schemes include type-length-value (TLV) encoding for flexible attribute representation, bit vector encoding for link health and availability metrics, or any other suitable scheme.

[0058] Encapsulation, decapsulation, segmentation and reassembly In some embodiments, the information encoding engine 76 in processor 60 may implement an encapsulation mechanism to efficiently encapsulate and transmit control plane information on interconnect network 28. This encapsulation mechanism may leverage BGP large community attributes to carry extended information about, for example, local network topology and control plane state.

[0059] In one embodiment, the information encoding engine 76 encapsulates local network control plane information (obtained by module 72) into one or more BGP large community attributes. Specifications for BGP large community attributes can be found, for example, in IETF RFC 8092, entitled "BGP Large Communities Attribute," published in February 2017. These attributes can hold substantially more information than standard BGP attributes, thus allowing for the transmission of comprehensive topology and state information.

[0060] In some cases, the amount of control plane information to be transmitted may exceed the capacity of a single BGP large community attribute (12 bytes). In such cases, Engine 76 can segment the information into multiple segments and construct multiple BGP large community attributes to carry this information. In one example implementation, Engine 76 uses a portion of the BGP attribute payload as a sequence identifier, thereby enabling the reconstruction of the complete information set within the BGP attribute size constraint.

[0061] When receiving BGP messages from interconnect network 28 (e.g., from peer control plane bridge 48), information encoding engine 76 typically performs the reverse process, decapsulating control plane information from BGP large community attributes and reconstructing the original control plane information for use by the local network. When the received control plane information is segmented into multiple BGP large community attributes, engine 76 typically reassembles the information from the multiple segments, for example, using the sequence identifiers described above.

[0062] In some embodiments, the BGP attribute format used by engine 76 is fully compatible with the BGP specification. Therefore, BGP messages generated by engine 76 can be transmitted across interconnect network 28 to peer bridge 48 without any modification to the interconnect network. On the other hand, the payload of the BGP attributes is used in a proprietary manner to transmit control plane information between bridges 48.

[0063] Control Plane Bridging Process Figure 3 and Figure 4 The flowchart illustrating a method for control plane bridging is schematically shown according to embodiments described herein. Both methods are typically continuously executed by the processor 60 of each control plane bridge 48. Figure 3 The diagram illustrates the process of collecting and publishing control plane information for use by control plane bridges in peer-to-peer local networks. Figure 4 The diagram illustrates the process of receiving and ingesting control plane information advertised by the control plane bridge of a peer-to-peer local network. The following method description uses network topology information as an example, focusing on control plane information. Other types of control plane information can be processed in a similar manner.

[0064] Collection and notification of control plane information Figure 3 The method shown first involves the processor 60 of the control plane bridge 48 (in this example, the local network module 72 within the processor 60) obtaining the latest topology information from its corresponding local network 24 via interface 52 during the topology monitoring phase 90. Module 72 typically communicates with control plane elements of the local network (e.g., the IB subnet manager) using a native control plane API. Module 72 may collect information such as changes in the availability of PE 32 and / or switch 36 in the local network, the routing capabilities of switch 36, the status of link 44, and / or any other suitable information.

[0065] When a topology change is detected, processor 60 (in this example, the information encoding engine 76 and BGP protocol engine 68 within processor 60) constructs a BGP large community message (e.g., BGPUPDATE) in message construction phase 94 to deliver the update. As mentioned above, if the size of the update exceeds the capacity of a single BGP large community attribute, engine 68 can split the message into multiple BGP large community attributes. Engine 76 can encode the information in the BGP message using any suitable encoding scheme (e.g., TLV).

[0066] During the announcement phase 98, processor 60 (in this example, BGP protocol engine 68 in processor 60) sends a BGP message to interconnect network 28 via interface 56 for distribution to control plane bridge 48 of peer local network 24.

[0067] Because the BGP control plane responds to changes in an event-driven manner (rather than polling), the BGP protocol engine 68 immediately generates a BGP message (e.g., BGP UPDATE) upon detecting a change (e.g., a new PE, a link failure). This message is distributed to the individual bridges 48 via the interconnect network 28 with extremely low latency.

[0068] Control plane information reception and ingestion Figure 4The method shown begins with the processor 60 of the control plane bridge 48 (in this example, the BGP protocol engine 68 within the processor 60) receiving a BGP message through interface 56 during message reception phase 100. This BGP message carries topology update information about a specific peer-to-peer local network 24, which is announced by the peer control plane bridge 48 of the peer-to-peer local network.

[0069] If a BGP message is segmented into multiple BGP attributes, the BGP protocol engine 68 reassembles the message from the segments. The information encoding engine 76 decodes the topology information in the message according to the encoding scheme used (e.g., TLV). The processor 60 saves the updated topology information to the database 64.

[0070] In the conversion phase 104, the processor 60 (e.g., the local network module 72 in the processor 60) converts the topology information into a format compatible with the IB control plane protocol used in the local network.

[0071] During the local distribution phase 108, module 72 distributes topology information (topology information of the peer-to-peer local network) to switch 36 of the local network via interface 52. Distribution can be direct or indirect: in one embodiment, module 72 sends the topology information to a control plane element (e.g., SM) of the local network, which then makes the information available to switch 36. In another embodiment, module 72 sends the information directly to switch 36.

[0072] Example use cases Figure 5 This is a schematic diagram illustrating a multi-layer communication system 110 according to embodiments described herein. Figure 1 The system 20 is different. Figure 5 System 110 has a planar network topology. In this example, the system is divided into two planes, as shown below: - Plane I: Switches 40A, 40C, and 40D in interconnection network 28, and switches 36A, 36C, and 36E in local network 24.

[0073] - Plane II: Switches 40B, 40D, and 40F in interconnection network 28, and switches 36B, 36D, and 36F in local network 24.

[0074] It should be noted that in the planar topology, the interconnection network 28 does not have a complete mesh structure consisting of links 44 connecting each pair of switches 40.

[0075] When all links 44 and all switches 36 and 40 are functioning correctly, source PE 32 can send packets to the target PE through any plane. However, once a packet enters a particular plane, it cannot cross to another plane. If a link or switch fails, the target PE may only be reachable through one of the planes, meaning the source PE must make the correct routing decisions to successfully transmit the packet.

[0076] In some embodiments, bridge 48 identifies such faults and announces them as control plane information in system 110. Using this information, PE 32 can make correct routing decisions, such as ensuring that packets are sent only via planes capable of reaching the target PE.

[0077] The relevant control plane information in a planar topology is not limited to fault identification. For example, even if all planes are functioning correctly, different planes may differ in performance metrics such as available bandwidth, latency, or congestion. Announcement of this information using disclosed techniques allows PEs to distribute their traffic among the planes according to criteria designed to balance traffic load (e.g., using rate limiting or weighting). Generally, such criteria prioritize planes with better performance (e.g., less congestion, lower latency, and / or higher available bandwidth). In this context, reachability information (e.g., link status) is also considered a performance metric that may vary from plane to plane.

[0078] Figure 6 This is a block diagram schematically illustrating a computing system 1000, such as a data center or high-performance computing (HPC) cluster, according to embodiments described herein. According to at least one embodiment, system 1000 includes multiple subsystems, such as multiple mutually coupled processing devices, multiple network devices, and multiple networks. The computing system 1000 is designed with multiple integrated circuits (referred to as processing devices), each of which may contain one or more CPUs and GPUs, thus forming a powerful and flexible architecture.

[0079] Various processing devices are interconnected via NVLink or other high-speed interconnects to enable high-speed communication between subsystems. These processing devices are also connected via NICs or DPUs to ensure efficient data transmission on the computing system 1000 and efficient data transmission with one or more external networks 1030, 1036. In this example, system 1000 includes packet switch 1048 for connecting NIC / DPU 1028 and network 1030; and packet switch 1050 for connecting NIC / DPU 1032 and network 1036.

[0080] The processing unit, coupled via NVLink, enables seamless data exchange and parallel processing, thereby enhancing overall computing performance. Connecting to multiple networks via one or more Network Interface Cards (NICs) or Data Processing Units (DPUs), the system can handle complex multi-network tasks with high bandwidth and low latency. This configuration is ideal for demanding applications requiring extremely high processing power, such as artificial intelligence (AI), machine learning (ML), and data-intensive computing, while ensuring robust connectivity and scalability across diverse network environments. The integrated circuits of the Computing System 1000 can contain one or more CPUs and one or more GPUs.

[0081] Figure 6 An example architecture of a multi-GPU architecture is also shown. As shown in the figure, the computing system 1000 includes a processing device 1002 employing a multi-GPU architecture. Specifically, the processing device 1002 may be a system-on-a-chip and includes multiple subsystems, such as CPU 1006, GPU 1008, and GPU 1010. CPU 1006 can be coupled to GPU 1008 via die-to-die (D2D) or chip-to-chip (C2C) interconnects 1012 (e.g., ground reference signal interconnects (GRS interconnects)). CPU 1006 can be coupled to GPU 1010 via D2D or C2C interconnects 1014. CPU 1006 can also be coupled to GPU 1008 and GPU 1010 via PCIe interconnects.

[0082] The CPU 1006 can be coupled to one or more NICs or DPUs, which in turn can be coupled to one or more networks. For example, Figure 6 As shown, CPU 1006 is coupled to a first NIC / DPU 1026, which is coupled to network 1030. CPU 1006 is also coupled to a second NIC / DPU 1028, which is coupled to network 1030 via switch 1048. NIC / DPU 1026 and NIC / DPU 1028 can be coupled to network 1030 via Ethernet (ETH), NVLINK, or InfiniBand (IB) connections.

[0083] The computing system 1000 also includes a processing device 1004 employing a multi-GPU architecture. Specifically, the processing device 1004 includes multiple subsystems, such as a CPU 1016, a GPU 1018, and a GPU 1020. The CPU 1016 is connected to the GPU 1018 via a D2D or C2C interconnect 1022. The CPU 1016 is connected to the GPU 1020 via a D2D or C2C interconnect 1024. The CPU 1016 can also be coupled to the GPU 1018 and GPU 1020 via a PCIe interconnect. The CPU 1016 can be coupled to one or more NICs or DPUs, which in turn are coupled to one or more networks. For example, as... Figure 6 As shown, CPU 1016 is coupled to a first NIC / DPU 1034, which is coupled to network 1036. CPU 1016 is also coupled to a second NIC / DPU 1032, which is coupled to network 1036 via switch 1050. NIC / DPU 1032 and NIC / DPU 1034 can be coupled to network 1036 via Ethernet (ETH), NVLink, or InfiniBand (IB) connections.

[0084] In at least one embodiment, processing device 1002 and processing device 1004 can communicate with each other via NIC / Data Processing Element (NIC / DPU) 1038, for example via PCIe interconnect. Processing device 1002 and processing device 1004 can also communicate with each other via high-bandwidth communication interconnect 1040, such as NVLink interconnect or other high-speed interconnect.

[0085] In one embodiment, any network in system 1000 (e.g., network 1030 or 1036) or the entire system 1000 may be bridged in the control plane using the techniques described herein. Figure 6 The packet switch in the diagram could be, for example, an Nvidia Quantum-2 switch. The NIC / DPU in the diagram could be, for example, an Nvidia Bluefield DPU.

[0086] The above-mentioned and Figure 1 , Figure 2 , Figure 5 and Figure 6 The configurations shown (including various system, network, and control plane bridge configurations) are merely illustrative and are shown purely for conceptual clarity. Any other suitable configuration may be used in other embodiments. The different elements of the disclosed system, network, and control plane bridges may be implemented using suitable hardware (e.g., one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs)), software, hardware, or a combination of hardware and software elements.

[0087] In various embodiments, some or all of the functions of the disclosed control plane bridge may be implemented using one or more general-purpose processors that are software-programmed to perform the functions described herein. For example, the software may be electronically delivered to the processor via a network or downloaded from a host computer; alternatively, the software may be provided and / or stored on a non-transitory tangible medium, such as magnetic memory, optical memory, or electronic memory.

[0088] Although the embodiments described herein are primarily for control plane bridging, the methods and systems described herein can also be used in other applications, such as proprietary schemes for announcing information by reusing BGP community messages.

[0089] Therefore, it should be understood that the embodiments described above are merely examples, and the present invention is not limited to the specific details shown and described above. Rather, the scope of the present invention includes not only combinations and sub-combinations of the various features described above, but also various variations and modifications that can be conceived by those skilled in the art after reading the foregoing specification and that are not disclosed in the prior art. Documents incorporated herein by reference shall be considered an integral part of this application; however, if any definition of a term in these incorporated documents conflicts with the express or implied definitions in this specification, only the definitions in this specification shall be considered.

Claims

1. A control plane bridge, the control plane bridge comprising: One or more interfaces for (i) communicating with a local network operating according to a first control plane protocol, and (ii) communicating with an interconnecting network connecting the local network to one or more other local networks, the interconnecting network operating according to a second control plane protocol different from the first control plane protocol; and A processor is configured to share control plane information between the local network and the other local networks by: (i) exchanging a first message with the local network according to a first control plane protocol, and (ii) exchanging a second message with one or more peer control plane bridges via the interconnection network using control messages of a second control plane protocol.

2. The control plane bridge according to claim 1, wherein, The processor is used for: Receive given control plane information from the local network; Generate one or more messages of the second control plane protocol carrying the given control plane information; as well as The message is sent to one or more peer control plane bridges via the interconnection network for sharing with other local networks.

3. The control plane bridge according to claim 2, wherein, The processor is configured to monitor the topology of the local network using the first control plane protocol, and to generate one or more messages of the second control plane protocol in response to detecting a change in the topology, for announcing the change.

4. The control plane bridge according to claim 2, wherein, The processor is used to segment the given control plane information into multiple messages of the second control plane protocol.

5. The control plane bridge according to claim 1, wherein, The processor is used for: The second control plane protocol is received from the peer control plane bridge via the interconnection network, and the second control plane protocol message carries given control plane information related to one of the other local networks. Generate one or more messages of the first control plane protocol carrying the given control plane information; as well as The component that sends one or more messages to the local network.

6. The control plane bridge according to claim 5, wherein, The processor is used to reassemble the given control plane information from multiple messages of the second control plane protocol.

7. The control plane bridge according to claim 1, wherein, The processor is used for: Receive one or more messages of the second control plane protocol from the interconnection network, wherein the one or more messages of the second control plane protocol carry given control plane information related to the interconnection network; Generate one or more messages of the first control plane protocol carrying the given control plane information; as well as Send one or more messages to one or more components in the local network.

8. The control plane bridge according to claim 1, wherein, The control plane information includes one or more of the following: Discovery information about one or more processing elements (PEs) in a specific local network; Routing information for reaching one or more processing elements (PEs) in a specific local network; Network discovery information about a specific local network; as well as Changes in network topology within a specific local network or changes in network topology with respect to the interconnected network.

9. The control plane bridge according to claim 1, wherein, The second control plane protocol is a protocol based on Border Gateway Protocol (BGP), and the first control plane protocol is a protocol other than BGP.

10. The control plane bridge according to claim 1, wherein: A multi-layered communication system, including the local network, the other local networks, and the interconnected networks, is divided into two or more planes; and The control plane information includes one or more performance metrics for one or more of the planes.

11. A method for controlling planar bridging, the method comprising: Communicating with the local network operating according to the first control plane protocol; Communicating with an interconnecting network that connects the local network to one or more other local networks, the interconnecting network operating according to a second control plane protocol different from the first control plane protocol; and The control plane information is shared between the local network and the other local networks in the following manner: (i) exchanging a first message with the local network according to the first control plane protocol; (ii) Using control messages of the second control plane protocol, exchange second messages with one or more peer control plane bridges via the interconnection network.

12. The method according to claim 11, wherein, Sharing the control plane information includes: Receive given control plane information from the local network; Generate one or more messages of the second control plane protocol carrying the given control plane information; and The message is sent to one or more peer control plane bridges via the interconnection network for sharing with other local networks.

13. The method according to claim 12, wherein, Sharing the control plane information includes: monitoring the topology of the local network using the first control plane protocol, and generating one or more messages of the second control plane protocol in response to detecting changes in the topology, for announcing the changes.

14. The method according to claim 12, wherein, Generating the one or more messages includes segmenting the given control plane information into multiple messages of the second control plane protocol.

15. The method according to claim 11, wherein, Sharing the control plane information includes: The second control plane protocol is received from the peer control plane bridge via the interconnection network, and the second control plane protocol message carries given control plane information related to one of the other local networks. Generate one or more messages of the first control plane protocol carrying the given control plane information; and The component that sends one or more messages to the local network.

16. The method according to claim 15, wherein, Generating the one or more messages includes reassembling the given control plane information from multiple messages of the second control plane protocol.

17. The method according to claim 11, wherein, Sharing the control plane information includes: Receive one or more messages of the second control plane protocol from the interconnection network, wherein the one or more messages of the second control plane protocol carry given control plane information related to the interconnection network; Generate one or more messages of the first control plane protocol carrying the given control plane information; and Send one or more messages to one or more components in the local network.

18. The method according to claim 11, wherein, The control plane information includes one or more of the following: Discovery information about one or more processing elements (PEs) in a specific local network; Routing information for reaching one or more processing elements (PEs) in a specific local network; Network discovery information about a specific local network; as well as Changes in network topology within a specific local network or changes in network topology with respect to the interconnected network.

19. The method according to claim 11, wherein: A multi-layered communication system, including the local network, the other local networks, and the interconnected networks, is divided into two or more planes; and The control plane information includes one or more performance metrics for one or more of the planes.

20. A computer software product comprising a tangible, non-transitory computer-readable medium storing program instructions, which, when read by a processor, cause the processor to: Communicating with the local network operating according to the first control plane protocol; Communicating with an interconnecting network that connects the local network to one or more other local networks, the interconnecting network operating according to a second control plane protocol different from the first control plane protocol; and Control plane information is shared between the local network and the other local networks in the following manner: (i) exchanging a first message with the local network according to the first control plane protocol; and (ii) exchanging a second message with one or more peer control plane bridges via the interconnection network using control messages of the second control plane protocol.

Images