Data routing system and data routing method

Abstract

Disclosed in the present application is a data routing system. The system comprises a first topological network and a second topological network, wherein the first topological network is nested in the second topological network; the network structure of the first topological network is a fat-tree network topology structure consisting of a first node group and a switch, the first node group comprises a first number of network nodes, and there is a cascade relationship between the network nodes and the switch in the first topological network; and the network structure of the second topological network is a full-mesh network topology structure consisting of a second node group, the second node group comprises a second number of first node groups, and there is a connection relationship among the second number of first node groups. In the system, a multilevel hierarchical network structure where a fat-tree network topology is nested in a full-mesh network topology is established by means of small switching nodes having a small number of ports, and thus a large-scale or ultra-large-scale network is established, thereby effectively reducing the network cost and power consumption and improving the network scalability.

Description

Data routing system and data routing method

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on March 9, 2024, with application number 202410270219.7 and invention name “A Data Routing System and Data Routing Method”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computer network technology, and in particular to a data routing system, a data routing method, a method for constructing a data routing system, a management device, a computer-readable storage medium, and a computer program product. Background Art

[0003] As business scale continues to grow, more and more applications require the use of large-scale data centers for deployment. Data centers can be categorized into different types based on their application scenarios and needs. For example, data centers can be divided into cloud data centers and supercomputing centers (or supercomputing centers). Cloud data centers can provide internet services to a large number of users, including but not limited to email services and database services, while supercomputing centers primarily provide high-performance computing for scientific researchers.

[0004] The following uses a supercomputing center as an example to illustrate data center networking. Supercomputing centers use powerful "supercomputers" to perform calculations, reducing total solution time and achieving high-performance computing. Supercomputers, also known as supercomputing systems, are computers capable of performing high-speed calculations beyond the reach of personal computers, with specifications and performance significantly superior to those of personal computers. Several countries have successively released exascale supercomputing systems capable of performing 100 billion mathematical operations per second. Supercomputing systems in the post-exascale era face significant challenges in terms of network scale. Using the example of a future 10E-class supercomputing system, these systems could reach hundreds of thousands of nodes, requiring a network with 500,000 access ports.

[0005] How to design the network topology of a data center to meet the needs of large-scale or ultra-large-scale networking has become a key issue of concern in the industry. Summary of the Invention

[0006] This application provides a data routing system that uses small switching nodes with a small number of ports to create a multi-level hierarchical network structure with nested fat-tree network topologies and fully interconnected network topologies, thereby forming large-scale or ultra-large-scale networks, effectively reducing network costs and power consumption and improving network scalability. This application also provides a data routing method corresponding to the above-mentioned data routing system, a method for constructing a data routing system, a management device, a computer-readable storage medium, and a computer program product.

[0007] In a first aspect, the present application provides a data routing system. The data routing system is used to implement data routing between computing nodes. The data routing system includes a first topology network and a second topology network. The first topology network is nested in the second topology network. Nesting refers to insertion or embedding. Specifically, the first topology network or a component of the first topology network can be inserted into the second topology network. Specifically, the network structure of the first topology network is a fat tree network topology structure composed of a first node group and a switch, the first node group includes a first number of network nodes, and each network node in the first topology network is in a cascade relationship with the switch. The network structure of the second topology network is a fully interconnected network topology structure composed of a second node group, wherein the second node group includes a second number of first node groups, and there is a connection relationship between the second number of first node groups.

[0008] In this application, each network node in a node group is connected to a switch and network nodes in other node groups. Therefore, the network node can be implemented by a small switching node with a small number of ports, such as a low-port switching node. A large number of low-port switching nodes can be used to build a multi-level hierarchical network topology that nests fat tree network topology and fully interconnected network topology, achieving low-cost, low-power ultra-large-scale networking. Moreover, the performance of different topologies in this fusion topology complements each other, achieving high reliability, high performance, and high scalability.

[0009] In some possible implementations, in a first topology network, network nodes belonging to the same first node group are directly connected to the same switch. Specifically, when the number of ports on a single switch can meet the cascading requirements of network nodes in the first topology network, network nodes belonging to the same first node group can be directly connected to the same switch. This allows traffic to be redirected within the first node group and achieve single-hop reachability, thereby improving network performance.

[0010] In some possible implementations, the first node group includes multiple subgroups. The first topology network includes multiple first-layer switches corresponding to the multiple subgroups and at least one second-layer switch. In the first topology network, network nodes belonging to the same subgroup are directly connected to the first-layer switch corresponding to the subgroup, and the multiple first-layer switches are upwardly connected to the at least one second-layer switch.

[0011] In this way, the cascading requirements of network nodes in a first node group with more network nodes can be met through a multi-layer fat tree network topology structure including multiple first-layer switches and at least one second-layer switch. Even if there are more network nodes in the first node group, traffic can be jumped within the first node group through the first-layer switches and the second-layer switches, and can be reached within a few hops.

[0012] In some possible implementations, the subgroups are formed based on cabinets, cabinet groups, or computer rooms. By dividing first node groups of different specifications into subgroups based on size, directly connecting network nodes belonging to the same subgroup to the first-layer switch corresponding to that subgroup, and connecting the first-layer switches corresponding to multiple subgroups upward to the second-layer switch, the number of layers in the fat-tree network can be increased to meet the cascading requirements of more network nodes.

[0013] In some possible implementations, in a first topology network, network nodes belonging to the same first node group are directly connected to the same first switch and the same second switch. The first switch is used to forward intra-group traffic, and the second switch is used to forward inter-group traffic. The source and destination nodes of intra-group traffic are in the same first node group, while the source and destination nodes of inter-group traffic are in different first node groups. This allows for stratification of intra-group and inter-group traffic, avoiding congestion.

[0014] In some possible implementations, each network node includes n ports, m of which are used to connect to switches in the first topology network. By providing a small number of ports and cascading with switches, the network nodes can route traffic within the first node group through the switches, thereby achieving low communication latency.

[0015] In some possible implementations, h of the n ports are used to connect network nodes in a fully interconnected network topology structure formed by the second node group. The second number is less than or equal to h·the first number+1. This allows at least one global link to be included between different first node groups, providing direct routing for global communications, alleviating global network congestion, and reducing expensive global fiber links compared to a fat-tree network topology.

[0016] In some possible implementations, the data routing system further includes a third topology network. The second topology network is nested within the third topology network. The network structure of the third topology network is a fully interconnected network topology structure composed of a third node group, wherein the third node group includes a third number of second node groups, and a connection relationship exists between the third number of second node groups.

[0017] The system increases the number of layers of the fully interconnected network topology, such as adopting a nested two-level fully interconnected network topology, to compress the fat-tree network scale, and reduces the network diameter by increasing the network level, thereby reducing communication delay, improving communication performance, and increasing network throughput.

[0018] In some possible implementations, k ports out of the n ports of each network node are used to connect network nodes in the fully interconnected network topology structure formed by the third node group, and the third number is less than or equal to k·the number of nodes in the second node group+1.

[0019] In this way, different second node groups may include at least one global link, which can provide a direct route for global communication, alleviate global network congestion, and reduce expensive global optical fiber links compared to a fat-tree network topology.

[0020] In some possible implementations, the network nodes are switching nodes independent of the computing nodes, or routing engines integrated into the computing nodes. When the network nodes are routing engines integrated into the computing nodes, switch interconnection can be eliminated, further reducing physical costs and power consumption.

[0021] In some possible implementations, the node coordinates of the network node are determined based on the network node's number within the first node group and the number of the first node group to which the network node belongs. The network node's number within the first node group corresponds to a port number of a switch. The network node is configured to determine a data routing path based on the node coordinates, and implement data routing between the computing nodes according to the data routing path.

[0022] In this way, data routing paths can be determined based on node coordinates using a shortest path routing algorithm. This algorithm provides the shortest distance communication between source and destination nodes, minimizing communication latency. Because the data routing system of this application utilizes a hierarchical network structure that nests fat-tree and fully interconnected network topologies, a deterministic shortest path routing algorithm that adapts to topological characteristics offers rapid computational speed and facilitates hardware implementation.

[0023] In some possible implementations, compute nodes are used to execute high-performance computing (HPC) tasks. Traffic generated by executing HPC tasks can include intra-group traffic and cross-group traffic. By adopting a hierarchical network structure with nested fat-tree and fully interconnected network topologies, different topologies can be used to route different traffic flows, ensuring overall performance.

[0024] In a second aspect, the present application provides a data routing method. The method is applied to a data routing system. The data routing system is used to implement data routing between computing nodes, and the data routing system includes a first topology network and a second topology network. The first topology network is nested within the second topology network. The network structure of the first topology network is a fat tree network topology structure composed of a first node group and a switch. The first node group includes a first number of network nodes. Each network node in the first topology network is in a cascade relationship with the switch. The network structure of the second topology network is a fully interconnected network topology structure composed of a second node group, the second node group includes a second number of first node groups, and a connection relationship exists between the second number of first node groups.

[0025] Specifically, a current node in a data routing system receives a data packet to be routed. The current node is the source node of the data packet or a node on a path from the source node to a destination node. The current node then determines a data routing path based on the node coordinates of the current node and the node coordinates of the destination node. The node coordinates are determined based on the node's number within a first node group and the number of the first node group to which the node belongs. The current node then routes the data packet according to the data routing path.

[0026] This method introduces the node coordinates in the data routing system, adapts the topological characteristics of the data routing system to perform data routing, provides the shortest distance communication between the source node and the destination node, and achieves low communication delay. Moreover, this method is simple to implement and has high availability.

[0027] In some possible implementations, when the number of the first node group where the current node is located is equal to the number of the first node group where the destination node is located, the data routing system may determine the routing port according to the number of the destination node in the first node group.

[0028] Among them, if the number of the first node group where the current node is located is equal to the number of the first node group where the destination node is located, it means that the current node and the destination node are in the same first node group. The current node can achieve shortest path routing through the switch. For example, the current node can determine the routing port based on the number of the destination node in the first node group to achieve low communication latency.

[0029] In some possible implementations, when the number of the first node group in which the current node is located is not equal to the number of the first node group in which the destination node is located, the current node may determine whether a first direct link exists between the current node and the first node group in which the destination node is located. If so, the current node may determine that the data routing path includes forwarding the data packet along the first direct link to the first node group in which the destination node is located. If not, the current node may determine that the data routing path includes forwarding the data packet to a second jump node, and that a second direct link exists between the second jump node and the first node group in which the destination node is located.

[0030] In some possible implementations, the data routing system further includes a third topology network, wherein the second topology network is nested within the third topology network. The network structure of the third topology network is a fully interconnected network topology structure composed of a third node group, wherein the third node group includes a third number of second node groups. A connection relationship exists between the third number of second node groups, and the node coordinates are determined based on the node's number within the first node group, the number of the first node group to which the node belongs, and the number of the second node group to which the node belongs.

[0031] When the number of the second node group in which the current node is located is not equal to the number of the second node group in which the destination node is located, the current node determines whether a third direct link exists between the current node and the second node group in which the destination node is located. If so, the current node determines that the data routing path includes forwarding the data packet via the third direct link to the second node group in which the destination node is located. If not, the current node determines that the data routing path includes forwarding the data packet to the first jump node, which has a fourth direct link with the second node group in which the destination node is located.

[0032] When the current node and the destination node are in different second node groups, the method identifies whether the current node is directly connected to the second node group where the destination node is located. If so, routing is performed through the direct link; if not, routing is first performed to the corresponding jump node, thereby achieving routing through the shortest path.

[0033] In some possible implementations, the current node may further determine whether the number of the first node group in which the first jump node is located is equal to the number of the first node group in which the current node is located. If not, the current node further determines the data routing path by forwarding the data packet to a ferry node, where a fifth direct link exists between the ferry node and the first node group in which the first jump node is located.

[0034] In this method, when the first jump node and the current node are in different first node groups, the current node can first be routed to the ferry node to reach the first node group where the first jump node is located, and then the data packet can be routed to the first jump node through the intra-group routing of the first node group, and then routed based on the direct link of the jump node, thereby achieving shortest path routing.

[0035] In some possible implementations, the data packets include data packets for HPC tasks. By adopting a hierarchical network with nested fat-tree network topologies and fully interconnected network topologies, different topologies can be used to route different traffic flows, ensuring overall performance.

[0036] In a third aspect, the present application provides a method for constructing a data routing system. The method is applied to a management device and includes:

[0037] determining a plurality of network nodes and switches for constructing a data routing system, the plurality of network nodes being divided into a second number of first node groups, the first node groups including a first number of network nodes;

[0038] A configuration file is sent to the plurality of network nodes and the switch, so that the network nodes in the first node group are cascaded with the switch respectively according to the network structure of the data routing system in the configuration file to form a first topology network having a fat-tree network topology structure, and connections are established between the second number of first node groups to form a second topology network having a fully interconnected network topology structure, wherein the data routing system includes the first topology network and the second topology network, and the first topology network is nested within the second topology network.

[0039] The method determines switches and multiple network nodes used to build a data routing system, and instructs the network nodes and switches to establish connections according to the network structure of the data routing system in the configuration file, thereby constructing a data routing system with a nested fat tree network topology structure and a fully interconnected topology network structure, thereby realizing the formation of large-scale or ultra-large-scale networks, effectively reducing network costs and power consumption, and improving network scalability.

[0040] In some possible implementations, the plurality of network nodes are divided into a third number of second node groups, where the second node groups include a second number of first node groups. The configuration file is further used to establish connections between the network nodes in the first node group and the third number of second node groups, thereby forming a third topology network having a fully interconnected network topology. The data routing system further includes a third topology network, wherein the second topology network is nested within the third topology network.

[0041] By nesting a multi-layer fully interconnected network topology, it is possible to provide more ports and connect more computing nodes, thereby building an ultra-large-scale network to meet business needs.

[0042] In some possible implementations, the management device can also check the connection relationships of multiple network nodes based on the network structure of the data routing system in the configuration file. If the connection relationships of multiple network nodes are inconsistent with the connection relationships in the network structure of the data routing system, the management device can send a prompt to the user, prompting the user to adjust the connection relationships of the multiple network nodes. This ensures the accuracy and reliability of the network.

[0043] In some possible implementations, the management device may further generate a routing table based on node coordinates of the plurality of network nodes, where the node coordinates are determined based on a number of the network node within the first node group and a number of the first node group to which the node belongs. The management device may then distribute an entry in the routing table related to the at least one network node to at least one of the plurality of network nodes.

[0044] The management device sends the table entries related to the network node in the routing table to at least one network node, which can reduce the storage resources occupied by the routing table entries in the network node and save storage space.

[0045] In a fourth aspect, the present application provides a management device. The management device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor is configured to execute the instructions stored in the at least one memory, so that the management device performs the method for constructing a data routing system as described in the third aspect or any implementation of the third aspect.

[0046] In a fifth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and the instructions instruct a management device to execute the method for constructing a data routing system described in the third aspect or any implementation of the third aspect.

[0047] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on a management device, enables the management device to execute the method for constructing a data routing system as described in the third aspect or any one of the implementations of the third aspect.

[0048] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical methods of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.

[0050] FIG1 is a schematic diagram of the architecture of a data routing system provided by the present application;

[0051] FIG2 is a schematic diagram of ports of a node in a data routing system provided by the present application;

[0052] FIG3 is a schematic structural diagram of a low-port router provided by the present application;

[0053] FIG4 is a schematic diagram of the structure of a routing engine integrated in a processor provided by the present application;

[0054] FIG5 is a schematic structural diagram of a data routing system provided by the present application;

[0055] FIG6 is a flow chart of a data routing method provided by the present application;

[0056] FIG7 is a schematic diagram of a data routing process in a data routing system provided by the present application;

[0057] FIG8 is a schematic diagram of data routing through a virtual channel provided by the present application;

[0058] FIG9 is a schematic diagram of a fault recovery provided by the present application;

[0059] FIG10 is a flowchart of a method for constructing a data routing system provided by the present application;

[0060] FIG11 is a schematic structural diagram of a management device provided by this application;

[0061] FIG12 is a schematic diagram of the structure of a computing device provided by the present application;

[0062] FIG13 is a schematic diagram of the structure of a computing device cluster provided by the present application;

[0063] FIG14 is a schematic diagram of the structure of another computing device cluster provided by the present application;

[0064] FIG15 is a schematic diagram of the structure of another computing device cluster provided in this application. DETAILED DESCRIPTION

[0065] The terms "first" and "second" in the embodiments of this application are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0066] First, some technical terms involved in the embodiments of this application are introduced.

[0067] Network topology refers to the specific arrangement or connection between network components (e.g., switches, routers, and other network or communication devices). It is generally categorized as either a physical, real, online structure or a logical, virtual, programmatic structure. If two networks have different physical connections and node distances, but the connections between the network components are the same, then the network topologies of the two networks are considered the same.

[0068] Networking, short for network formation, typically involves designing a suitable network topology based on networking requirements, including scale and performance requirements, and then constructing the network according to that topology. In scenarios such as cloud computing, high-performance computing (HPC), or artificial intelligence (AI), it's often necessary to connect computing devices (also known as compute nodes) through network devices (such as switches, routers, and other switching nodes) to form large-scale or ultra-large-scale interconnected networks, such as cloud data centers, supercomputing centers, or AI computing centers. This provides efficient communication capabilities with high bandwidth, low latency, and high throughput, effectively unleashing computing power.

[0069] Among them, cloud computing, also known as network computing, is an Internet-based computing method through which shared software and hardware resources and information can be provided to various terminals or other devices on demand. Terminals and other devices use the infrastructure provided by the service provider for computing. High-performance computing refers to providing more powerful computing performance than traditional computers or servers by aggregating computing power. It can usually be used in scenarios such as weather forecasting, astrophysical simulation, molecular dynamics simulation, and earthquake simulation. AI computing is a mathematically intensive process for calculating machine learning algorithms, which can usually use accelerated systems and software. It can extract new insights from large data sets and learn new capabilities in the process. AI computing can be applied to model training or model reasoning, especially large language model (LLM) reasoning.

[0070] For ease of description, this application mainly uses the example of building a supercomputing center for high-performance computing.

[0071] Supercomputing centers include supercomputers. Supercomputers are of strategic importance in science and technology, leading various countries to release numerous exascale-class supercomputing systems. Exascale systems are capable of performing exascale mathematical operations per second. In the post-exascale era, supercomputing centers face significant challenges in network scale, scalability, performance, and reliable availability. These challenges are addressed below.

[0072] Currently, network equipment in data centers, such as the aforementioned supercomputing centers, accounts for 20%-30% of energy consumption, and network costs account for 10%-20%. Future 10E-class supercomputing systems could see network scales reach hundreds of thousands of nodes, requiring access to 500,000 ports and system energy consumption reaching 50 megawatts (MW). For 10E-class supercomputing systems, network costs and power consumption pose significant challenges. Furthermore, as network scale increases, the number of network hops increases, latency increases, and congestion intensifies, leading to a sharp decline in network bandwidth utilization. Furthermore, dynamic random access memory (DRAM) technology lags behind by two to three generations, resulting in a failure rate five times higher than the international average, severely impacting system reliability and availability. Designing data center network topologies to effectively reduce network costs and power consumption is a crucial challenge in network architecture design.

[0073] Many data centers, such as hyperscale supercomputing centers, require interconnection networks to provide high bandwidth, low latency, and high throughput to deliver high-performance communication capabilities. Currently, mainstream high-performance interconnection network topologies include the fat tree network topology (abbreviated as fat tree network, fat tree topology), the dragonfly network topology (abbreviated as dragonfly network, dragonfly topology), and the torus network topology (abbreviated as torus network, torus topology).

[0074] Some exascale supercomputer systems under construction use the Dragonfly topology. This topology is built using large-port switches. Specifically, large-port switches create a fully interconnected network structure within and between groups. This network is non-blocking, has a low network diameter of only four hops, and offers low networking costs, reducing the use of valuable global fiber links compared to fat-tree networks. However, system scale is limited by switch specifications, requiring a high number of ports. Based on a 64-port switch, the maximum network size is 270,000 ports, which cannot meet the networking requirements of future 10E-class supercomputer systems.

[0075] Based on the Dragonfly topology, the industry has also proposed an evolved Dragonfly+ topology. The Dragonfly+ topology is also built on large-port switches. Specifically, it uses large-port switches to build a symmetrical fat-tree structure within a group and a fully interconnected structure between groups. This provides strong network scalability, is non-blocking, and maintains a network diameter of four hops. At the same scale, network costs are reduced by 20% in core switches compared to a fat-tree network, and global fiber links are reduced by 50%. However, this solution also requires large-port switches to increase the number of nodes within the group. However, based on 48-port switches, the maximum network size can connect to 332,352 ports, which is difficult to meet the networking requirements of a 10E-class supercomputer system. Moreover, for adversarial traffic, the throughput can only reach a maximum of 50%.

[0076] Given the advantages of the fat-tree network topology, such as multi-path load balancing, non-blocking, stable performance, mature technology, and wide adaptability, it can be used to build very large-scale networks. Building very large-scale networks based on this topology requires high-radix switches (such as 64-port switches) to meet scale requirements, reduce network layers, and reduce network diameter. Furthermore, network performance can be improved by increasing port bandwidth and switch forwarding capacity. However, high-radix switches face significant challenges in power consumption and manufacturing processes, making it difficult to design and manufacture high-performance switches with high radix and bandwidth by improving process specifications.

[0077] Furthermore, increases in port bandwidth and port count can lead to increased network power consumption and costs. Currently, mainstream commercial high-performance switches have reached 400Gbps port bandwidth, supporting 64 400Gb / s ports. For the 400Gbps bandwidth standard, the serializer / deserializer (SerDes) energy efficiency can reach 20pJ / bit. Consequently, the SerDes power consumption can be 8W / port, while the optical module power consumption can be 5W / port. A single-port high-speed SerDes and optical module power consumption reaches at least 13W. Therefore, reducing the number of switch ports, the number of network links, and the number of fiber links can effectively reduce network power consumption and costs. Furthermore, large-scale data centers (such as supercomputing centers) require a large number of fiber links to expand network scale. For a fat-tree network with 500,000 ports, the number of optical modules can reach 180,000. With an optical module failure rate as high as 4%, an average of 20 failures per day, this seriously impacts system reliability. Therefore, reducing the number of optical modules can effectively improve system reliability.

[0078] In view of this, the present application provides a data routing system. This data routing system can achieve ultra-large-scale networking through a novel fused topology structure. The data routing system is used to implement data routing between computing nodes. The data routing system includes a first topology network and a second topology network. The first topology network and the second topology network form a fused topology structure.

[0079] Specifically, the network structure of the first topology network is a fat tree network topology structure composed of a first node group and a switch. The first node group includes a first number of network nodes. Each network node in the first topology network is in a cascade relationship with the switch. The first topology network is nested in the second topology network. Here, nesting refers to embedding or insertion. Specifically, the first topology network or a component of the first topology network can be embedded in the second topology network as a component or constituent unit of the second topology network. The network structure of the second topology network is a fully interconnected network topology structure composed of a second node group. The second node group includes a second number of first node groups. A connection relationship exists between the second number of first node groups.

[0080] Among them, each network node in the node group is connected to the switch and the network nodes in other node groups respectively. Therefore, the network node can be realized by a small switching node with a small number of ports, such as a low-port switching node. A large number of low-port switching nodes can be used to build a multi-level hierarchical network topology to achieve low-cost, low-power ultra-large-scale networking. Moreover, the performance of different topologies in this fusion topology complements each other to achieve high reliability, high performance, and high scalability. Furthermore, computing nodes such as processors can also integrate high-speed interconnect core IO Die, such as IO ports with routing and forwarding functions, so that low-port switching nodes can be replaced, further reducing network costs and power consumption.

[0081] In order to make the technical solution of the present application clearer and easier to understand, the system architecture of the data routing system of the present application is introduced below with reference to the accompanying drawings.

[0082] Figure 1 shows an architectural diagram of a data routing system, where the data routing system 10 includes multiple network nodes (sometimes collectively referred to as "nodes 110" below), wherein the multiple nodes 110 are located in a communication network 140 that includes at least a first topology network (e.g., 120-1, 120-2, ..., 120-Q, sometimes collectively referred to as "first topology network 120" below) and a second topology network 130.

[0083] In an embodiment of the present application, the first topology network 120 is nested in the second topology network 130. The network structure of the first topology network 120 is a fat tree network topology structure consisting of a first node group and a switch. The first node group includes a first number of network nodes. For example, the first node group may include node 110-1, node 110-2, ..., node 110-P, where P is a positive integer greater than 1. Switches can be divided into different types according to their deployment location. For example, the switch can be a top of rack (TOR) switch, an end of row (EOR) switch, or a middle of row (MOR) switch. Each network node in the first topology network 120 (such as node 110-1, node 110-2, ..., node 110-P) has a cascade relationship with the switch. For example, the first node group may include 64 network nodes, and the switch may include 64 ports. Then, each network node in the first node group can be connected to one port of the switch.

[0084] The network structure of the second topology network 130 is a fully interconnected network topology structure composed of second node groups. The second node groups include a second number of first node groups, such as 120-1, 120-2, ..., and 120-Q. A connection relationship exists between the second number of first node groups. Using FIG1 as an example, a connection relationship can exist between 120-1, 120-2, ..., and 120-Q. It should be noted that the connection relationship between node groups can be a connection relationship between nodes in different node groups, such as a connection relationship between the nodes in 120-1 and the nodes in 120-2.

[0085] The data routing system 10 may include a single layer of fully interconnected networks, or multiple layers of fully interconnected networks. When the data routing system 10 includes multiple layers of fully interconnected networks, the multiple layers of fully interconnected networks may be nested. For example, the first layer of fully interconnected networks may be nested within the second layer of fully interconnected networks. Similarly, the L-1 layer of fully interconnected networks may be nested within the L layer of fully interconnected networks. Nesting refers to insertion or embedding. For example, the first layer of fully interconnected networks may be nested within the second layer of fully interconnected networks. The first layer of fully interconnected networks may serve as a component or unit of the second layer of fully interconnected networks and be embedded within the second layer of fully interconnected networks.

[0086] The present application does not impose any restrictions on the number of layers (or levels) of the fully interconnected network in the data routing system 10. The number of layers of the fully interconnected network can be configured according to the required network scale. The network scale can be characterized by the total number of ports. For example, when the total number of ports is less than or equal to the first threshold, the data routing system 10 can be configured with a one-layer fully interconnected network topology to meet the demand. For another example, when the total number of ports is greater than the first threshold and less than or equal to the second threshold, the data routing system 10 can be configured with a two-layer fully interconnected network topology to meet the demand. It should be noted that the above-mentioned first threshold can be the maximum number of ports that the data routing system 10 can provide when a one-layer fully interconnected network topology is nested, and the above-mentioned second threshold can be the maximum number of ports that the data routing system 10 can provide when a two-layer fully interconnected network topology is configured.

[0087] Based on this, the data routing system may further include a third topology network, and the second topology network 130 may be nested in the third topology network. The second topology network 130 may be nested in the third topology network. Similar to the second topology network 130, the network structure of the third topology network may also be a fully interconnected network topology structure. The main difference between the third topology network and the second topology network 130 is that the network structure of the third topology network is a fully interconnected network topology structure based on a third node group. The third node group includes a third number of second node groups, and a connection relationship exists between the third number of second node groups. The second node group is a node group formed by network nodes in the second topology network 130, thereby enabling the second topology network 130 to be nested in the third topology network.

[0088] Similar to a fully interconnected network topology, a fat-tree network topology can connect network nodes within the first node group via a single layer of switches or multiple layers of switches. In some possible implementations, the number of ports on a single switch can satisfy the cascading requirements of network nodes in the first topology network. Accordingly, in the first topology network, network nodes belonging to the same first node group can be directly connected to the same switch. For example, the first node group can include network nodes in a 4*4*4 cabinet. In other words, the number of network nodes in the first node group is less than or equal to 64. A single 64-port switch can satisfy the cascading requirements of each network node in the cabinet, and each network node in the cabinet can be directly connected to at least one port of the 64-port switch. In other possible implementations, the first topology network includes a large number of network nodes, and the number of ports on a single switch is insufficient to satisfy the cascading requirements of network nodes in the first topology network. Based on this, the cascading requirements of network nodes in the first topology network can be satisfied via multiple switches. Specifically, the first node group includes multiple subgroups, wherein the subgroups can be divided according to cabinets, cabinet groups, or computer rooms. For example, when the first node group includes 1024 network nodes, such as network nodes in a super node formed by 16 8*8 cabinets, subgroups can be obtained by dividing the cabinets. The first topology network includes multiple first-layer switches corresponding to multiple subgroups and at least one second-layer switch. In the first topology network, network nodes belonging to the same subgroup are directly connected to the first-layer switch corresponding to the subgroup. It should be noted that each subgroup can correspond to one or more first-layer switches. Multiple first-layer switches can be connected upward to at least one second-layer switch. Among them, the first-layer switch can be a switch set in the cabinet, such as a TOR switch, and the second-layer switch can be a switch set between cabinets.

[0089] Taking into account that the intra-group traffic of the first node group and the cross-group traffic (inter-group traffic) of the first node group can cause resource competition, multiple switches can be set in the first topology network, for example, including a first switch and a second switch. In the first topology network, network nodes belonging to the same first node group are directly connected to the same first switch and the same second switch. Among them, the first switch is used to forward intra-group traffic, and the second switch is used to forward cross-group traffic. The source node and destination node of the intra-group traffic are in the same first node group, and the source node and destination node of the cross-group traffic are in different first node groups. In this way, intra-group traffic and cross-group traffic stratification can be achieved to avoid congestion.

[0090] Next, the network topology is described in conjunction with the port connection status of the network nodes.

[0091] Specifically, referring to FIG2 , each network node may include n ports, such as port 111-1, port 111-2, ..., 111-n. Each network node may provide m ports for connecting to switches in the first topology network 120. m may be greater than or equal to 1. Network nodes within the first node group, such as network nodes within a rack, may be interconnected non-blockingly via the switch. Nodes only need a single forwarding pass from the switch to communicate, and are reachable within the rack (RACK) in a single hop.

[0092] The remaining nm ports of each network node are used for full interconnection. Among them, nm can be greater than or equal to 1. The remaining nm ports can be nested in levels according to the system scale requirements. Specifically, the network node can connect to the network nodes in other first node groups through at least one port other than the above-mentioned m ports to form a fully interconnected network topology. For example, the network node can connect to the network nodes in other first node groups through 1 port other than the m ports to form a second topology network. In this way, it can be guaranteed that there is at least one global direct link between each first node group. Furthermore, the network node can connect to the network nodes in other second node groups through another port other than the m ports to form a third topology network. In this way, it can be guaranteed that there is at least one global direct link between each second node group.

[0093] The node 110 in the embodiment of Figure 1 can be implemented as a physical node or a virtual node. The specific implementation of the physical node and the virtual node will be described below.

[0094] In some embodiments, node 110 can be a switching node independent of the computing node, such as an independent router. Node 110 can be a low-port switching node, such as a low-port router. By using low-port switching nodes for expansion, network costs are low and multi-path features are provided. Multi-path features refer to the use of multiple paths between nodes to avoid single points of failure and achieve load balancing based on multiple paths. For ease of description, a low-port router is used as an example. A low-port router is a router with a small number of ports. For example, a low-port router can be a 3-port router, or a 4-port, 5-port, or 6-port router.

[0095] Figure 3 shows a schematic diagram of a low-port router, which includes multiple communication engines, a crossbar, and multiple ports. The communication engines facilitate communication between the ports of a router or routing engine and the processor, enabling operations such as direct memory access (DMA), data reads and writes, and memory copies. The crossbar facilitates communication between the ports of a router or routing engine and the communication engines, enabling internal data forwarding in a low-port mode.

[0096] As shown in Figure 3, the multiple ports may include at least one port for constructing a fat tree network topology (such as the first topology network mentioned above) and two ports for constructing a fully interconnected network topology. The at least one port for constructing a fat tree network topology can be cascaded with a switch, for example, cascaded with a TOR switch, to construct a fat tree network topology. The two ports for constructing a fully interconnected network topology include a port L-Link for implementing a first-layer fully interconnected network (such as the second topology network mentioned above) and a port G-Link for implementing a second-layer fully interconnected network (such as the third topology network mentioned above), wherein the L in L-Link can represent local, and the G in G-Link can represent global.

[0097] In other embodiments, node 110 may be a routing engine integrated into a computing node. In other words, node 110 may be a computing node integrated with a routing engine, which has routing functionality. The routing engine may be a routing engine integrated into a processor. The processor may be a graphics processing unit (GPU), a neural network processing unit (NPU), or other accelerator device, or a central processing unit (CPU).

[0098] Figure 4 shows a schematic diagram of the architecture of a routing engine integrated into a processor. The processor may include multiple cores and a last-level cache (LLC). Processors typically employ multiple levels of cache, with each level of cache being faster than the next. When a core needs to access a piece of data or instruction, it first checks the nearest L1 cache. If the data is present, it represents a cache hit; otherwise, it represents a cache miss, requiring the next level of cache to be queried. In some possible implementations, the processor may be a CPU / GPU memory chip integrated with high-bandwidth memory (HBM). The HBM may be a stack of multiple Double Data Rate Synchronous Dynamic Random Access Memory (DDR SDRAM, abbreviated as DDR). After stacking, the stacked chips can be packaged together with the CPU / GPU die to create a large-capacity, high-bit-width DDR array.

[0099] The structure of the routing engine integrated in the processor can refer to the structure of the low-port router in Figure 3. The routing engine may include a communication engine, a crossbar switch matrix, and a port. In the example of Figure 4, the processor integrates or has a built-in multi-port routing engine. At least one port in the multi-port routing engine is used to connect to a switch, such as a TOR switch, thereby building a fat tree network topology. The two ports L-Link and G-Link in the multi-port routing engine are used to connect to network nodes in other node groups. For example, L-Link is used to connect to network nodes in other first node groups, and G-Link is used to connect to network nodes in other second node groups, thereby realizing a fully interconnected network topology.

[0100] The core of the processor may communicate with the routing engine via an internal bus, which may include but is not limited to a high-speed serial component interconnect bus (Peripheral Component Interconnect Express, PCIe) or a Compute eXpress Link (CXL) bus.

[0101] It should be understood that the processor or router or routing engine of the present application may include more or fewer components, functions or modules. In some embodiments, the processor may also include components such as a bus or interface, a processor core, a resonant circuit, etc. for interconnecting with the routing engine, and may also be connected to external devices such as storage devices. For example, the processor of the integrated routing engine may also include a memory management unit (MMU) and a memory controller (MC) for managing and controlling external or extended memory such as DDR. The memory management unit and the memory controller may communicate with the extended DDR via a unified bus (UB). Similarly, the processor may also be connected to or extended with a solid-state drive (SSD), wherein the processor set may communicate with the SSD via a UB bus or a Ctrl bus. In addition, the processor may also be connected to or extended with devices such as accelerators. In some embodiments, the router or routing engine may be any device for implementing data transmission, such as a network device such as a switch.

[0102] It should be noted that Figures 3 and 4 are both illustrated using examples of at least 3 ports. In other possible implementations of the embodiments of the present application, the routing engine integrated in the low-port router or processor may also include other numbers of ports, such as 4 ports, 5 ports, etc.

[0103] In the present application, a computing node can be used to perform HPC tasks. The computing node can be an independent computing node, or a computing node integrated with a routing engine, such as a processor integrated with a routing engine. The traffic generated by executing the HPC task may include intra-group traffic or cross-group traffic. The source node and destination node of the intra-group traffic are in the same first node group, such as in the same super node. The source node and destination node of the cross-group traffic are in different first node groups, such as in different super nodes. Accordingly, the first topology network of the data routing system can be used to route intra-group traffic, for example, the first switch in the first topology network can be used to route data packets whose source node and destination node are in the same super node, and the second topology network of the data routing system can be used to route cross-group traffic. It should be noted that when routing cross-group traffic, taking into account the situation that the current node is not a network node directly connected to the first node group where the destination node is located, the first topology network can also jump the cross-group traffic. For example, the second switch in the first topology network can jump the cross-group traffic to a network node that has a direct link to the first node group where the destination node is located, and then route it through the direct link.

[0104] In order to make the technical solution of the present application clearer and easier to understand, the following is an example of a network consisting of a processor with an integrated 3-port routing engine and a TOR switch.

[0105] Full mesh is an ideal topology that can provide high binary bandwidth, with direct links between any nodes, which can be reached in one hop, and has the shortest communication distance, the lowest latency, no network congestion, and good performance. However, full mesh requires that the number of node ports is proportional to the scale, which makes it difficult for full mesh to meet the needs of large-scale networks. To this end, this application adopts a layered approach to implement full mesh expansion step by step, and can achieve ultra-large-scale networking with only a small number of ports and switches. For example, by integrating a processor with a 3-port routing engine (which can also be replaced by a 3-port router) and a TOR switch, the networking needs of more than 17 million ports can be met, greatly reducing network costs and power consumption.

[0106] Referring to the structural diagram of a data routing system shown in FIG5 , a cabinet can include 64 processors, such as N01 to N64, each of which integrates a 3-port routing engine. A TOR switch, which can be a 64-port box switch, is also provided within the cabinet. Port 1 of each processor can be connected to a port of the TOR switch. The TOR switch within the L1-level rack (RACK) can directly connect to the 64 processors. Communication between processors (such as between N01 to N64) requires only a single forwarding by the TOR switch, and is reachable within a single hop within the RACK. Port 2 of each processor is used for full mesh connections between racks. The 64 processors within a cabinet can provide 64 global direct links, interconnecting nodes in up to 65 cabinets. Therefore, the L2 network scale can reach 64 * 65 = 4160 nodes, and the maximum communication distance between L2 networks (such as the second topology network) is only 3 hops. Port 3 is used for full mesh interconnection between multiple L2 networks. Each node or processor provides a global direct link. 65 cabinets can provide 4160 global direct links. Accordingly, a Layer 3 network can interconnect a maximum of 4160 * 4161 = 17,309,760 nodes, with a network diameter of only 7 hops. This layered approach enables the creation of a non-blocking network, offering the advantages of ultra-large-scale networking, ultra-low latency, ultra-low cost, and ultra-high performance.

[0107] In some possible implementations, the first topology network may further add switches, for example, at least one TOR switch. Specifically, the first topology network may include a first switch and a second switch. In the first topology network, network nodes belonging to the same first node group are directly connected to the same first switch and the same second switch. The first switch may be used for intra-group traffic forwarding, and the second switch may be used for cross-group traffic forwarding. The source node and destination node of the intra-group traffic are in the same first node group, and the source node and destination node of the cross-group traffic are in different first node groups. The added TOR switch can be used for traffic stratification, responsible for the jump of global traffic forwarding, and avoids competing with the internal traffic of RACK for link bandwidth. In this way, the network can be completely non-blocking, and the performance is better than the dragonfly topology and fat tree topology.

[0108] Figure 5 illustrates a three-port system where one port is used for direct connection to a TOR switch within a rack, one port is used for full mesh connections between racks, and one port is used for full mesh interconnection between multiple L2 networks. In actual applications, more ports can be used to connect more nodes or provide more links.

[0109] Specifically, each network node includes n ports, m of which are used to cascade with switches in the first network topology. FIG. 5 above illustrates an example of constructing a fat-tree network topology structure with m = 1. Furthermore, h of the n ports are used to connect to network nodes in a fully interconnected network topology structure formed by a second node group. The second node group includes a second number of first node groups. Considering the limited number of ports, the maximum value of the second number can be determined based on the number of ports used to connect to network nodes in the fully interconnected network topology structure formed by the second node group and the number of nodes in the first node group.

[0110] Based on this, the second quantity can satisfy the following formula: count 2nd ≤h·Q+1 (1)

[0111] Among them, count 2ndRepresents the second quantity, h represents the number of ports used to connect network nodes in the fully interconnected network topology structure composed of the second node group, and Q represents the number of nodes in the first node group. h can be greater than or equal to 1, so that different first node groups (or first topology networks) include at least one directly connected global link. Figure 5 uses h equal to 1 as an example. In actual application, h can also be a positive integer greater than 1, such as 2 or 3. In this way, the links between network nodes can be increased, the blocking probability can be reduced, or more first node groups can be connected to increase the network scale. Q can be greater than 1. For example, when the first node group is a network node in a cabinet, Q can be 4*4*4=64.

[0112] In some possible implementations, k of the n ports of a network node are used to connect to network nodes in a fully interconnected network topology structure formed by a third node group. The third node group includes a third number of second node groups. Given the limited number of ports, the maximum value of the third number can be determined based on the number of ports used to connect to network nodes in the fully interconnected network topology structure formed by the second node group and the number of nodes in the second node group. The maximum value of the number of nodes in the second node group can be determined based on the second number (the number of the first node group in the second node group) and the first number (the number of nodes in the first node group), for example, the product of the first number and the second number.

[0113] Based on this, the third quantity can satisfy the following formula: count 3rd ≤k·count 2nd count 1st +1 (2)

[0114] Among them, count 1st Indicates the first quantity, count 2nd Indicates the second quantity, count 3rd represents the third number, and k represents the number of ports used to connect network nodes in the fully interconnected network topology structure formed by the third node group. 2nd count 1st This can be the number of nodes in the second node group. k can be greater than or equal to 1, so that different second node groups (or second topology networks) include at least one direct global link. Figure 5 illustrates this with k being 1. In practice, k can also be greater than 1, increasing the number of links between network nodes and reducing the probability of blocking.

[0115] Based on the aforementioned embodiments, a network node can be uniquely identified by the number of the network node in different topology groups. Based on this, the above number or port number can be used as the node coordinates of the network node. In some possible implementations, the node coordinates of the network node can be determined based on the number of the network node in the first node group (such as a cabinet) and the number of the first node group in which the network node is located. The number of the network node in the first node group corresponds to the port number of the switch. It should be noted that when the number of network nodes in the first node group is equal to the number of switch ports, the number of the network node in the first node group and the port number of the switch can be one-to-one corresponding.

[0116] Furthermore, the data routing system also includes a third topology network. The second topology network is nested within the third topology network, and the network structure of the third topology network is a fully interconnected network topology structure composed of a third node group. In this case, the node coordinates of the network node can be determined based on the number of the network node within the first node group (e.g., the number within the cabinet), the number of the first node group in which the network node is located, and the number of the second node group in which the network node is located.

[0117] For example, when the data routing system is a hierarchical network with a nested fat tree network topology and a two-layer fully interconnected network topology, the node coordinates can be represented by the number of the network node in the first node group and the number of the first node group and the second node group in which the network node is located. For example, the node coordinates can be represented as {G, L, R}. Among them, G represents the number of the second node group in which the network node is located, such as the number of the L2 group, and L represents the number of the first node group in which the network node is located, such as the number of the L1 group. R represents the number of the network node in the cabinet. In some examples, the value range of G can be 0 to 4160 (including endpoint values), the value range of L can be 0 to 64 (including endpoint values), and the value range of R can be 1 to 64 (including endpoint values). Among them, the coordinates of the switch connecting the network nodes in the first node group in the data routing system can be represented as {G, L, 0}.

[0118] It should be noted that a unified numbering scheme can be used for network nodes such as routing engines integrated into processors or independent low-port routers. Accordingly, network nodes can determine data routing paths based on the node coordinates represented by the numbers, and implement data routing between computing nodes based on the data routing paths.

[0119] Among them, network nodes can determine data routing paths using the shortest path routing algorithm based on node coordinates. The shortest path routing algorithm can provide the shortest distance communication between the source node and the destination node, with the lowest communication latency, and is the most basic routing algorithm. Because this application is a hierarchical network with nested fat trees and full meshes (for example, a two-layer full mesh), a deterministic shortest path routing algorithm that adapts to topological characteristics has the advantages of rapid computation and is easy to implement in hardware.

[0120] The following describes the specific implementation of data routing using a deterministic shortest path routing algorithm based on adaptive topology features.

[0121] Referring to the flowchart of a data routing method shown in FIG6 , the method can be applied to the data routing system 10 of the aforementioned embodiment. The data routing system 10 is used to implement data routing between computing nodes. The data routing system 10 includes a first topology network 120 and a second topology network 130. The first topology network 120 is nested in the second topology network 130. The network structure of the first topology network 120 is a fat tree network topology structure composed of a first node group and a switch, the first node group includes a first number of network nodes, and each network node in the first topology network 120 has a cascade relationship with the switch, and the network structure of the second topology network is a fully interconnected network topology structure composed of a second node group, the second node group includes a second number of first node groups, and there is a connection relationship between the second number of first node groups. The method comprises the following steps:

[0122] S602: The current node receives a data packet to be routed.

[0123] Depending on the location of the source and destination nodes in the network, the source node can route data packets to the destination node directly, or it can first route to the next-hop network node and then continue routing from the next-hop network node until it reaches the destination node. When the data is at the source node, the source node can use the shortest path routing algorithm to determine the next-hop node. If the next-hop node is not the destination node, the next-hop node can use the shortest path routing algorithm to determine the next-hop node until it reaches the destination node.

[0124] For ease of description, this application refers to the node where data arrives for processing as the current node. The current node can be the source node or a node on the path from the source node to the destination node. Whether it is the source node or a node on the path from the source node to the destination node, the method of determining the data routing path (next hop node) through the shortest path routing algorithm is similar. This application provides a detailed description of data routing from the perspective of the current node.

[0125] The data packet may be a data packet for a computing task, such as a data packet for a high-performance computing task. The header of the data packet may record information about the source node and the destination node of the data packet, such as the node coordinates of the source node and the destination node, or other information that can be used to determine the node coordinates of the source node and the destination node.

[0126] S604: The current node determines a data routing path according to the node coordinates of the current node and the node coordinates of the destination node.

[0127] The node coordinates are determined based on the node's number within the first node group (e.g., a cabinet) and the number of the first node group in which the node is located. Furthermore, when the data routing topology 10 includes a third topology network, the node coordinates can be determined based on the node's number within the first node group, the number of the first node group in which the network node is located, and the number of the second node group in which the network node is located.

[0128] The current node can parse the data packet to obtain the node coordinates of the destination node. In some examples, the node coordinates of the destination node can be {Gd, Ld, Rd}, where Gd represents the number of the second node group in which the destination node is located, Ld represents the number of the first node group in which the destination node is located, and Rd represents the number of the destination node within the cabinet. The current node can also obtain the node coordinates of the current node. For example, each network node can store its own node coordinates after networking is completed. In this way, the current node can read the node coordinates of the current node from the storage.

[0129] The current node can determine the data routing path based on the node coordinates of the current node and the node coordinates of the destination node, combined with the topology of the data routing system 10 (for example, a topology map). It should be noted that when the node coordinates of the current node and the destination node represent that the current node and the destination node are in the same layer, the data routing path can be determined based on the topology of the layer. For example, when the current node and the destination node are both in the same first topology network (or the same first node group) of the first layer, the current node can determine the data routing path based on the topology of the layer, for example, the topology of the corresponding node group in the layer. For another example, when the current node and the destination node are in different node groups of the same layer, the current node can determine the data routing path based on the topology of the layer.

[0130] Based on this, the current node can first identify whether the node groups (such as the first node group and the second node group) where the current node and the destination node are located are the same based on the node coordinates of the current node and the node coordinates of the destination node, and then determine the data routing path based on the distribution of the node groups where the current node and the destination node are located.

[0131] S606: The current node routes the data packet according to the data routing path.

[0132] Specifically, the data routing path may indicate a routing port or the node coordinates of a next-hop node. Accordingly, the current node may route the data packet to the next-hop node according to the routing port indicated by the data routing path or the node coordinates of the next-hop node.

[0133] Based on the above description, the present application provides a data routing method, which introduces the node coordinates of the current node and the destination node in the data routing system, adapts the topological characteristics of the data routing system to perform data routing, provides the shortest distance communication between the source node and the destination node, and achieves low communication latency. Moreover, the method is simple to implement and has high availability.

[0134] Next, the specific implementation of determining the data routing path when the current node and the destination node are in node groups at different levels in the embodiment of FIG. 6 will be described.

[0135] In a first implementation, when the number of the first node group to which the current node belongs is equal to the number of the first node group to which the destination node belongs, the current node and the destination node are in the same first node group, and the current node can perform intra-group routing within the first node group. The intra-group routing within the first node group can be based on a fat-tree topology. Specifically, the current node can determine a routing port based on the number of the destination node within the first node group (e.g., a cabinet), and the data routing path can include forwarding data packets from the routing port to the destination node.

[0136] In a second implementation, when the number of the second node group to which the current node belongs is equal to the number of the second node group to which the destination node belongs, and the number of the first node group to which the current node belongs is different from the number of the first node group to which the destination node belongs, this indicates that the current node and the destination node are in different first node groups within the same second node group, and the data routing is intra-second node group routing, also known as L2 intra-group routing. Considering that the second node group includes different first node groups, the above routing is also known as inter-first node group routing, also known as L1 inter-group routing.

[0137] The inter-group routing of the first node group may include: the current node may determine whether there is a direct link (such as L-LINK) between the current node and the first node group where the destination node is located. In order to facilitate the distinction from the direct link (such as G-LINK) or other direct links of the second node group where the destination node is located, this application may refer to the direct link with the first node group where the destination node is located as the first direct link.

[0138] If so, the current node can determine that the data routing path includes forwarding the data packet along the first direct link to the first node group where the destination node is located. If not, the current node can determine that the data routing path includes forwarding the data packet to a second jump node. The second jump node has a second direct link to the first node group where the destination node is located. When the data packet arrives at the second jump node, it can reach the first node group where the destination node is located via the second direct link. If the data packet reaches the first node group where the destination node is located, intra-L1 Group routing can be performed.

[0139] In a third implementation, when the number of the second node group to which the current node belongs is different from the number of the second node group to which the destination node belongs, indicating that the current node is in a different second node group, the data routing is intra-group routing within the third node group, also known as L3 intra-group routing. The third node group includes multiple second node groups, and the routing described above can also be referred to as inter-group routing within the second node group, i.e., L2 inter-group routing.

[0140] Inter-group routing for the second node group may include: the current node may determine whether a third direct link exists between the current node and the second node group where the destination node is located. If so, the current node may determine that the data routing path includes forwarding the data packet along the third direct link to the second node group where the destination node is located. When the data packet arrives at the second node group where the destination node is located, intra-L2 group routing may be performed. If not, the current node may determine that the data routing path includes forwarding the data packet to the first jump node. A fourth direct link exists between the first jump node and the second node group where the destination node is located.

[0141] Furthermore, the current node can also determine whether the number of the first node group where the first jump node is located is equal to the number of the first node group where the current node is located. If not, it means that the current node and the first jump node are in different first node groups. The current node determines that the data routing path also includes forwarding the data packet to the ferry node. Among them, there is a fifth direct link between the ferry node and the first node group where the first jump node is located. That is, the current node can first route the data packet to the ferry node. The ferry node and the above-mentioned first jump node are in the same first node group, and the data packet can be routed to the above-mentioned first jump node through L1 group routing. There is a direct link between the first jump node and the second node group where the destination node is located, such as the above-mentioned fourth direct link. When the data packet reaches the first jump node, it can be routed to the second node group where the destination node is located through the above-mentioned fourth direct link. When the data packet reaches the first node group where the destination node is located, L2 group routing can be performed.

[0142] For ease of description, this application uses the topology of Figure 5 as an example to illustrate the data routing method. As shown in Figure 5, based on the physical location of the network node in the topology, the node coordinates can be defined as {G, L, R}, where G represents the number of the second node group in which the network node is located, L represents the number of the first node group in which the network node is located, and R represents the number of the network node within the cabinet. The following describes the connection relationship between the various levels of the topology.

[0143] For the L1 topology, port 1 of the routing engine integrated into each processor in the rack is directly connected to the corresponding port on the TOR switch. For example, port 1 of node 1 (such as N01) is connected to port 1 of the TOR switch, port 1 of node 2 (such as N02) is connected to port 2 of the TOR switch, and port 1 of node 64 (such as N64) is connected to port 64 of the TOR switch. In this way, the 64-port TOR switch can meet the non-blocking interconnection requirements of all 64 nodes in the rack.

[0144] For L2 topologies, port 2 of the routing engine integrated into each processor in the cabinet serves as the interconnection port between racks within the L2 topology and is defined as an L-Link. The L-Link number of each routing engine can be the same as the processor number within the rack, and the L-Link number PL can range from 0 to 63.

[0145] Specifically, the node coordinates of the source node can be expressed as {Gs, Ls, Rs}, the node coordinates of the destination node can be {Gd, Ld, Rd}, the L-Link port 2 of the source node is connected to the L-Link port 2 of the destination node, and the full mesh connection rule between L2 topologies can be expressed as: Ls+PLs=Ld && PLd+PLs=63 (3)

[0146] Among them, Ls is the number of the first node group where the source node is located, also known as the L2 topology number where the source node is located, Ld is the number of the first node group where the destination node is located, that is, the L2 topology number where the destination node is located, PLs is the L-Link port number of the source node connecting to the first node group (such as L2 group) where the destination node is located, and PLd is the L-Link port number of the destination node connecting to the first node group (such as L2 group) where the source node is located. Based on this connection relationship, the routing relationship between the first node groups can be calculated.

[0147] For the L3 topology, port 3 of the routing engine integrated in each processor in the cabinet serves as the interconnection port between each L2 full mesh topology within the L3 topology, and is defined as a G-Link. The numbering of each routing engine's G-Link can correspond one-to-one with the L2 full mesh topology and the node coordinates within the group. The G-Link numbering rule can be: PG = 64 * L + R, and the PG value range can be: 0-4160. Accordingly, the full mesh connection relationship between L3 topologies can be expressed as: Gs + PGs = Gd && PGd + PGs = 4160 (4)

[0148] Among them, Gs is the number of the second node group (such as) where the source node is located, Gd is the number of the second node group (such as L3 topology or L3 group) where the destination node is located, PGs is the number of the G-Link port connecting the source node to the second node group where the destination node is located, and PGd is the number of the G-Link port connecting the destination node to the second node group where the source node is located. Based on this connection relationship, the routing relationship between the second node groups can be calculated.

[0149] The data routing method of the present application is described in detail below based on the connection relationship of each level of topology.

[0150] Referring to FIG7 , a schematic diagram illustrating a data routing process in a data routing system, the current node can first determine whether the destination node and the current node are in the same second node group. If not, inter-group routing within the second node group is performed. If so, the current node then determines whether the destination node and the current node are in the same first node group. If the destination node and the current node are not in the same first node group, inter-group routing within the first node group is performed. If the destination node and the current node are in the same first node group, intra-group routing within the first node group is performed.

[0151] The specific implementations of inter-group routing of the second node group, inter-group routing of the first node group, and intra-group routing of the first node group are described below.

[0152] If the destination node and the current node are in different second node groups, that is, Gd = !Gc, define the first jump node with a direct link to the second node group where the destination node {Gd, Ld, Rd} is located. Based on the topological connection relationship, the node coordinates of the first jump node can be determined as:

[0153] If the current node has a direct link to the second node group (denoted as Gd) where the destination node is located, the following formula is satisfied: PGc=64*Lc+Rc=4161-64*Ld-Rd (6)

[0154] Then the data can be directly forwarded from the G-Link port of the current node to the second node group where the destination node is located. Otherwise, it needs to be routed from the current node to the first jump node first.

[0155] The above formula is explained based on the example that the first node group includes 64 nodes and 65 first node groups are interconnected to form the second node group. In actual application, the maximum number of nodes in the first node group can be expressed as Kr, and the maximum number of first node groups in the second node group can be Kl. Accordingly, the current node can determine whether the current node is the first jump node of the second node group where the destination node is located by judging whether the following formula is true: Kr*Lc+Rc==Kl-Kr*Ld-Rd (7)

[0156] Accordingly, the node coordinates of the first jump node can be expressed as {Gc, Lm, Rm}, where Lm and Rm satisfy the following formula:

[0157] If the first node group where the first jump node is located is different from the first node group where the current node is located, that is, Lm=!Lc, the data packet must first be routed to the first node group where the first jump node is located (denoted as Lm). The node with a direct link to the first node group Lm where the first jump node is located is defined as a ferry node. The node coordinates of the ferry node can be expressed as {Gn, Ln, Rn}. Therefore, the data packet can be routed to the ferry node first, and then forwarded from the direct link L-Link to the first node group where the first jump node is located. The physical location of the ferry node can be obtained according to the following formula: Gn=Gc, Ln=Lc, Rn=PLn=Kr-1-Rm (9)

[0158] Upon reaching the ferry node, the packet can be forwarded by the TOR switch to the first hop node via the first node group's intra-group routing (e.g., L1-level RACK internal routing). After reaching the first hop node, the packet can be forwarded via the direct link L-Link port, reaching the second node group where the destination node is located. The packet can then reach the destination node via the second node group's intra-group routing.

[0159] The following describes in detail the intra-group routing of the second node group (eg, the inter-group routing of the first node group).

[0160] The current node can determine whether the current node and the destination node are in the same second node group based on the node coordinates. If the destination node and the current node are in the same second node group, that is, Gd = Gc, but Ld = !Lc, the current node can determine whether there is a direct link between the current node and the first node group where the destination node is located based on the topological connection relationship.

[0161] If the current node is a direct link to the first node group where the destination node is located, data can be forwarded directly from the direct link. For details, see the following formula: Ld-Kr+1+Rd==Rc (10)

[0162] If formula (10) is satisfied, it means that the current node is a direct link connected to the first node group where the destination node is located. Taking Kr=64 as an example, if Lc=Ld-PLc&&PLc=63-PLd, that is, Lc=Ld-63+Rd=Rc, it means that the current node is a direct link connected to the first node group where the destination node is located. The current node can directly forward the data packet from the direct link connected to the first node group where the destination node is located, such as L-Link, to the first node group where the destination node is located. Otherwise, the node with a direct link connected to the first node group where the destination node is located is defined as the second jump node, and the node coordinates of the second jump node can be determined by the following formula: Gm=Gc, Lm=Ld-PLm && PLm=Kr-1-PLd, where PLd=Rd (11)

[0163] Using Kr = 64 as an example, the node coordinates of the second hop node can be Gm = Gc, Lm = Ld - PLm & PLm = 63 - PLd, and PLd = Rd. In other words, Gm = Gc, Lm = Ld - 63 + Rd & PLm = 63 - Rd. Therefore, the route is first to the second hop node: Rm = PLm. Then, the route is forwarded to the first node group's intra-group routing, where the TOR switch performs routing based on the first node group's internal numbering.

[0164] The following describes the intra-group routing of the first node group.

[0165] The destination node and the current node are in the same first and second node groups, i.e., Gd == Gc && Ld == Lc. Therefore, the route can be determined based solely on the destination node's number within the first node group, such as its number within the RACK. The TOR switch forwards the data packet from the Rd port to the destination node. Upon arrival, the DMA communication engine receives the data packet, completing the communication.

[0166] Considering the existence of loops in a full mesh topology, deadlock risk can arise. Deadlock refers to the cyclical occupation of communication resources (such as caches). For example, each node sends data to the next adjacent node: node 0 sends to node 2, node 1 sends to node 3, and so on. If four nodes send data simultaneously, the port resources will be cyclically occupied, which will cause deadlock, communication congestion, and even network paralysis in severe cases. Based on this, the data routing system of the present application can also achieve anti-deadlock through virtual lanes (VL).

[0167] Deadlock can be effectively avoided based on virtual channels. In the data routing system of the present application, since the second topology network (such as L2 level topology) and the third topology network (such as L3 level topology) adopt a full mesh architecture, multiple virtual channels can be allocated to the ports of network nodes in the second topology network and the third topology network to avoid deadlock.

[0168] Specifically, ports corresponding to the second topology network are configured with a first virtual channel and a second virtual channel, while ports corresponding to the third topology network are configured with a third virtual channel and a fourth virtual channel. Using the second topology network as an example, if the destination address is higher than the current address, the first virtual channel (e.g., virtual channel 0) can be used; if the destination address is lower than the current address, the second virtual channel (e.g., virtual channel 1) can be used. As shown in Figure 8, inter-node communication relies on non-looping, thus avoiding deadlock. Similar to the second topology network, in the third topology network, loops will not form between virtual channels, thus preventing loop deadlock between layers.

[0169] Considering the long runtimes and large network scales of large-scale applications like HPC, component failures are inevitable. A data routing system with backup resources that can quickly replace failed nodes would greatly improve system availability. The topology proposed in this application provides rack-level resource backup. For node-level or rack-level failures, backup rack-level resources can be used to quickly replace failed nodes, improving system availability.

[0170] In some possible implementations, the faults may include faults of different types or granularities. The following provides examples of rule recovery for different types of faults.

[0171] The L2-level topology reserves a first node group, such as a RACK, as a hot backup resource. When a service node in the first node group other than the hot backup resource fails, as shown in FIG9 , the job scheduling system (such as a job scheduler) can allocate a node in the backup RACK that has a direct link to the RACK where the failed node is located as a replacement node, which can be routed to the node directly connected to the backup RACK through the TOR switch, and the job can be scheduled to the node in the backup RACK, so that the failed node can be replaced quickly without affecting the original job, effectively improving system performance. Among them, the backup RACK has a direct link to other RACKs in the L2-level topology, and the corresponding node is directly connected to the failed RACK through the L2 global optical fiber link, which can quickly replace a single node failure.

[0172] If the failure involves multiple nodes, the job scheduling system can also use the TOR within the rack to connect multiple backup nodes within the backup rack. For example, if there are two failed nodes, the six direct links between the two failed nodes can be connected, and the backup nodes in the backup rack connected by the 12 direct links can be used as replacement nodes to continue participating in the operation.

[0173] When the fault is a RACK-level fault, the job scheduling system can use the backup RACK to replace the faulty RACK, thereby quickly restoring operation and greatly improving system availability.

[0174] The cabinet-level resource backup solution of this application can support the rapid replacement of single-node, multi-node or even entire cabinet failures, which can effectively improve system availability and accelerate application operation efficiency.

[0175] Based on the aforementioned data routing system 10 and data routing method, the present application also provides a method for constructing a data routing system. The method can be executed by a management device. The following describes the method for constructing a data routing system from the perspective of a management device.

[0176] Referring to the flowchart of a method for constructing a data routing system shown in FIG10 , the method includes the following steps:

[0177] S1002: The management device determines a plurality of network nodes and switches for constructing a data routing system.

[0178] The plurality of network nodes are divided into a second number of first node groups, each of which includes the first number of network nodes. After the plurality of network nodes are physically connected, for example, via a physical link, the management device can determine the plurality of network nodes used to construct the data system through node discovery. The management device can discover new network nodes through a heartbeat mechanism, for example. This is not further described here.

[0179] Furthermore, the plurality of network nodes may be divided into a third number of second node groups, wherein the second node groups may include the second number of first node groups, and the third number of second node groups constitute a third node group.

[0180] Switches can include in-cabinet switches, specifically switches installed within the cabinet where the network nodes reside. To facilitate cabling, the cabinet switch can be installed on top of the cabinet, meaning it can be a TOR switch. In some examples, switches can also include inter-cabinet switches. Different in-cabinet switches can be connected to the inter-cabinet switch. This allows network nodes in different cabinets to communicate through the switch.

[0181] S1004. The management device sends a configuration file to the switch and multiple network nodes, so that the network nodes in the first node group are cascaded with the switch respectively according to the network structure of the data routing system in the configuration file to form a first topology network with a fat tree network topology structure, and establish connections between the second number of first node groups to form a second topology network with a fully interconnected network topology structure.

[0182] Among them, the network nodes in the first node group can establish connections with the corresponding ports of the switch according to the network structure of the data routing system in the configuration file, thereby realizing cascading with the switch, and the switch cascaded with the network nodes can realize data routing between different network nodes. Similarly, the network nodes in the first node group can establish connections between the second number of first node groups according to the network structure of the data routing system in the configuration file. It should be noted that the connection between the node groups can include the connection between the network node in one node group and the network node in another node group. In the third topology network, the second node groups can include at least one connection, for example, the connection between the second node group A and the second node group B can include the connection between the node 1 in the second node group A and the node 2 in the second node group B. Furthermore, the connection between the second node group A and the second node group B can also include the connection between the node 11 in the second node group A and the node 12 in the second node group B.

[0183] Furthermore, when the plurality of network nodes are divided into a third number of second node groups, the configuration file is further used to establish connections between the network nodes in the first node group and the third number of second node groups, thereby forming a third topology network having a fully interconnected network topology. Accordingly, the data routing system further includes a third topology network, with the second topology network nested within the third topology network.

[0184] In some possible implementations, the management device may also check the connection relationships of multiple network nodes based on the network structure of the data routing system in a configuration file, such as a network structure represented by a topology diagram. The network structure in the configuration file is the desired network structure, and the connection relationships are the desired connection relationships. The management device may also obtain the actual connection relationships of multiple network nodes through probing. The management device may compare the desired connection relationships with the actual connection relationships to check the connection relationships of multiple network nodes.

[0185] If the connection relationship between multiple network nodes (e.g., the actual connection relationship) is inconsistent with the connection relationship in the network structure of the data routing system, the management device can also send a prompt message to the user. This prompt message can be used to prompt the user to adjust the connection relationship between the multiple network nodes. In this way, the accuracy and reliability of the network can be guaranteed.

[0186] The management device is further configured to distribute a routing table so that the data routing system can perform data routing based on the routing table. Specifically, the management device may generate a routing table based on the node coordinates of multiple network nodes. The node coordinates are determined based on the number of the network node within the first node group and the number of the first node group to which the node belongs. The management device may then distribute an entry in the routing table related to the at least one network node to at least one of the multiple network nodes.

[0187] The management device sends routing table entries related to at least one network node to reduce storage resources occupied by routing table entries in the network node, thereby saving storage space. It should be noted that if the network node has sufficient storage resources, the management device may also send a complete routing table to at least one network node, and this embodiment does not impose a limitation on this.

[0188] Based on the above description, it can be seen that the method of constructing a data routing system of the present application is to determine the switch and multiple network nodes used to construct the data routing system, and instruct the network nodes to establish connections with the switch and the network nodes in other node groups according to the network structure of the data routing system in the configuration file, thereby constructing a data routing system with a nested fat tree network topology structure and a fully interconnected topology network structure, thereby realizing the formation of a large-scale or ultra-large-scale network, effectively reducing network costs and power consumption, and improving network scalability. The fusion topology structure of the present application provides direct routing for global communications, alleviates the global network congestion level, and can reduce expensive global optical fiber links relative to the fat tree topology. In addition, the network node can be implemented by integrating a routing engine in the processor, so that a switch-free architecture can be realized, effectively reducing communication delays, and further reducing network costs and power consumption.

[0189] The hierarchical network structure of nested fat tree and full mesh topologies proposed in this application can effectively reduce the network radius and provide a low-cost, low-power, high-performance, high-availability, high-reliability, and high-scalability network.

[0190] Table 1 compares the cost and power consumption of networking for future 10E-class supercomputers. Switches cost approximately $1,000 and consume approximately 220W of power, while optical modules cost approximately $200 and consume approximately 12W of power. A 10E-class supercomputer network can scale to 500,000 nodes. Currently, mainstream high-performance networks primarily use a fat-tree topology. Based on the industry's leading 64-port high-performance switches with 400Gbps port bandwidth, a four-layer fat-tree topology is typically required to support a 500,000-node network.

[0191] Table 1

[0192] As shown in Table 1, based on a four-layer fat tree network, 54,880 switches and 3,010,560 optical modules are usually required. Using the converged topology structure of this application, 7,865 switches of the same specifications are usually required, which is only 14.33% of the number of fat trees, and the number of optical modules is only 33.44% of the fat tree. Reducing the number of optical modules can effectively improve system reliability. The normalized network cost is only 31.74% of that of a fat tree; the network power consumption is only 26.13% of that of a fat tree. While network costs usually account for 10% of the cluster cost, using this application can save nearly 70% of the network cost.

[0193] This application's multi-level full mesh networking achieves non-blocking performance with a network diameter of only seven hops, enabling low latency and high throughput, significantly improving performance. Furthermore, a low-port router or processor with at least a three-port routing engine integrated within it can build a 17 million-port network, offering excellent scalability. Compared to a four-layer fat-tree network, this application's networking solution reduces costs by 68% and power consumption by 73%, achieving low-cost, low-power networking. Furthermore, it enables rack-level resource backup, increasing system availability by 70%.

[0194] Furthermore, the present application can also support the use of optical switches to replace optical fibers. Optical switches have the characteristics of adjustable topology, dynamic programming, fault tolerance, etc., and can adapt to large-scale network requirements.

[0195] Based on the aforementioned method for constructing a data routing system, the present application further provides a management device. The management device will be introduced below from the perspective of functional modularization.

[0196] Referring to FIG. 11 , a schematic diagram of a management device is shown. As shown in FIG. 11 , the management device 1100 includes:

[0197] A determining module 1102 is configured to determine a plurality of network nodes and switches for constructing a data routing system, wherein the plurality of network nodes are divided into a second number of first node groups, the first node groups including a first number of network nodes;

[0198] The sending module 1104 is used to send a configuration file to multiple network nodes and switches, so that the network nodes in the first node group are cascaded with the switches respectively according to the network structure of the data routing system described in the configuration file to form a first topology network with a fat tree network topology structure, and establish connections between the second number of first node groups to form a second topology network with a fully interconnected network topology structure.

[0199] The data routing system includes a first topology network and a second topology network, and the first topology network is nested in the second topology network.

[0200] The above-mentioned determination module 1102 and sending module 1104 can be implemented by hardware or software.

[0201] When implemented by software, the determination module 1102 and the sending module 1104 can be applications running on the management device 1100, such as a computing engine. Taking the determination module 1102 as an example, the determination module 1102 can be virtualized through a virtualization service and provided to users for use. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, and container services. Among them, the VM service can be a service that uses virtualization technology to virtualize a virtual machine (VM) resource pool on multiple physical hosts to provide VMs for users to use on demand. The BMS service is a service that virtualizes a BMS resource pool on multiple physical hosts to provide BMS for users to use on demand. The container service is a service that virtualizes a container resource pool on multiple physical hosts to provide containers for users to use on demand. VM is a simulated virtual computer, that is, a logical computer. BMS is a high-performance computing service that is elastically scalable and has computing performance that is no different from that of a traditional physical machine and has the characteristics of secure physical isolation. Containers are a kernel virtualization technology that can provide lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service in the above virtualization services are only specific examples. In actual applications, virtualization services can also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0202] When implemented in hardware, determination module 1102 may include at least one computing device, such as a server. Alternatively, determination module 1102 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Transmission module 1104 may be implemented using a transceiver or a transceiver unit. The transceiver may include, but is not limited to, a radio frequency transceiver and a fiber optic transceiver.

[0203] In some possible implementations, the plurality of network nodes are divided into a third number of second node groups, where the second node groups include the second number of first node groups.

[0204] The configuration file is further used to establish connections between the network nodes in the first node group and a third number of the second node groups to form a third topology network having a fully interconnected network topology. The data routing system also includes the third topology network, and the second topology network is nested within the third topology network.

[0205] In some possible implementations, the management device 1100 may further include a checking module 1106 and a prompting module 1108. The checking module 1106 is configured to check the connection relationship of the multiple network nodes based on the network structure of the data routing system in the configuration file; the prompting module 1108 is configured to send a prompt message to the user if the connection relationship of the multiple network nodes is inconsistent with the connection relationship in the network structure of the data routing system, and the prompt message is configured to prompt the user to adjust the connection relationship of the multiple network nodes.

[0206] The checking module 1106 and the prompting module 1108 may be implemented by software or hardware.

[0207] When implemented by software, the checking module 1106 and the prompting module 1108 may be applications running on the management device 1100. The applications may also be virtualized through a virtualization service and provided to the user. For example, the checking module 1106 may be virtualized through a VM service, a BMS service, or a container service and provided to the user.

[0208] When implemented by hardware, the checking module 1106 and the prompting module 1108 may include at least one computing device, such as a server, etc. Alternatively, the checking module 1106 and the prompting module 1108 may also be implemented using an ASIC or a programmable logic device (PLD).

[0209] In some possible implementations, the management device 1100 may further include a generation module 1109. The generation module 1109 is configured to generate a routing table based on node coordinates of multiple network nodes, where the node coordinates are determined based on the number of the network node within the first node group and the number of the first node group to which the node belongs. The sending module 1104 is further configured to send an entry in the routing table related to the at least one network node to at least one of the multiple network nodes.

[0210] The following describes the management device 1100 of the present application from the perspective of hardware implementation. The management device 1100 can be a computing device, such as a server, or a terminal device. The terminal device can be a desktop computer, a laptop computer, a thin client, or a smartphone. The following describes the structure of the computing device of the present application with reference to the accompanying drawings.

[0211] This application also provides a computing device 1200. As shown in Figure 12, computing device 1200 includes a bus 1202, a processor 1204, a memory 1206, and a communication interface 1208. Processor 1204, memory 1206, and communication interface 1208 communicate with each other via bus 1202. Computing device 1200 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 1200.

[0212] Bus 1202 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG12 shows a single bus line, but this does not imply a single bus or type of bus. Bus 1202 may include a path for transmitting information between various components of computing device 1200 (e.g., memory 1206, processor 1204, and communication interface 1208).

[0213] The processor 1204 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0214] The memory 1206 may include volatile memory, such as random access memory (RAM). The memory 1206 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). The memory 1206 stores executable program code, and the processor 1204 executes the executable program code to implement the aforementioned method for constructing a data routing system. Specifically, the memory 1206 stores instructions for the management device 1100 to execute the method for constructing a data routing system.

[0215] The communication interface 1208 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1200 and other devices or a communication network.

[0216] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0217] As shown in Figure 13, the computing device cluster includes at least one computing device 1200. The memory 1206 of one or more computing devices 1200 in the computing device cluster may store instructions for executing the method for building a data routing system by the same management device 1100.

[0218] In some possible implementations, one or more computing devices 1200 in the computing device cluster may also be used to execute some of the instructions of the management device 1100 for executing the method for building a data routing system. In other words, the combination of one or more computing devices 1200 may jointly execute the instructions of the management device 1100 for executing the method for building a data routing system.

[0219] It should be noted that the memories 1206 in different computing devices 1200 in the computing device cluster may store different instructions for executing part of the functions of the management device 1100 .

[0220] FIG14 illustrates a possible implementation. As shown in FIG14 , two computing devices 1200A and 1200B are connected via a communication interface 1208. The memory of computing device 1200A stores instructions for executing the functions of determination module 1102. The memory of computing device 1200B stores instructions for executing the functions of sending module 1104. Furthermore, the memory of computing device 1200A also stores instructions for executing the functions of checking module 1106 and prompting module 1108, and the memory of computing device 1200B also stores instructions for executing the functions of generating module 1109. In other words, the memories 1206 of computing devices 1200A and 1200B jointly store instructions for management device 1100 to execute the method for constructing a data routing system.

[0221] The connection method between the computing device clusters shown in Figure 14 may be considered to be based on the fact that the method for building a data routing system provided in this application requires a large amount of resources to send configuration files and routing table entries to a large number of network nodes. Therefore, it is considered to transfer the functions implemented by the sending module 1104 and the generating module 1109 to the computing device 1200B.

[0222] It should be understood that the functionality of the computing device 1200A shown in FIG14 may also be implemented by multiple computing devices 1200. Similarly, the functionality of the computing device 1200B may also be implemented by multiple computing devices 1200.

[0223] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), etc. FIG. 15 illustrates a possible implementation. As shown in FIG. 15 , two computing devices 1200C and 1200D are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 1206 in the computing device 1200C stores instructions for executing the functions of the determination module 1102. Simultaneously, the memory 1206 in the computing device 1200D stores instructions for executing the functions of the sending module 1104. The memory in the computing device 1200C also stores instructions for executing the functions of the inspection module 1106 and the prompt module 1108, and the memory in the computing device 1200D also stores instructions for executing the functions of the generation module 1109.

[0224] It should be understood that the functionality of the computing device 1200C shown in FIG15 may also be implemented by multiple computing devices 1200. Similarly, the functionality of the computing device 1200D may also be implemented by multiple computing devices 1200.

[0225] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct a management device to execute a method for constructing a data routing system.

[0226] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a management device, the management device executes the method for constructing a data routing system.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data routing system, characterized in that: The data routing system is used to implement data routing between computing nodes, and the data routing system includes a first topology network and a second topology network, wherein the first topology network is nested in the second topology network; The network structure of the first topology network is a fat tree network topology structure consisting of a first node group and a switch, the first node group includes a first number of network nodes, and each network node in the first topology network is in a cascade relationship with the switch; The network structure of the second topology network is a fully interconnected network topology structure composed of second node groups, wherein the second node groups include a second number of first node groups, and a connection relationship exists between the second number of first node groups.

2. The system according to claim 1, wherein: In the first topology network, network nodes belonging to the same first node group are directly connected to the same switch.

3. The system according to claim 1, wherein: The first node group includes multiple subgroups, and the first topology network includes multiple first-layer switches and at least one second-layer switch corresponding to the multiple subgroups. In the first topology network, network nodes belonging to the same subgroup are directly connected to the first-layer switch corresponding to the subgroup, and the multiple first-layer switches are upwardly connected to the at least one second-layer switch.

4. The system according to claim 3, characterized in that The subgroups are divided according to cabinets, cabinet groups or computer rooms.

5. The system according to any one of claims 2 to 4, characterized in that In the first topology network, network nodes belonging to the same first node group are directly connected to the same first switch and the same second switch. The first switch is used to forward intra-group traffic, and the second switch is used to forward inter-group traffic. The source node and destination node of the intra-group traffic are in the same first node group, and the source node and destination node of the inter-group traffic are in different first node groups.

6. The system according to any one of claims 1 to 5, characterized in that Each network node includes n ports, and m ports among the n ports are used to connect to switches in the first topology network.

7. The system according to claim 6, characterized in that H ports among the n ports are used to connect network nodes in a fully interconnected network topology structure formed by the second node group, and the second number is less than or equal to h·the first number+1.

8. The system according to any one of claims 1 to 7, characterized in that The data routing system further includes a third topology network, wherein the second topology network is nested within the third topology network; The network structure of the third topology network is a fully interconnected network topology structure composed of a third node group, wherein the third node group includes a third number of second node groups, and a connection relationship exists between the third number of second node groups.

9. The system according to claim 8, characterized in that K ports out of the n ports of each network node are used to connect network nodes in the fully interconnected network topology structure formed by the third node group, and the third number is less than or equal to k·the number of nodes in the second node group+1.

10. The system according to any one of claims 1 to 9, characterized in that The network node is a switching node independent of the computing node, or a routing engine integrated into the computing node.

11. The system according to any one of claims 1 to 10, characterized in that The node coordinates of the network node are determined based on the number of the network node in the first node group and the number of the first node group to which the network node belongs, and the number of the network node in the first node group corresponds to the port number of the switch; The network node is used to determine a data routing path based on the node coordinates, and implement data routing between the computing nodes according to the data routing path.

12. The system according to any one of claims 1 to 11, characterized in that The computing nodes are used to perform high performance computing (HPC) tasks.

13. A data routing method, characterized in that: Applied to a data routing system, the data routing system is used to implement data routing between computing nodes, the data routing system includes a first topology network and a second topology network, the first topology network is nested within the second topology network, the network structure of the first topology network is a fat tree network topology structure composed of a first node group and a switch, the first node group includes a first number of network nodes, each network node in the first topology network is in a cascade relationship with the switch, the network structure of the second topology network is a fully interconnected network topology structure composed of a second node group, the second node group includes a second number of first node groups, and the second number of first node groups are connected to each other, the method comprising: A current node in the data routing system receives a data packet to be routed, the current node being a source node of the data packet or a node on a path from the source node to a destination node; The current node determines a data routing path according to node coordinates of the current node and node coordinates of the destination node, wherein the node coordinates are determined based on a number of the node in the first node group and a number of the first node group to which the node belongs; The current node routes the data packet according to the data routing path.

14. The method according to claim 13, characterized in that The current node determines a data routing path according to the node coordinates of the current node and the node coordinates of the destination node, including: When the number of the first node group where the current node is located is equal to the number of the first node group where the destination node is located, the routing port is determined according to the number of the destination node in the first node group.

15. The method according to claim 13, characterized in that The current node determines a data routing path according to the node coordinates of the current node and the node coordinates of the destination node, including: When the number of the first node group where the current node is located is not equal to the number of the first node group where the destination node is located, the current node determines whether there is a first direct link between the current node and the first node group where the destination node is located; If so, the current node determines that the data routing path includes forwarding the data packet from the first direct link to the first node group where the destination node is located; if not, the current node determines that the data routing path includes forwarding the data packet to a second jump node, and the second jump node has a second direct link with the first node group where the destination node is located.

16. The method according to claim 13, characterized in that The data routing system further includes a third topology network, wherein the second topology network is nested within the third topology network, wherein the network structure of the third topology network is a fully interconnected network topology structure composed of a third node group, wherein the third node group includes a third number of second node groups, and a connection relationship exists between the third number of second node groups, and the node coordinates are determined based on the number of the node in the first node group, the number of the first node group to which the node belongs, and the number of the second node group to which the node belongs; The current node determines a data routing path according to the node coordinates of the current node and the node coordinates of the destination node, including: When the number of the second node group where the current node is located is not equal to the number of the second node group where the destination node is located, the current node determines whether a third direct link exists between the current node and the second node group where the destination node is located; If so, the current node determines that the data routing path includes forwarding the data packet from the third direct link to the second node group where the destination node is located; if not, the current node determines that the data routing path includes forwarding the data packet to the first jump node, and there is a fourth direct link between the first jump node and the second node group where the destination node is located.

17. The method according to claim 16, characterized in that The method further comprises: The current node determines whether the number of the first node group where the first jump node is located is equal to the number of the first node group where the current node is located; If not, the current node determines that the data routing path further includes forwarding the data packet to a ferry node, and a fifth direct link exists between the ferry node and the first node group where the first jump node is located.

18. The method according to any one of claims 13 to 17, characterized in that The data packets include data packets for high performance computing (HPC) tasks.

19. A method for constructing a data routing system, characterized in that: Applied to managing devices, the method includes: determining a plurality of network nodes and switches for constructing the data routing system, the plurality of network nodes being divided into a second number of first node groups, the first node groups including a first number of network nodes; A configuration file is sent to the multiple network nodes and the switch, so that the network nodes in the first node group are cascaded with the switch respectively according to the network structure of the data routing system in the configuration file to form a first topology network with a fat tree network topology structure, and connections are established between the second number of first node groups to form a second topology network with a fully interconnected network topology structure, the data routing system includes the first topology network and the second topology network, and the first topology network is nested in the second topology network.

20. The method according to claim 19, wherein the plurality of network nodes being divided into a third number of second node groups, the second node groups including the second number of first node groups; The configuration file is also used for the network nodes in the first node group to establish connections between the third number of second node groups to form a third topology network with a fully interconnected network topology structure. The data routing system also includes the third topology network, and the second topology network is nested in the third topology network.

21. The method according to claim 19 or 20, characterized in that The method further comprises: Checking the connection relationship of the plurality of network nodes according to the network structure of the data routing system in the configuration file; If the connection relationship between the plurality of network nodes is inconsistent with the connection relationship in the network structure of the data routing system, a prompt message is sent to the user, where the prompt message is used to prompt the user to adjust the connection relationship between the plurality of network nodes.

22. The method according to any one of claims 19 to 21, characterized in that The method further comprises: generating a routing table according to node coordinates of the plurality of network nodes, wherein the node coordinates are determined based on the number of the network node in the first node group and the number of the first node group to which the node belongs; Sending an entry related to the at least one network node in the routing table to at least one network node among the multiple network nodes.

23. A management device, characterized in that: The management device includes at least one processor and at least one memory, wherein the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions to enable the computing device cluster to execute the method according to any one of claims 19 to 22.

24. A computer-readable storage medium, characterized in that The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 19 to 22.

Images