Computing node and computing cluster

Abstract

The invention provides a computing node and a computing cluster, the computing node comprises a universal central processing unit and at least one computing accelerator, the central processing unit is in communication connection with the computing accelerator through an on-chip communication interface, and the central processing unit is further provided with a distributed system communication interface. The central processor and the computing accelerator communicate with other computing nodes in the computing cluster by means of the distributed system communication interface. According to the computing node provided by the technical scheme, high-efficiency and low-energy-consumption communication between the central processing unit and the computing accelerator is realized through the on-chip communication interface, and high-efficiency and low-delay data exchange among the nodes is realized through the distributed system communication interface. Therefore, according to the technical scheme, the communication energy consumption can be remarkably saved, and the communication delay is reduced, so that the overall performance and energy efficiency of the computing cluster are improved.

Description

Technical Field

[0001] The present invention relates to the field of computing power service technology, and in particular to a computing node and a computing cluster. Background Art

[0002] With the rapid development of large-model technology and the initial emergence of general artificial intelligence, the demand for computing clusters to support the training and development of various artificial intelligence algorithms is growing, and the challenges to computing power and energy are becoming more severe.

[0003] According to the Scaling Law, the intelligence level of a large model is proportional to the size of the model parameters and the computing power required for training. Currently, the parameter size of large models is 1,000 times larger than that of the human brain. Therefore, the computing power required to support the training of these large models is expected to increase to 1,000 times the current level.

[0004] Data centers have become one of the main sources of energy consumption in supporting large model training. Therefore, improving the energy efficiency of artificial intelligence computing power has become a top priority.

[0005] However, as Moore's Law slows down, the computing power of a single computing node has grown very slowly. Currently, the main way to increase computing power is through the interconnection of multiple computing nodes, such as "thousand-card interconnection", which is to interconnect thousands of GPGPU / NPU / TPU acceleration cards through a dedicated network to form a computing cluster; the larger-scale "ten thousand-card interconnection" or "hundreds of thousands of cards interconnection" further expands the computing cluster to tens of thousands or even hundreds of thousands of cards.

[0006] In the design of computing clusters, the choice of Internet protocol is crucial. Currently, the most widely used is the RDMA network protocol. This protocol is mature, open, supports large-scale networking, and has the characteristics of low latency, making it the first choice for building large-scale computing clusters.

[0007] In computing nodes, a heterogeneous computing platform is usually composed of CPU and accelerator cards such as GPGPU / NPU / TPU. How to efficiently implement the RDMA network interface on this platform has become a key challenge in designing and optimizing high-performance computing clusters.

[0008] In the prior art, the CPU and GPGPU within a node are interconnected via NVLINK-C2C, the CPUs between nodes are interconnected via RDMA, and the GPGPUs between nodes are interconnected via NVLINK. The three interconnection networks exist simultaneously, resulting in large redundancy in performance, high energy consumption, and unsatisfactory communication delay. Summary of the invention

[0009] The purpose of the present invention is to provide a computing node and a computing cluster that can effectively reduce the energy consumed in the communication process and reduce the communication delay.

[0010] In order to achieve the above-mentioned object, the present invention provides a computing node for constructing a computing cluster, wherein the computing node includes a general-purpose central processing unit and at least one computing accelerator, wherein the central processing unit and the computing accelerator are connected to each other through an on-chip communication interface, and the central processing unit is also provided with a distributed system communication interface, wherein the central processing unit and the computing accelerator communicate with other computing nodes in the computing cluster by means of the distributed system communication interface.

[0011] Preferably, the intra-chip communication interface includes any one of a CXL interface and a UCIe interface.

[0012] Preferably, the distributed system communication interface includes an RDMA network interface.

[0013] Preferably, the computing accelerator includes any one or more of GPGPU, NPU, and TPU.

[0014] The present invention also provides a computing cluster, which includes a plurality of computing nodes, any computing node including a general-purpose central processing unit and at least one dedicated computing accelerator; in any of the computing nodes, the central processing unit and the computing accelerator are communicatively connected via an on-chip communication interface; the central processing unit is also provided with a distributed system communication interface, and the central processing unit and the computing accelerator in any of the computing nodes communicate with other computing nodes via the distributed system communication interface.

[0015] Preferably, the intra-chip communication interface includes any one of a CXL interface and a UCIe interface.

[0016] Preferably, the distributed system communication interface includes an RDMA network interface.

[0017] Preferably, the computing accelerator includes any one or more of GPGPU, NPU, and TPU.

[0018] Compared with the prior art, the computing nodes provided by the above technical solution of the present invention realize efficient and low-energy communication between the central processing unit and the computing accelerator through the on-chip communication interface, and realize efficient and low-latency data exchange between nodes through the distributed system communication interface. Therefore, the technical solution can significantly save communication energy consumption and reduce communication delay, thereby improving the overall performance and energy efficiency of the computing cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1A communication interface diagram of a computing node in one embodiment of the present invention.

[0020] Figure 2 A communication interface diagram of a computing node in another embodiment of the present invention.

[0021] Figure 3 A communication interface diagram of a computing cluster in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0023] This embodiment discloses a computing cluster for providing large-scale computing services, such as Figures 1 to 3 , which includes several computing nodes, each of which includes a general-purpose central processing unit CPU and at least one dedicated computing accelerator.

[0024] In any of the computing nodes, the CPU and the computing accelerator are communicatively connected via an on-chip communication interface J1.

[0025] The CPU is also provided with a distributed system communication interface J2, and the CPU and the computing accelerator in any computing node communicate with other computing nodes by means of the distributed system communication interface J2.

[0026] For example, Figure 3 In a computing cluster, there are three computing nodes, namely, node A, node B, and node C. Each computing node has a CPU and a computing accelerator. For any computing node, such as node A, in node A, the CPU and the computing accelerator are connected to each other through the on-chip communication interface J1. For any two computing nodes, such as node A and node B, the processing unit in the CPU in node A communicates with the CPU or the processing unit in the computing accelerator in node B based on the distributed system communication interface J2. At the same time, the processing unit in the computing accelerator in node A communicates with the CPU or the processing unit in the computing accelerator in node B based on the distributed system communication interface J2. It should be noted that the processing unit is a general term for dedicated acceleration processing components that perform general processing and various types of algorithms.

[0027] On the other hand, the intra-chip communication interface J1 includes any one of a CXL interface and a UCIe interface.

[0028] The distributed system communication interface J2 includes an RDMA network interface.

[0029] On the other hand, the computing accelerator includes any one or more of GPGPU, NPU, and TPU.

[0030] GPGPU (full name: General-purpose computing on graphics processing units) is a graphics accelerator that supports general-purpose computing.

[0031] NPU (full name: Neural Processing Unit) is a processor used to accelerate artificial intelligence neural network processing algorithms.

[0032] TPU (full name: Tensor Processing Unit) is a processor specifically designed to accelerate tensor processing in machine learning algorithms.

[0033] RDMA (full name: Remote Direct Memory Access) is a technology that bypasses the kernel of the remote host operating system to access data in its memory.

[0034] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. A computing node, used to build a computing cluster, characterized in that: The computing node includes a general-purpose central processing unit and at least one dedicated computing accelerator. The central processing unit and the computing accelerator are communicatively connected via an on-chip communication interface. The central processing unit is also provided with a distributed system communication interface. The central processing unit and the computing accelerator communicate with other computing nodes in the computing cluster via the distributed system communication interface.

2. The computing node according to claim 1, characterized in that: The intra-chip communication interface includes any one of a CXL interface and a UCIe interface.

3. The computing node according to claim 1, characterized in that: The distributed system communication interface includes an RDMA network interface.

4. The computing node according to claim 1, characterized in that: The computing accelerator includes any one or more of GPGPU, NPU, and TPU.

5. A computing cluster, characterized in that: The invention comprises a plurality of computing nodes, any computing node comprises a general-purpose central processing unit and at least one dedicated computing accelerator; in any of the computing nodes, the central processing unit and the computing accelerator are communicatively connected via an on-chip communication interface; the central processing unit is also provided with a distributed system communication interface, and the central processing unit and the computing accelerator in any of the computing nodes communicate with other computing nodes via the distributed system communication interface.

6. The computing cluster according to claim 5, characterized in that: The intra-chip communication interface includes any one of a CXL interface and a UCIe interface.

7. The computing cluster according to claim 5, characterized in that: The distributed system communication interface includes an RDMA network interface.

8. The computing cluster according to claim 5, characterized in that: The computing accelerator includes any one or more of GPGPU, NPU, and TPU.

Images