An intelligent computing system architecture

Abstract

The embodiment of the present application discloses an intelligent computing system architecture, wherein the computing board includes multiple first network chips, multiple neural network processors and multiple central processing units, and the multiple neural network processors and multiple central processing units are connected to form a first computing network layer through multiple first network chips; the computing frame includes multiple computing boards and multiple switching boards, and the multiple computing boards are connected to form a second computing network layer; the computing cabinet includes at least two computing frames; the network cabinet is connected to the computing cabinet, and the network cabinet is connected to the storage cabinet to form a third computing network layer, and the network cabinet includes a fourth computing network layer built based on the third computing network layer, and the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer constitute a computing network, and the computing network is used for data transmission during the training of artificial intelligence models. The intelligent computing system architecture can improve the training efficiency of artificial intelligence models.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an intelligent computing system architecture. Background Art

[0002] With the rapid advancement and widespread application of artificial intelligence technology, the demand for computing power is also growing rapidly.

[0003] Artificial intelligence (AI) models often have a huge number of parameters, which have now grown to trillions. Therefore, training AI models requires huge computing power and requires large-scale parallel training on large processor clusters. This is because the training process of AI models involves a large number of computing tasks, including a large number of matrix operations, vector operations, etc. These tasks have extremely high requirements for computing power.

[0004] However, when multiple processors work together to process tasks in parallel, they need to exchange a large amount of intermediate data closely to ensure the accuracy of the calculation results. This requires the computing cluster composed of multiple processors to have higher network performance, such as bandwidth, latency, etc. Therefore, how to improve the network performance of the computing network composed of multiple processors has become a problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide an intelligent computing system architecture that can improve the network performance in the intelligent computing system architecture composed of a computing cabinet, a network cabinet, and a storage cabinet, thereby improving the training efficiency of the artificial intelligence model during the training process.

[0006] In order to achieve the above objectives, the present application embodiment provides an intelligent computing system architecture, including:

[0007] A computing board, the computing board is used to provide the computing power required in the artificial intelligence model process, the computing board includes a plurality of first network chips, a plurality of neural network processors and a plurality of central processing units, and the plurality of neural network processors and the plurality of central processing units are connected to form a first computing network layer through the plurality of first network chips;

[0008] A computing box, wherein the computing box includes a plurality of computing boards and a plurality of switching boards, wherein the plurality of switching boards are used to implement data exchange between the plurality of computing boards, wherein the plurality of computing boards are communicatively connected with the plurality of switching boards to form a second computing network layer, wherein the plurality of switching boards are arranged in parallel to each other in a first direction, wherein the plurality of computing boards are arranged in parallel to each other in a second direction, wherein the first direction and the second direction are perpendicular to each other, and wherein the computing box is used to provide computing power required for use in an artificial intelligence model process;

[0009] A computing cabinet, the computing cabinet comprising at least two computing frames, the computing cabinet being used to provide computing power required in the artificial intelligence model process;

[0010] A storage cabinet and a network cabinet, the storage cabinet is used to provide storage space required in the process of artificial intelligence model training, the network cabinet is used to realize data interaction between the computing cabinet and the storage cabinet, the network cabinet is communicatively connected with the computing cabinet, and the network cabinet and the storage cabinet are communicatively connected to form a third computing network layer, the network cabinet also includes a fourth computing network layer built based on the third computing network layer, the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer constitute a computing network, and the computing network is used for data transmission in the process of artificial intelligence model training.

[0011] In some embodiments, the computing board includes a remote direct data access network card, which is communicatively connected to the first network chip to access the first computing network layer, and the remote direct data access network card is used to realize communication between the current computing board and other computing boards in the intelligent computing system architecture.

[0012] In some embodiments, the computing board includes a high-speed serial transmission chip and a high-speed storage hard disk, the high-speed storage hard disk and the high-speed serial transmission chip are communicatively connected, the multiple neural network processors and the multiple central processing units are communicatively connected to the high-speed serial transmission chip, and the high-speed serial transmission chip is used to read data from and write data to the high-speed storage hard disk.

[0013] In some embodiments, the computing box also includes multiple management boards, which are used to manage and control the connected computing boards and the switching boards. The multiple computing boards are communicatively connected to the multiple management boards, and the multiple computing boards are communicatively connected to the multiple switching boards to form a second computing network layer. The multiple management boards are oriented in the first direction and are arranged parallel to the multiple switching boards.

[0014] In some embodiments, the first computing network layer and the second computing network layer constitute a super node network, the super node network is a part of the computing network, and the super node network is used to realize data transmission between the multiple neural network processors and the multiple central processing units contained in the computing box.

[0015] In some embodiments, the number of the computing boards in the computing box is 16, the number of the neural network processors in the computing boards is 4, the super node network is formed by communication connections between at least 64 of the neural network processors, and the network bandwidth of the super node network is at least 650 GB per second.

[0016] In some embodiments, the network cabinet includes a plurality of switches, each of the switches includes a plurality of first network ports, and the plurality of first network ports are ports corresponding to the third computing network layer;

[0017] The switching board includes a plurality of second network ports, and the plurality of second network ports are connected to the plurality of first network ports, so that the computing cabinet is connected to the third computing network layer and the fourth computing network layer.

[0018] In some implementations, the network cabinet communicates with the computing cabinet and the network cabinet communicates with the storage cabinet using a unified communication protocol, and the unified communication protocol supports remote direct data access.

[0019] In some embodiments, the computing power of each of the computing cabinets is 80P floating-point operations per second, and each of the network cabinets is connected to multiple computing cabinets, and a computing network with a computing power of at least 10E floating-point operations per second is formed through the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer.

[0020] In some embodiments, the storage cabinet includes a hard disk control chip, a second network chip and a baseboard management control chip. The hard disk control chip is connected to the solid-state hard disk and the mechanical hard disk to control the data reading and writing of the solid-state hard disk and the mechanical hard disk. The baseboard management control chip is used to monitor and manage the storage units constituted by the solid-state hard disk and the mechanical hard disk in the storage cabinet. The second network chip realizes communication connection between multiple storage units to form a storage network.

[0021] In some embodiments, the storage cabinet further comprises a network switch, and the network switch is used to connect the storage network with the third computing network layer.

[0022] In an embodiment of the present application, a computing board is used to provide computing power, and the computing board includes multiple first network chips, multiple neural network processors and multiple central processing units. The multiple neural network processors and the multiple central processing units are communicatively connected through multiple first network chips to form a first computing network layer; the computing frame includes multiple computing boards and multiple switching boards, and the multiple switching boards are used to realize data exchange between the multiple computing boards. The multiple computing boards are communicatively connected with the multiple switching boards to form a second computing network layer. The multiple switching boards are oriented in a first direction and are arranged parallel to each other, and the multiple computing boards are oriented in a second direction and are arranged parallel to each other, and the first direction and the second direction are perpendicular to each other; the computing cabinet includes at least two computing frames, and the computing cabinet is used to provide the computing power required in the artificial intelligence model process; the storage cabinet is used to provide the storage space required in the artificial intelligence model training process, and the network cabinet is used to realize data interaction between the computing cabinet and the storage cabinet. The network cabinet is communicatively connected with the computing cabinet, and the network cabinet and the storage cabinet are communicatively connected to form a third computing network layer. The network cabinet also includes a fourth computing network layer constructed based on the third computing network layer. The first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer constitute a computing network, and the computing network is used for data transmission in the artificial intelligence model training process. In this way, an intelligent computing system architecture with high computing power, high network performance, and high storage performance has been built, which can improve the training efficiency of artificial intelligence models.

[0023] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a schematic diagram of the structure of the intelligent computing system architecture provided in the embodiment of the present application;

[0026] Figure 2 is a schematic diagram of the network structure of a computing network provided in an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of the structure of a computing cabinet provided in an embodiment of the present application;

[0028] Figure 4 is a schematic diagram of the structure of a computing board provided in an embodiment of the present application;

[0029] Figure 5 is another structural schematic diagram of a computing board provided in an embodiment of the present application;

[0030] Figure 6 is another structural schematic diagram of a computing board provided in an embodiment of the present application;

[0031] Figure 7 It is a schematic diagram of the structure of the computing box provided in the embodiment of the present application;

[0032] Figure 8 is another structural diagram of a computing frame provided in an embodiment of the present application;

[0033] Fig. 9 is another structural diagram of a computing frame provided in an embodiment of the present application;

[0034] Fig.10 is another structural diagram of a computing frame provided in an embodiment of the present application;

[0035] Fig.11 It is a structural view of the switch board provided in the embodiment of the present application;

[0036] Fig.12 is a schematic diagram of the structure of a storage cabinet provided in an embodiment of the present application;

[0037] Fig.13 This is another structural schematic diagram of the storage cabinet provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0039] It should be noted that in the specific implementation of this application, user-related data such as training data of artificial intelligence models is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0041] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations:

[0042] Floating Point Operations Per Second (FLOPS): It is an indicator to measure the performance of a computer or processor. It is mainly used to evaluate the speed at which it performs mathematical operations, especially in the fields of scientific computing, graphics processing, machine learning, and deep learning.

[0043] Network bandwidth: refers to the amount of data that can be transmitted per unit time. It measures the amount of information that the network can process at a certain moment. This concept is similar to the number of lanes on a highway. The larger the bandwidth, the more lanes on a highway, and the stronger its traffic capacity.

[0044] Graphics Processing Unit (GPU): A microprocessor specifically designed for graphics and video rendering. It processes large amounts of data in parallel, enabling efficient graphics-intensive tasks such as graphics rendering, video decoding, and gaming. The core components of a GPU include an arithmetic unit, a controller, and registers, which are similar to CPUs, but the GPU is designed with a greater focus on parallel computing capabilities to cope with the large-scale data parallel processing requirements in graphics processing.

[0045] Neural Processing Unit (NPU): is a new type of processor designed for deep learning and neural network operations, aiming to perform machine learning, especially deep learning tasks, efficiently and with low power consumption. The core concept of NPU is to simulate the working principle of human brain neural network, and accelerate complex calculations such as large-scale matrix operations and convolution operations in deep neural networks through large-scale parallel processing units and efficient interconnection structures.

[0046] Remote Direct Memory Access (RDMA): is a network technology that allows data to be transferred directly from the memory of one computer to the memory of another computer without the intervention of the CPU or operating system. This technology significantly reduces latency and CPU burden by reducing the number of copies during data transmission, thereby improving the efficiency and performance of network transmission.

[0047] Fat-Tree is a network topology mainly used in the network design of high-performance computer clusters and data centers. It combines the characteristics of star topology and tree topology, and aims to provide high-bandwidth, low-latency network connections while maintaining good scalability and fault tolerance. The Fat-Tree network topology consists of multiple layers of switches 210, and the number of switches 210 in each layer gradually decreases, forming a tree-like structure. Generally, the Fat-Tree topology can be divided into k layers, where k is a positive integer. The bottom layer contains k^2 leaf nodes, the middle layer contains k(k-1) switches 210, and the top layer contains k root switches 210.

[0048] First, the relevant technical content is described.

[0049] In recent years, the most advanced artificial intelligence (AI) models are based on pre-trained large models. These large models use deep learning technology to integrate the distribution and characteristics of data into a neural network with a huge number of parameters based on a large amount of data, and build pre-trained large models with ultra-large-scale parameters and ultra-high precision. By combining industry knowledge, these large models can quickly adapt to different industry scenarios, thereby accelerating the application of artificial intelligence models in various industries.

[0050] Artificial intelligence models often have a huge number of parameters, which have now grown to trillions. Therefore, training artificial intelligence models requires huge computing power. Large-scale parallel training is required on large processor clusters to complete. This is because the training process of artificial intelligence models involves a large number of computing tasks, including a large number of matrix operations, vector operations, etc. These tasks have extremely high requirements for computing power.

[0051] However, when multiple processors work together to process tasks in parallel, they need to exchange a large amount of intermediate data closely to ensure the accuracy of the calculation results. This requires the computing cluster composed of multiple processors to have higher network performance, such as bandwidth, latency, etc. Therefore, how to improve the network performance of the computing network composed of multiple processors has become a problem that needs to be solved urgently.

[0052] For example, in some related technologies, large-scale parallel training of artificial intelligence models is carried out on large-scale graphics processing unit (GPU) clusters. This is because the training process of artificial intelligence models involves a large number of computing tasks, including a large number of matrix operations, vector operations, etc. These tasks have extremely high requirements for computing power. Graphics processors have become the preferred hardware for artificial intelligence model training due to their strong parallel processing capabilities and high computing efficiency.

[0053] In related technologies, supercomputers built based on graphics processors include NVIDIA's GH200 supercomputer and supercomputers built based on HPE's Shasta supercomputer architecture, such as the Aurora supercomputer at the Argonne National Laboratory in the United States.

[0054] The maximum number of graphics processors that can be accommodated in a single computing cabinet 100 of the GH200 supercomputer is 16, and the computing power of a single computing cabinet 100 is 16P floating point operations per second (16PFLOPS). The network bandwidth of the Aurora supercomputer is 160GB per second.

[0055] However, some high-performance graphics processors cannot be popularized in domestic laboratories, research institutes and technology companies for various reasons. Therefore, this application proposes to use common neural network processors and central processing units on the market as the computing power core, and combine other peripheral devices to form an intelligent computing system architecture with high network performance, high computing power performance and high storage performance.

[0056] In the present application, the computing power of a single computing cabinet 100 is at least 80P floating point operations per second (80PFLOPS), and the network bandwidth is at least 650GB per second. The intelligent computing system architecture based on the computing cabinet 100 in the present application has higher computing power performance and network performance than the supercomputer in the related art.

[0057] The intelligent computing system architecture provided in this application will be described in detail below.

[0058] See also Figure 1 , Figure 1 It is a schematic diagram of the composition structure of the intelligent computing system architecture provided in the embodiment of the present application.

[0059] Among them, the intelligent computing system architecture is mainly composed of three parts of hardware communication connection, namely, the computing cabinet 100, the network cabinet 200 and the storage cabinet 300, among which the computing cabinet 100 and the network cabinet 200 are communication connected, the network cabinet 200 and the storage cabinet 300 are communication connected, and the computing cabinet 100 and the storage cabinet 300 are communication connected through the network cabinet 200.

[0060] The computing cabinet 100 includes at least two computing boxes 110, and the computing boxes 110 include multiple computing boards 101, and the computing boards 101 include multiple first communication chips, multiple neural network processors and multiple central processing units. The multiple neural network processors and multiple central processing units in each computing board 101 are connected to each other through multiple first network chips to form a first computing network layer. The communication connection between multiple computing boards 101 constitutes a second computing network layer.

[0061] The storage cabinet 300 and the network cabinet 200, the storage cabinet 300 is used to provide the storage space required in the artificial intelligence model training process, the network cabinet 200 is used to realize the data interaction between the computing cabinet 100 and the storage cabinet 300, the network cabinet 200 is communicated with the computing cabinet 100, and the network cabinet 200 and the storage cabinet 300 are communicated with each other to form a third computing network layer, and the network cabinet 200 also includes a fourth computing network layer built based on the third computing network layer.

[0062] The first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer together constitute a computing network, which is used for data transmission during the training of the artificial intelligence model. The computing network can be specifically formed by a fat tree network topology structure, wherein the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer are specifically different network layers in the fat tree network topology structure.

[0063] To better understand the computing network provided by the present application embodiment, please continue to refer to Figure 2 , Figure 2 It is a schematic diagram of the network structure of the computing network provided in the embodiment of the present application.

[0064] In order to clearly express the network structure of the computing network, Figure 2 In the figure, only one computing box 110 in a computing cabinet 100 is shown in communication connection with a network cabinet 200. Figure 2 The network structure of the computing network shown cannot be used as a limitation on the computing network in the intelligent computing system architecture of this application.

[0065] The computing board 101 includes multiple neural network processors, such as NPU0, NPU1, NPU2, and NPU3, and multiple central processing units, such as CPU0 and CPU1. Multiple neural network processors and multiple central processing units are interconnected to form a first computing network layer, that is, Figure 2 The L1 network layer in the network includes multiple network nodes, such as L1-1, L1-2...L1-N and other network nodes.

[0066] A computing box 110 may include multiple computing boards 101, and multiple neural network processors and multiple central processing units inside each computing board 101 are interconnected to form a first computing network layer.

[0067] In the computing frame 110, based on the first computing network layer of the plurality of computing boards 101, a second computing network layer can be constructed and generated. For example, the plurality of first computing network layers are connected in the computing frame 110 to generate the second computing network layer. Figure 2 The L2 network layer shown in the figure includes multiple network nodes in the second computing network layer. For example, the L2 network layer includes multiple network nodes such as L2-1, L2-2...L2-N.

[0068] Each computing cabinet 100 has multiple computing boxes 110. Multiple computing cabinets 100 can be connected to the network cabinet 200 for communication. For example, a computing box 110 in a computing cabinet 100 is Figure 2 As shown, the computing box 110 and the network cabinet 200 are connected in communication, and the storage cabinet 300 and the network cabinet 200 are also connected in communication, thus forming a third computing network layer, that is, Figure 2 The L3 network layer in the third computing network layer includes multiple network nodes. For example, the L3 network layer includes multiple network nodes such as L3-1, L3-2...L3-N.

[0069] Based on the multiple network nodes of the third computing network layer, the network topology is performed again, for example, based on the fat tree network topology structure, so as to generate the fourth computing network layer, that is, Figure 2 The L4 network layer in the network includes multiple network nodes such as L4-1, L4-2...L4-N.

[0070] In this way, through Figure 2 The L1 network layer, L2 network layer, L3 network layer and L4 network layer constitute a computing network, which can be used for training artificial intelligence models.

[0071] In some embodiments, the number of network nodes corresponding to the first computing network layer, the second computing network layer, the third computing network layer, and the fourth computing network layer respectively decreases in sequence, for example, the number of network nodes in the second computing network layer is less than the number of network nodes in the first computing network layer. The network node of the fourth computing network layer may be the root node of the entire computing network.

[0072] From the above content, we can see that Figure 2 The main composition of the computing network in this application is demonstrated. The computing network has the characteristics of high network bandwidth, low network latency, and high computing performance, which will be described in detail below.

[0073] In the following, the components of the computing cabinet 100, the network cabinet 200 and the storage cabinet 300 will be described respectively, as well as the computing network formed by the communication connection between the components.

[0074] First, the description will be made from the perspective of the computing cabinet 100. Figure 3 , Figure 3 It is a structural diagram of the computing cabinet 100 provided in an embodiment of the present application.

[0075] Each computing cabinet 100 includes at least two computing boxes 110. Figure 3 FIG. 1 shows a computing cabinet 100 including two computing frames 110. Each computing frame 110 includes a computing board 101, a switching board 102 and a management board 103.

[0076] The computing cabinet 100 can be understood as a high computing power cluster composed of multiple computing frames 110, each computing frame 110 can be understood as a high computing power cluster composed of multiple computing boards 101, and each computing board 101 can be understood as a high-performance computing node.

[0077] The computing cabinet 100 mainly provides sufficient computing power for the entire intelligent computing system architecture, and the computing board 101 is the core component of the computing cabinet 100. First, the computing board 101 is described in detail.

[0078] Please also read Figure 4 , Figure 4 It is a schematic diagram of the structure of the computing board 101 provided in an embodiment of the present application.

[0079] Among them, the computing board 101 includes multiple first network chips, multiple neural network processors and multiple central processing units. The multiple neural network processors and multiple central processing units are connected to each other through multiple first network chips to form a first computing network layer.

[0080] The neural network processor can be the Ascend NPU. The Ascend NPU adopts the domestically developed Da Vinci architecture and 3D Cube architecture, integrates a rich set of computing units with high energy efficiency and flexible programmability, and can provide higher computing power for artificial intelligence model training tasks, and achieve efficient accelerated computing of artificial intelligence neural networks under power consumption constraints. The neural network processor is also equipped with large-capacity, high-bandwidth memory, with up to 125GB of high-performance memory (High Bandwidth Memory, HBM) packaged on-chip, and a memory access bandwidth of more than 3TB / s, which can support large-scale data storage and efficient access. The high-performance computing power and high-bandwidth memory of the Ascend NPU are the basis for artificial intelligence model training, and the high energy efficiency ratio and high-speed on-chip network are important supports for 10E-class supercomputer clusters (intelligent computing system architecture).

[0081] The central processing unit can be Kunpeng CPU, which is currently the industry's leading general-purpose processor independently designed based on the ARM instruction set. The processor performance is greatly improved by optimizing the branch prediction algorithm, increasing the number of computing units, improving the memory subsystem architecture, and other micro-architecture designs. At the same time, it supports a dedicated power management architecture, freely switching energy-saving modes, and better dynamic energy saving according to business load. The Kunpeng CPU module is based on the powerful Kunpeng CPU, combined with TB-level large-capacity DDR5 memory, high-speed serial transmission chips (PCIe controllers), NVMe controllers and other devices to form a complete CPU module, which is connected to the first network chip through a cable and works with other modules to achieve a complete high-performance computing node.

[0082] A computing board 101 is a high-performance computing node. A computing board 101 can include two central processing units and four neural network processors. The two central processing units and four neural network processors can form a heterogeneous computing architecture with an overall computing power of more than 700 TFLOPS. It is equipped with large-capacity, high-bandwidth memory. At the same time, the first network chip in the node supports intra-node interconnection between neural network processors, realizing high-performance artificial intelligence model training and meeting the needs of artificial intelligence model training scenarios.

[0083] like Figure 4 As shown, the first network chip is the core device that constitutes the first computing network layer in the node, and is also the core device that connects multiple central processing units and multiple neural network processors in the node to work together. The neural network processor and the central processing unit inside the computing board 101 are formed by the first network chip (HCCS L1 chip) integrated on the computing board 101 to form a fully interconnected first computing network layer, which uses all-electric signals to transmit data, and can reach a bidirectional transmission network bandwidth of up to 650GB / s, which can be used for high-speed data transmission between multiple central processing units and multiple neural network processors. The first network chip can include multiple, such as Figure 4 There are multiple first network chips such as HCCS L1-1, HCCS L1-2, HCCS L1-3, HCCS L1-4, etc.

[0084] The first network chip is responsible for the communication between the neural network processor and the central processor inside the computing board 101, and is also connected to the remote direct data access (RDMA) network card. Figure 4The ROH L1 network card in the HCCS chip is an HCCS chip with RDMA (RDMA Over HCCS). The remote direct data access network card is responsible for the communication between the computing board 101 and other computing boards 101. The remote direct data access network card is a device that allows one computer to directly access the memory of another computer. The remote direct data access network card can significantly reduce the CPU load and improve the efficiency and speed of network communication. It is particularly suitable for high-performance and low-latency data transmission scenarios.

[0085] in, Figure 4 The "L1" marked on the first network chip and the remote direct data access network card in the figure represents the first computing network layer. By setting the first network chip and the remote direct data access network card, the number of multiple central processors and multiple neural network processors in the computing board 101 can be flexibly configured.

[0086] The first network chip and the remote direct data access network card can be regarded as a dedicated network module within the computing board 101, which is used to forward the communication data generated when multiple central processing units and multiple neural network processors within the computing board 101 work.

[0087] Please also read Figure 5 , Figure 5 It is another structural schematic diagram of the computing board 101 provided in an embodiment of the present application.

[0088] The computing board 101 also includes a liquid cooling module and a power supply module. Figure 5 Not shown in Figure 5 The liquid cooling interface on the lower left is the interface of the liquid cooling module, and the power supply interface is the interface of the power supply module. The liquid cooling module can be used to dissipate heat when the computing board 101 is working, such as dissipating the heat generated when the central processing unit and the neural network processor are working, and supports high-temperature water cooling, and can automatically shut down when leaking liquid, so as to ensure that the central processing unit and the neural network processor will not experience frequency reduction, so as to maintain the high-performance working stability of the central processing unit and the neural network processor. The power supply module can provide a stable DC power supply for the entire computing board 101. The computing board 101 adopts liquid cooling as a whole, which can effectively manage heat, achieve high-density deployment, and ensure the scale of deployment of computing clusters.

[0089] Figure 5 The interface shown on the left is the network interface of computing board 101 . Figure 5 The right side is the storage area, which may be a storage area composed of high-speed storage hard disks, such as an NVMe hard disk.

[0090] Please also read Figure 6 , Figure 6It is another structural schematic diagram of the computing board 101 provided in an embodiment of the present application.

[0091] The computing board 101 also includes a high-speed serial transmission chip and a high-speed storage hard disk, the high-speed storage hard disk and the high-speed serial transmission chip are communicatively connected, multiple neural network processors and multiple central processing units are communicatively connected to the high-speed serial transmission chip, and the high-speed serial transmission chip is used to read and write data in the high-speed storage hard disk. The high-speed serial transmission chip can be a PCIe chip, and the high-speed storage hard disk can be an NVMe hard disk.

[0092] The local high-speed storage hard disk in the computing board 101 provides services such as model storage, data storage, data caching, and logging during the training process for the computing board 101. It is an efficient and reliable data storage component that ensures fast access and secure storage of data during the artificial intelligence model training task. Multiple components and chips in the computing board 101 are mounted on a high-speed serial transmission chip. For example, when the central processing unit needs to read data from the high-speed storage hard disk, the central processing unit sends a read request, which is transmitted to the high-speed storage hard disk at high speed through the high-speed serial transmission chip. The high-speed storage hard disk receives the request, reads data from the memory, and transmits the data back to the central processing unit at high speed through the high-speed serial transmission chip.

[0093] Figures 4 to 6 The corresponding description content is the relevant content of the computing board 101, wherein the computing board 101 comprises a first computing network layer through multiple central processing units, multiple neural network processors and multiple first network chips.

[0094] Next, the calculation block 110 provided in this application will be introduced. Figure 7 , Figure 7 It is a schematic diagram of the structure of the computing box 110 provided in an embodiment of the present application.

[0095] The computing frame 110 includes a plurality of computing boards 101 and a plurality of switching boards 102, wherein the plurality of switching boards 102 are used to realize data exchange between the plurality of computing boards 101, and the plurality of computing boards 101 are communicatively connected with the plurality of switching boards 102 to form a second computing network layer, wherein the plurality of switching boards 102 are arranged in parallel to each other in a first direction, and the plurality of computing boards 101 are arranged in parallel to each other in a second direction, and the first direction and the second direction are perpendicular to each other. The first direction may be a vertical direction, and the second direction may be a horizontal direction.

[0096] The computing frame 110 also includes multiple management boards 103, which are used to manage and control the connected computing boards 101 and switch boards 102. Multiple computing boards 101 are communicatively connected with multiple management boards 103, and multiple computing boards 101 are communicatively connected with multiple switch boards 102 to form a second computing network layer. The multiple management boards 103 are oriented in the first direction and are arranged parallel to the multiple switch boards 102.

[0097] In some embodiments, the first computing network layer and the second computing network layer constitute a super node network, which is a part of the computing network and is used to implement data transmission between multiple neural network processors and multiple central processing units included in the computing box 110. The first computing network layer and the second computing network layer constituted in each computing box 110 can be understood as an independent super node network.

[0098] like Figure 7 As shown, multiple switch boards 102 and multiple management boards 103 face the vertical direction, and multiple switch boards 102 and multiple management boards 103 are arranged in parallel with each other. Multiple computing boards 101 face the horizontal direction, and multiple computing boards 101 are arranged in parallel with each other. Through the numerical switch boards 102, multiple computing boards 101 can be orthogonally connected to form a second computing network layer.

[0099] Among them, the role of the switch board 102 is to realize the communication connection between multiple computing boards 101, and the switch board 102 can also be connected to the network cabinet 200. The role of the management board 103 is to manage and control the connected computing boards 101 and the switch boards 102, such as monitoring the status of each switch board 102 and the status of each computing board 101, so as to realize the global perception of the computing frame 110 to ensure the stability of the computing frame 110 when working. When a component in the computing board 101 or the switch board 102 in the computing frame 110 fails, the fault can be discovered in time to remind the technician to solve the fault.

[0100] For a more detailed understanding of the communication connection between the computing board 101 and the switching board 102, please refer to Figure 8 , Figure 8 It is another structural diagram of the computing box 110 provided in an embodiment of the present application.

[0101] Among them, the switch board 102 has a dedicated network interface for application, combined with Figure 5 There is also a corresponding network interface on the computing board 101. The dedicated network interface on the switching board 102 can be connected to the network interface on the computing board 101 to achieve communication connection between the computing board 101 and the switching board 102. The dedicated network interface on the switching board 102 can be blindly connected to the network interface on the computing board 101, avoiding complicated manual wiring.

[0102] By adopting this orthogonal connection method between the computing board 101 and the switching board 102, the entire computing frame 110 does not need to design a backplane, while reducing the number of operation and maintenance equipment and reducing the difficulty of deployment and operation and maintenance.

[0103] Please combine Fig. 9 , Fig. 9 It is another structural diagram of the computing box 110 provided in an embodiment of the present application.

[0104] In some embodiments, the number of computing boards 101 in the computing box 110 is 16, the number of neural network processors in the computing board 101 is 4, the super node network is composed of at least 64 neural network processors communicating with each other, and the network bandwidth of the super node network is at least 650 GB per second.

[0105] The 16 computing boards 101 and the 8 switching boards 102 in the computing frame 110 form the second computing network layer, that is, Fig. 9 The L2 network layer shown in the figure, wherein the communication connection between the computing board 101 and the switching board 102 is mainly realized by the remote direct data access network card in the computing board 101, that is, Fig. 9 The ROH network card in the system can achieve high-speed and low-latency communication between multiple neural network processors and multiple central processing units.

[0106] The first computing network and the second computing network can form a super node network, and the super node network can realize a high-speed connection between one computing board 101 and another computing board 101 through a remote direct data access network card and a first network chip. The network bandwidth for communication between different computing boards 101 and different chips is at least 650 GB per second.

[0107] See also Fig.10 , Fig.10 It is another structural diagram of the computing box 110 provided in an embodiment of the present application.

[0108] In some embodiments, within a computing frame 110 , there are 8 switch boards 102 and 16 computing boards 101 , each computing board 101 has 4 first network chips and 1 remote direct data access network card, and each switch board 102 has 4 network nodes of the second computing network.

[0109] From a macroscopic perspective, the remote direct data access network card in the computing board 101 can realize the communication connection between the computing board 101 and the switching board 102, that is, realize the connection between the first computing network layer in the computing board 101 and the second computing network layer in the switching board 102. For example, the remote direct data access network card of the first computing board 101 can realize the communication connection with each switching board 102 respectively, that is, realize the connection with the second computing network layer corresponding to the switching board 102.

[0110] From a specific perspective, the remote direct data access network card is actually connected to multiple first network chips, and each first network chip can be communicatively connected to a network node in the switch board 102. For example, taking a computing board 101 as an example, the first first network chip HCCS L1-1 can be connected to the L2-1 network node in the first switch board 102, the second first network chip HCCS L1-2 can be connected to the L2-2 network node in the first switch board 102, the third first network chip HCCS L1-3 can be connected to the L2-3 network node in the first switch board 102, and the fourth first network chip HCCS L1-4 can be connected to the L2-4 network node in the first switch board 102.

[0111] The first first network chip HCCS L1-1 can be connected to the L2-1 network node in the second switch board 102, the second first network chip HCCS L1-2 can be connected to the L2-2 network node in the second switch board 102, the third first network chip HCCS L1-3 can be connected to the L2-3 network node in the second switch board 102, and the fourth first network chip HCCS L1-4 can be connected to the L2-4 network node in the second switch board 102.

[0112] In this way, the communication connection between each switching board 102 and each computing board 101 is realized. The first computing network layer can be understood as being constructed inside the computing board 101, and the second computing network layer can be understood as being constructed as the communication connection between the computing board 101 and the switching board 102.

[0113] In the present application, the supernode network is implemented on the first computing network layer and the second computing network layer in the computing box 110, and has no effect on traditional dedicated network communications. The neural network processors in the computing board 101 are connected to the first network chip to form the first computing network layer of the supernode network. The network bandwidth of the first computing network layer can reach 650GB per second, and up to 4 neural network processors and 2 central processing units can be connected together. The switching board 102 in the computing box 110 connects the computing boards 101 to each other to form the second computing network layer of the supernode network. The network bandwidth of the second computing network layer of the supernode network is as high as 650GB / s, and up to 64 neural network processors in the computing box 110 can be connected. The first computing network layer and the second computing network layer together constitute a low-latency, high-performance, large-bandwidth, non-convergent high-performance supernode network.

[0114] The supernode network can realize high-speed interconnection between the neural network processors in the computing frame 110. For example, when NPU0 and NPU1 in the computing board 101 transmit data, the optimal forwarding path can be automatically selected according to the load of the four first network chips (HCCS L1). After the data of NPU0 is forwarded to the HCCS L1 chip, it can directly reach NPU1, avoiding the occurrence of network congestion caused by data forwarding from the remote direct data access network card. The first computing network layer and the second computing network layer of the supernode network described in this application are used to connect 64 NPU chips on 16 computing boards 101 to realize a non-blocking and non-convergent supernode network, which is more friendly to artificial intelligence model training tasks.

[0115] From the above content, we can see that Figures 7 to 10 In the described content, the computing box 110 provided in the embodiment of the present application is described in detail, and the connection between the first computing network layer and the second computing network layer is realized through the communication connection between the switching board 102 and the computing board 101 in the computing box 110, thereby forming a super node network, which is part of the computing network in the entire intelligent computing system architecture.

[0116] In the present application, each computing cabinet 100 may include two such computing frames 110. A single computing cabinet 100 may contain 128 neural network processors, with a total computing power of 80P FLOPS and an ultra-high energy efficiency ratio.

[0117] The computing cabinet 100 uses 380V DC power supply, which is directly input into the computing cabinet 100, and realizes 380V power distribution to various computing boards 101, switching boards 102 and other components in the computing cabinet 100. This solution has the characteristics of less power conversion times and small power supply path loss, and is suitable for the high power and high energy efficiency requirements of the computing cabinet 100.

[0118] Since the computing cabinet 100 has high computing density and high power, the computing cabinet 100 generates a large amount of heat, and the traditional air cooling solution is far from meeting the requirements of the computing cabinet 100. Therefore, liquid cooling is used to dissipate heat for the computing board 101 and the switching board 102, and normal temperature water is supported on the primary side, so that the natural cold source can be used to dissipate heat to the greatest extent.

[0119] The computing cabinet 100 uses a high-performance advanced network topology, that is, a super-node network composed of the first computing network layer and the second computing network layer, which provides each computing board 101 with high network bandwidth and low data latency interconnection. The computing board 101 and the switch board 102 are connected using a backplane-free orthogonal connector, which reduces the first-level connector, reduces the wiring of the PCB backplane, and reduces high-speed signal loss. The computing board 101 in the computing frame 110 adopts a full blind plug design, and the power connector, liquid cooling quick connector, and signal connector are all blind plugged. The computing board 101 does not have any cable or liquid cooling pipeline interface. When replacing the computing node, there is no need to manually plug and unplug the cables and liquid cooling pipelines, which achieves plug-and-play, greatly simplifying the work complexity of the operation and maintenance personnel.

[0120] Different computing boxes 110 are interconnected to form three computing network layers and a fourth computing network layer, which is a network based on a fat tree network structure, which can effectively expand the computing scale. The first computing network layer inside the computing board 101 and the first computing network layer inside the computing box 110 together constitute a super-node network. The super-node network connection inside the computing box 110 is based on an ultra-low latency and ultra-high bandwidth local bus, which supports cross-node communication of neural network processors. The network bandwidth for bidirectional data transmission between neural network processors in a super-node reaches 650GB / s, which can effectively solve the network bandwidth bottleneck for large model training.

[0121] The above is a detailed description of the structure of the computing cabinet 100 and the first computing network layer and the second computing network layer.

[0122] Next, the network cabinet 200 provided in the present application is described. The network cabinet 200 is connected to the computing cabinet 100 in communication, and the network cabinet 200 is connected to the storage cabinet 300 in communication, thereby forming a third computing network layer. Based on the third computing network layer in the network cabinet 200, the network node topology can be constructed to form a fourth computing network layer. The first computing network layer, the second computing network layer, the third computing network layer, and the fourth computing network layer constitute a computing network in the intelligent computing system architecture.

[0123] There may be multiple computing cabinets 100, multiple network cabinets 200, multiple storage cabinets 300, and communication connections can be achieved between the network cabinets 200, thereby building a supercomputer cluster.

[0124] In some embodiments, the computing power of each computing cabinet 100 is 80P floating-point operations per second, and each network cabinet 200 is connected to multiple computing cabinets 100, forming a computing network with a computing power of at least 10E floating-point operations per second through the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer.

[0125] In the computing cabinet 100, multiple switch boards 102 connect multiple computing boards 101 to form the first computing network layer and the second computing network layer. In the computing cluster, the high-performance network cabinet 200 connects multiple storage cabinets 300 and multiple computing cabinets 100 to form the third computing network layer and the fourth computing network layer. The number of computing cabinets 100 can be flexibly expanded as needed to support the networking of neural network processors of more than 10,000 levels to realize parallel computing. The storage cabinet 300 is connected to the third computing network layer to provide large-capacity, high-bandwidth, and highly available data storage services for the entire supercomputer cluster.

[0126] Please also read Fig.11 , Fig.11 It is a structural view of the switch board 102 provided in the embodiment of the present application.

[0127] In some embodiments, the network cabinet 200 includes multiple switches 210, each switch 210 includes multiple first network ports, the multiple first network ports are ports corresponding to the third computing network layer, and each first network port is connected to the third computing network layer and the fourth computing network layer; the switch board 102 includes multiple second network ports, and the multiple second network ports are connected to the multiple first network ports, so that the computing cabinet 100 can access the third computing network layer and the fourth computing network layer.

[0128] like Fig.11 As shown, the switch board 102 includes a plurality of network switch chips, each of which can be connected to a corresponding second network port. Fig.11 The interface shown on the left in the figure may be a second network port.

[0129] Each switch 210 can be connected through multiple first network ports and multiple second network ports. For example, each switch 210 can realize the connection of 192 second network ports. Each second network port can be a 200G port. For example, the uplink port of the second network port is a 200G port, which has high bandwidth and throughput and supports multiple network protocols. Each second network port can be connected to the third computing network layer and the fourth computing network layer through the connection with the first network port. This can provide flexible connection options and is suitable for large-scale artificial intelligence model training.

[0130] The switch 210 adopts a modular design, allowing expansion according to demand, for example, the number of switches 210 can be set according to actual needs. The switch 210 is designed to provide excellent performance and scalability, enabling it to handle the demanding requirements of artificial intelligence model workloads.

[0131] Each switch 210 includes multiple first network ports, and the multiple first network ports are connected to the third computing network layer and the fourth computing network layer. The first network port is a 400G port. Through the dual-carrier 400G technology, each 400G port in the switch 210 can be connected to the two 200G ports in the switch board 102, that is, one first network port can be connected to two second network ports, specifically, a downlink port of a first network port can be connected to the uplink ports of two second network ports, so that the second network port can access the third computing network layer and the fourth computing network layer. Since the second network port is already connected to the first computing network layer and the second computing network layer, the communication connection of the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer is realized through the connection of multiple first network ports and multiple second network ports.

[0132] Switch 210 uses full liquid cooling and 380V DC power supply, which can provide better thermal management and higher port density, providing a connection bridge for the entire computing network.

[0133] The network cabinet 200 is composed of multiple switches 210, and a single switch 210 supports a maximum of 192 second network ports, that is, 200G ports. The network cabinet 200 communicates with the computing cabinet 100 and the storage cabinet 300 using a unified communication protocol, which supports remote direct data access. The unified communication protocol is a high-performance communication protocol that enables high-speed access to devices and memory within the computing board 101 and between different computing boards 101.

[0134] The entire computing network supports mainstream communication libraries and adopts a 4-layer fat tree network architecture, namely the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer, to achieve efficient traffic routing.

[0135] The network cabinet 200 is a bridge connecting computing power and storage. At the same time, the third computing network layer and the fourth computing network layer formed in the network cabinet 200 can flexibly expand the scale of the computer cluster, so that the computer cluster supports large-scale networking of more than 10,000 cards. According to the 10080PFLOPS computing power calculation of a single computing cabinet, a single high-performance network cabinet 200 can connect hundreds of computing cabinets 100, with a computing power of over 10E, to the third computing network layer and the fourth computing network layer to form a large-scale artificial intelligence computing power cluster. The high-speed network of the first computing network layer and the second computing network layer in the computing board 101 together with the third computing network layer and the fourth computing network layer constitute a high-performance computing network in this application, which can effectively ensure high-speed data transmission during the training process of the artificial intelligence model.

[0136] The computing network provided in this application can be divided into two planes. One is a dedicated network plane, which is mainly used for large-scale computing power networking and storage system access. It can realize the networking of tens of thousands of neural network processors. This dedicated network plane can be understood as a computing network.

[0137] The other is the super-node network plane, which is mainly used for high-performance computing. It can integrate up to 64 neural network processors into a dedicated super-node network, namely the first computing network layer and the second computing network layer, to provide more efficient acceleration for AI training.

[0138] The computing network is based on a high-performance, scalable 4-layer fat tree network structure, in which the first computing network layer and the second computing network layer are respectively located inside the computing board 101 and the computing box 110, and the third computing network layer and the fourth computing network layer are located outside the computing box 110, and are composed of a network cabinet 200 connecting multiple computing cabinets 100 and multiple storage cabinets 300. The third computing network layer and the fourth computing network layer can make the intelligent computing system architecture in this application constitute. The intelligent computing system architecture provided in this application is more scalable and flexible than related technologies, and more computing resources and storage resources are linked to the same high-speed network. The computing cluster implemented using this computing architecture can flexibly expand computing power and storage capacity. At the same time, the storage cabinet 300 connected to the third computing network layer can cooperate with the computing cabinet 100 to better complete the training task of the artificial intelligence model.

[0139] The network switch of the storage cabinet 300 is connected to the third computing network layer of the intelligent computing system architecture, achieving parallel read and write capabilities for tens of thousands of clients, which can meet the needs of large-scale parallel computing. This design ensures efficient data access and excellent read and write performance of up to 1 billion read and write operations per second (Input / Output Operations Per Second, IOPS). The reliability of the high-performance storage system exceeds 99.999%, ensuring data security and stability.

[0140] The above is a detailed description of the network cabinet 200 .

[0141] Finally, the storage cabinet 300 provided in the embodiment of the present application will be described. Fig.12 , Fig.12 It is a schematic diagram of the structure of the storage cabinet 300 provided in an embodiment of the present application.

[0142] Among them, the storage cabinet 300 can organize storage hardware resources into a unified storage resource pool, and provide unstructured storage services for the intelligent computing system architecture on demand.

[0143] The storage cabinet 300 is composed of a high-performance storage cabinet 300 and a large-capacity storage cabinet 300. The high-performance data storage cabinet 300 provides IO (Input / Output) access acceleration for the intelligent computing system architecture and provides a unified namespace for computing tasks. The large-capacity data storage cabinet 300 provides high-density and large-capacity storage to meet the system's demands for large-capacity, high-expansion and low-cost goals, and supports data migration between the high-performance layer and the capacity layer. Multiple high-transmission performance SSD solid-state hard drives can be used in the high-performance storage cabinet 300, while the large-capacity data storage cabinet 300 can use a large-capacity HDD mechanical hard drive.

[0144] like Fig.12 As shown, in order to simultaneously meet the requirements of high-bandwidth, high-IOPS, and high-OPS applications, the storage client provided by the data storage service is compatible with standard POSIX and MPI-IO interfaces. A single storage client connects to multiple storage nodes, improves the bandwidth of a single stream and a single storage client, aggregates random small IOs into large IOs, and directly writes to persistent nodes through the storage client, improving data write performance.

[0145] In terms of hardware, storage is achieved by using different storage media, especially mechanical hard disks for capacity and solid-state disks for IOPS throughput, to build multiple hard disk pools with different performance and capacity. Through an effective back-end storage scheduling algorithm, hot data is placed in a high-performance storage cabinet 300 to provide sufficient service capabilities for high-throughput IO.

[0146] See also Fig.13 , Fig.13 It is another structural schematic diagram of the storage cabinet 300 provided in an embodiment of the present application.

[0147] In some embodiments, the storage cabinet 300 includes a hard disk control chip, a second network chip, and a baseboard management control chip. The hard disk control chip is connected to the solid-state hard disk and the mechanical hard disk to control the data reading and writing of the solid-state hard disk and the mechanical hard disk. The baseboard management control chip is used to monitor and manage the storage units composed of the solid-state hard disk and the mechanical hard disk in the storage cabinet 300. The second network chip realizes the communication connection of multiple storage units to form a storage network. The storage cabinet 300 also includes a storage central processor, which can be used to process some data transmission tasks during the operation of the storage cabinet 300. The storage cabinet 300 also includes a network switch, which is used to connect the storage network to the third computing network layer.

[0148] The Storage Cabinet 300 combines proprietary computing power storage and high-performance read and write capabilities, which can effectively eliminate the IO performance bottleneck in the AI ​​model training process. By streamlining the IO storage stack and configuration, it provides upper-level applications with 10TB / s aggregate bandwidth and up to 1 billion IOPS of data access capabilities. It supports access to high-performance layer storage devices through RDMA and supports network pass-through, reducing dependence on the CPU.

[0149] The storage cabinet 300 uses SSD controller chips and full SSD cache solutions, and can provide SSD hard disks with SAS and PCIe interfaces, with high performance and low power consumption. At the same time, it supports enhanced error checking and correction technology (Error Checking and Correcting, ECC), disk array (Redundant Arrays of Independent Disks, RAID) and other technologies to effectively extend the life of SSD, and supports hardware acceleration technology, such as SSD software layer acceleration technology (Flash Translation Layer, FTL), to achieve stable, low-latency high-performance storage.

[0150] It provides massive storage space for computing services, supports elastic expansion, adopts high-density hardware design, and uses large-capacity HDD hard drives to save computer room space. It can be easily expanded to thousands of nodes and EB-level capacity to meet users' business scale growth needs.

[0151] The storage cabinet 300 provides an automatic load balancing strategy. Through technologies such as intelligent stripe aggregation and IO priority intelligent scheduling, combined with a series of key technologies such as multi-level cache and big data pass-through, it eliminates metadata and small file access bottlenecks, ensures system performance in scale expansion scenarios, and provides users with high bandwidth and low latency ultimate performance.

[0152] The storage cabinet 300 uses a baseboard management controller (BMC). When the environment or components are abnormal, the storage cabinet 300 automatically issues an abnormal alarm and automatically protects the storage device to avoid damage to the storage system. At the same time, in order to improve the high availability of the storage cabinet 300, the link aggregation-based technology is used to achieve transmission link redundancy, and link switching operations are performed when the link fails or is sub-healthy to ensure business continuity.

[0153] When some storage devices go offline abnormally, the management end of the storage system can quickly detect the fault and switch the services carried by it to other normal nodes to ensure that the fault does not affect the availability of storage within the redundancy range of the system design. At the same time, after the faulty node returns to normal and comes back online, the services on other nodes can be automatically balanced / switched back to the node to restore the availability of the system and provide higher performance capabilities and more storage space capabilities.

[0154] Storage Cabinet 300 supports Erasure Coding mode to ensure data reliability and strike a balance between storage space utilization and data reliability. It provides data protection across nodes / cabinets and can continue to provide services even when multiple hard disks, nodes, and cabinets fail.

[0155] From the above, it can be seen that the storage cabinet 300 in the present application can solve problems such as data leakage, data corruption, and network attacks. The storage cabinet 300 has multi-faceted security designs such as hardware equipment security, network security, business security, and management security.

[0156] The above is a detailed description of the composition of the entire intelligent computing system architecture in the embodiment of the present application. Compared with the computer cluster described in the related art, the intelligent computing system architecture provided by the embodiment of the present application, in which the first computing network layer and the second computing network layer can constitute a super node network to achieve sufficient computing power for training artificial intelligence models. On the other hand, the computing network composed of the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer in the present application has a computing power of 10E floating point operations per second, which can achieve a relatively powerful data parallel processing capability. When training artificial intelligence models, more parallel tasks can be performed, thereby improving the training efficiency of artificial intelligence models. In addition, the first network chip is used in the computing network, and the first network chip can be used to realize bidirectional data transmission between chips at 650GB per second, such as bidirectional data transmission between different neural network processors. In addition, the computing network also has a strong storage capacity, which can store the training data required for artificial intelligence model training and the iterative data, cache data, etc. generated during the training process, which can improve the training efficiency of artificial intelligence model training. Through the various components of the intelligent computing system architecture described in the above content, an intelligent computing system architecture with a computing power of at least 10E floating point operations per second can be formed.

[0157] In the present application, the computing board 101 is used to provide the computing power required in the artificial intelligence model process. The computing board 101 includes multiple first network chips, multiple neural network processors and multiple central processing units. The multiple neural network processors and the multiple central processing units are connected to each other through multiple first network chips to form a first computing network layer; the computing frame 110 includes multiple computing boards 101 and multiple switching boards 102. The multiple switching boards 102 are used to realize data exchange between the multiple computing boards 101. The multiple computing boards 101 are connected to the multiple switching boards 102 to form a second computing network layer. The multiple switching boards 102 are arranged in parallel to each other in the first direction, and the multiple computing boards 101 are arranged in parallel to each other in the second direction. The first direction and the second direction Mutually perpendicular; the computing cabinet 100 includes at least two computing frames 110, and the computing cabinet 100 is used to provide the computing power required in the process of artificial intelligence model; the storage cabinet 300 is used to provide the storage space required in the process of artificial intelligence model training, and the network cabinet 200 is used to realize the data interaction between the computing cabinet 100 and the storage cabinet 300. The network cabinet 200 is connected to the computing cabinet 100 in communication, and the network cabinet 200 and the storage cabinet 300 are connected in communication to form a third computing network layer. The network cabinet 200 also includes a fourth computing network layer built based on the third computing network layer. The first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer constitute a computing network, and the computing network is used for data transmission in the process of artificial intelligence model training. In this way, an intelligent computing system architecture with high computing power performance, high network performance and high storage performance is constructed, which can improve the training efficiency of artificial intelligence models.

[0158] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0159] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0160] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0161] The above is a detailed introduction to an intelligent computing system architecture provided by an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0162] The above description of the embodiments of the present application with reference to the accompanying drawings does not limit the scope of the rights of the embodiments of the present application. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the rights of the embodiments of the present application.

Claims

1. An intelligent computing system architecture, characterized in that: include: A computing board, the computing board is used to provide the computing power required in the artificial intelligence model process, the computing board includes a plurality of first network chips, a plurality of neural network processors and a plurality of central processing units, and the plurality of neural network processors and the plurality of central processing units are connected to form a first computing network layer through the plurality of first network chips; A computing box, wherein the computing box includes a plurality of computing boards and a plurality of switching boards, wherein the plurality of switching boards are used to implement data exchange between the plurality of computing boards, wherein the plurality of computing boards are communicatively connected with the plurality of switching boards to form a second computing network layer, wherein the plurality of switching boards are arranged in parallel to each other in a first direction, wherein the plurality of computing boards are arranged in parallel to each other in a second direction, wherein the first direction and the second direction are perpendicular to each other, and wherein the computing box is used to provide computing power required for use in an artificial intelligence model process; A computing cabinet, the computing cabinet comprising at least two computing frames, the computing cabinet being used to provide computing power required in the artificial intelligence model process; A storage cabinet and a network cabinet, wherein the storage cabinet is used to provide storage space required for use in the artificial intelligence model training process, and the network cabinet is used to realize data interaction between the computing cabinet and the storage cabinet. The network cabinet is communicatively connected to the computing cabinet, and the network cabinet and the storage cabinet are communicatively connected to form a third computing network layer. The network cabinet also includes a fourth computing network layer constructed based on the third computing network layer. The first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer constitute a computing network, and the computing network is used for data transmission during the artificial intelligence model training process; The computing frame further includes a plurality of management boards, the management boards are used to manage and control the connected computing boards and the switching boards, the plurality of computing boards are communicatively connected with the plurality of management boards, and the plurality of computing boards are communicatively connected with the plurality of switching boards to form a second computing network layer, and the plurality of management boards are arranged in a first direction and parallel to the plurality of switching boards; The first computing network layer and the second computing network layer constitute a super node network, and the super node network is a part of the computing network. The super node network is used to realize data transmission between the multiple neural network processors and the multiple central processing units contained in the computing box.

2. The intelligent computing system architecture according to claim 1, characterized in that: The computing board includes a remote direct data access network card, which is communicatively connected to the first network chip to access the first computing network layer. The remote direct data access network card is used to realize communication between the current computing board and other computing boards in the intelligent computing system architecture.

3. The intelligent computing system architecture according to claim 1, characterized in that: The computing board includes a high-speed serial transmission chip and a high-speed storage hard disk. The high-speed storage hard disk is communicatively connected to the high-speed serial transmission chip. The multiple neural network processors and the multiple central processing units are communicatively connected to the high-speed serial transmission chip. The high-speed serial transmission chip is used to read data from and write data to the high-speed storage hard disk.

4. The intelligent computing system architecture according to claim 1, characterized in that: The number of computing boards in the computing box is 16, the number of neural network processors in the computing boards is 4, the super node network is formed by communication connections between at least 64 neural network processors, and the network bandwidth of the super node network is at least 650 GB per second.

5. The intelligent computing system architecture according to claim 1, characterized in that: The network cabinet includes a plurality of switches, each of the switches includes a plurality of first network ports, and the plurality of first network ports are ports corresponding to the third computing network layer; The switching board includes a plurality of second network ports, and the plurality of second network ports are connected to the plurality of first network ports, so that the computing cabinet is connected to the third computing network layer and the fourth computing network layer.

6. The intelligent computing system architecture according to claim 1, characterized in that: The network cabinet communicates with the computing cabinet and the network cabinet communicates with the storage cabinet using a unified communication protocol, and the unified communication protocol supports remote direct data access.

7. The intelligent computing system architecture according to claim 1, characterized in that: The computing power of each computing cabinet is 80P floating-point operations per second, and each network cabinet is connected to multiple computing cabinets, and a computing network with a computing power of at least 10E floating-point operations per second is formed through the first computing network layer, the second computing network layer, the third computing network layer and the fourth computing network layer.

8. The intelligent computing system architecture according to claim 1, characterized in that: The storage cabinet includes a hard disk control chip, a second network chip and a baseboard management control chip. The hard disk control chip is connected to the solid-state hard disk and the mechanical hard disk, and is used to control the data reading and writing of the solid-state hard disk and the mechanical hard disk. The baseboard management control chip is used to monitor and manage the storage unit composed of the solid-state hard disk and the mechanical hard disk in the storage cabinet. The second network chip realizes the communication connection of multiple storage units to form a storage network.

9. The intelligent computing system architecture according to claim 8, characterized in that: The storage cabinet also includes a network switch, and the network switch is used to connect the storage network with the third computing network layer.

Images