Computer system, node operation method, electronic device, and storage medium

Abstract

A computer system, a node operation method, an electronic device, and a storage medium. The computer system comprises a plurality of computing nodes and a first switching device, each computing node comprises a plurality of artificial intelligence processors, and the plurality of artificial intelligence processors comprise a first artificial intelligence processor group and a second artificial intelligence processor group; the artificial intelligence processors in the first artificial intelligence processor group are directly connected with each other, and the artificial intelligence processors in the second artificial intelligence processor group are directly connected with each other; the first artificial intelligence processor group and the second artificial intelligence processor group are connected through a first optical interconnection link; and the first artificial intelligence processor group and the second artificial intelligence processor group are connected with the first switching equipment through a second optical interconnection link. According to the computer system provided by the invention, high-bandwidth and low-delay communication can be realized in the nodes and among the nodes, so that ensemble communication required by large-scale tensor parallel is supported, and a model with a larger parameter quantity is accommodated.

Description

Technical Field

[0001] The embodiments disclosed herein relate to the field of computer technology, specifically to a computer system, a node operation method, an electronic device, and a storage medium. Background Technology

[0002] With the development of artificial intelligence technologies such as large language models and multimodal models, the scale of model parameters has reached a high level. The training and inference processes of models, especially when the entire model needs to be loaded into video memory for full parameter calculation or efficient inference, place higher demands on the capacity and bandwidth of video memory. Traditional single-machine eight-GPU hardware solutions are increasingly unable to accommodate the complete parameters and activation states of the new generation of large models. Summary of the Invention

[0003] At least one embodiment of this disclosure provides a computer system including multiple computing nodes and a first switching device. Each computing node includes multiple artificial intelligence processors, which include a first artificial intelligence processor group and a second artificial intelligence processor group. The artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are also directly connected to each other. The first and second artificial intelligence processor groups are connected via a first optical interconnect link and configured to communicate via the first optical interconnect link. The first and second artificial intelligence processor groups are connected to the first switching device via a second optical interconnect link and configured to communicate via the second optical interconnect link. Each computing node is configured to communicate via the second optical interconnect link.

[0004] For example, in at least one embodiment of this disclosure, a computer system is provided in which a first artificial intelligence processor group includes a first artificial intelligence processor subgroup and a second artificial intelligence processor subgroup, and the second artificial intelligence processor group includes a third artificial intelligence processor subgroup and a fourth artificial intelligence processor subgroup; wherein the first artificial intelligence processor subgroup, the second artificial intelligence processor subgroup, the third artificial intelligence processor subgroup and the fourth artificial intelligence processor subgroup do not overlap with each other, and each artificial intelligence processor subgroup includes the same number of artificial intelligence processors.

[0005] For example, in at least one embodiment of this disclosure, a computer system is provided in which each artificial intelligence processor in a first artificial intelligence processor subgroup and each artificial intelligence processor in a third artificial intelligence processor subgroup are connected through at least one first optical interconnect link, and a one-to-one connection is formed between each artificial intelligence processor in the first artificial intelligence processor subgroup and each artificial intelligence processor in the third artificial intelligence processor subgroup.

[0006] For example, in at least one embodiment of this disclosure, a computer system is provided in which each artificial intelligence processor in a second artificial intelligence processor subgroup is connected to a first switching device via at least one second optical interconnect link, and each artificial intelligence processor in a fourth artificial intelligence processor subgroup is connected to the first switching device via at least one second optical interconnect link.

[0007] For example, in at least one embodiment of this disclosure, a computer system is provided in which a first switching device includes a central processing unit and a bus switch, a first artificial intelligence processor group and a second artificial intelligence processor group are connected to the bus switch through at least one second optical interconnect link, and the bus switch and the central processing unit communicate with each other.

[0008] For example, at least one embodiment of this disclosure provides a computer system configured to perform a many-to-many set communication algorithm.

[0009] For example, in at least one embodiment of this disclosure, a computer system is provided in which multiple artificial intelligence processors within each computing node are disposed in the same chassis.

[0010] At least one embodiment of this disclosure provides a node operation method for a computer system, wherein the computer system includes multiple computing nodes, each computing node includes multiple artificial intelligence processors, the multiple artificial intelligence processors include a first artificial intelligence processor group and a second artificial intelligence processor group; wherein the artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are directly connected to each other; wherein the node operation method includes: responding to the first artificial intelligence processor group and the second artificial intelligence processor group being connected through a first optical interconnect link, such that the first artificial intelligence processor group and the second artificial intelligence processor group communicate through the first optical interconnect link; responding to the first artificial intelligence processor group and the second artificial intelligence processor group being connected to a first switching device through a second optical interconnect link, such that the first artificial intelligence processor group and the second artificial intelligence processor group communicate through the second optical interconnect link, and the computing nodes communicate with each other through the second optical interconnect link. At least one embodiment of this disclosure provides an electronic device, including: at least one processor; at least one memory, including one or more computer program modules; wherein the one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, the one or more computer program modules being used to implement the node operation method provided in the at least one embodiment above.

[0011] At least one embodiment of this disclosure provides a non-transitory computer-readable storage medium storing computer-readable instructions thereon, wherein the computer-readable instructions, when executed by at least one processor, perform the node operation method provided in the at least one embodiment described above.

[0012] In at least one embodiment of this disclosure, a hierarchical heterogeneous communication connection is achieved by directly connecting the AI ​​processors within the first AI processor group and the second AI processor group within a computing node, connecting the AI ​​processors between the first AI processor group and the second AI processor group within a computing node through a first optical interconnect link, and connecting the AI ​​processors between computing nodes through a second optical interconnect link. This enables high-bandwidth, low-latency communication both within and between computing nodes, thereby supporting the aggregated communication required for large-scale tensor parallelism, such as TP32, to accommodate models with a larger number of parameters. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0014] Figure 1 This is a schematic structural diagram of a general-purpose graphics processor;

[0015] Figure 2 A schematic structural diagram of a computer system model provided for at least one embodiment of this disclosure;

[0016] Figure 3 A schematic structural diagram of a computing node model provided in at least one embodiment of this disclosure;

[0017] Figure 4 A schematic structural diagram of a computer system model provided for at least one embodiment of this disclosure;

[0018] Figure 5 A schematic flowchart illustrating a node operation method provided in at least one embodiment of this disclosure;

[0019] Figure 6 A schematic block diagram of an electronic device provided in one embodiment of this disclosure;

[0020] Figure 7 A schematic block diagram of another electronic device provided for at least one embodiment of this disclosure; and

[0021] Figure 8 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0023] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0024] In distributed computing scenarios, computing nodes are the basic building blocks of the system. Through collaborative communication between nodes, efficient parallel processing of large-scale tasks is achieved. Each computing node may include one or more hosts, and each host connects to one or more devices. The hosts and devices work together to complete computing tasks. Within a computing node, data transmission between hosts and devices can be achieved in various ways. Devices (also known as "slave devices") can directly connect to the host through hardware interfaces (e.g., via a PCI Express, PCIe bus) to achieve low-latency data transmission; or they can work with the host in a software-defined manner (such as virtualization technology) through a network.

[0025] For example, the host computer may include a central processing unit (CPU), and the device may include an artificial intelligence processor, such as a graphics processing unit (GPU), a general-purpose graphics processing unit (GPGPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), or a neural network processing unit (NPU), which can be implemented using application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs). In the following description, a GPU will be used as an example, for instance.

[0026] Figure 1 This is a schematic diagram of a general-purpose graphics processing unit (GPGPU).

[0027] like Figure 1 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 1 The diagram shows streaming processor clusters 1, ..., M, where M is a positive integer greater than 1. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0028] like Figure 1 As shown, taking streaming processor cluster 1 as an example, one streaming processor cluster includes multiple computing units, such as... Figure 1The system is structured as Computation Unit 1, Computation Unit 2, ..., Computation Unit N, where N is a positive integer. Each Computation Unit (CU) performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A Computation Unit includes multiple cores (also called computational kernels), each of which includes an Arithmetic Logic Unit (ALU), a floating-point unit, etc. These cores are used to execute specific computational tasks. Furthermore, the Computation Unit also includes registers (e.g., ...). Figure 1 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0029] like Figure 1 As shown, each streaming processor cluster also provides a buffer for caching data across the N computing units within the streaming processor cluster.

[0030] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 1 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0031] In each computing unit, the thread beam scheduling / distribution module ( Figure 1 (Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., [example cache]). Figure 1 High Bandwidth Memory (HBM) is used for read and write operations.

[0032] It should be noted that in distributed computing scenarios, not only is efficient communication required between hosts and devices, but there is also a need for frequent data exchange between devices. Therefore, efficient collaboration between devices relies on high-performance data transmission mechanisms, and cross-device data interaction can be achieved through various technical paths.

[0033] As the parameter size of large language models (LLMs) and multimodal models gradually increases, the demand for computing device memory capacity and bandwidth has grown significantly. During model training, the total number of parameters, optimizer states, gradients, and forward activation values ​​that need to reside simultaneously has exceeded the upper limit of the memory capacity of a homogeneous server node based on eight graphics cards (e.g., a single-machine graphics processor) and high-speed interconnect technology, constituting a major expansion bottleneck.

[0034] To overcome the limitations of single-point GPU memory, distributed training strategies are now widely adopted. Tensor parallelism, as a model parallelism method, divides a single computational operation (such as matrix multiplication) along the row or column dimension and distributes it across multiple AI processors, such as graphics processing units (GPUs), for parallel execution, thus addressing the problem of a single device being unable to handle large operators. This method requires aggregated communication operations between the parallel GPUs to synchronize intermediate data during each forward and backward propagation. This communication is frequent, involves large amounts of data, and is synchronously blocking, thus placing high demands on the latency and bandwidth of the interconnect links. Therefore, the communication group of tensor parallelism can be confined within a single physical server node to utilize ultra-high bandwidth interconnect technologies (such as NVLink), thereby avoiding the performance loss caused by cross-node communication.

[0035] At the cross-node communication level, current architectures primarily rely on Remote Direct Memory Access (RDMA) networks based on InfiniBand or Converged Ethernet (RoCE) protocols. RDMA technology allows one computer to directly access data from the memory of another computer without going through the other's operating system kernel and CPU, effectively reducing communication latency and CPU overhead. However, despite the superior performance of RDMA networks in cross-node communication, its typical bandwidth is orders of magnitude lower than that of GPU interconnects within a node based on high-speed interconnect protocols such as NVLink. Furthermore, cross-node communication inevitably introduces higher network transmission latency. This significant heterogeneous interconnection characteristic of "high bandwidth and low latency within nodes" versus "lower bandwidth and higher latency between nodes" is a key constraint in architecture design.

[0036] However, the existing architecture and parallel strategy have gradually revealed the following shortcomings when supporting larger-scale models.

[0037] For example, model scalability is limited because this approach strongly binds the maximum parallelism of tensor parallelism to the number of GPUs (typically 8) on a single physical node and the total video memory capacity. When the video memory required for the parameters and states of a single computational layer of the model (such as the feedforward network layer in the Transformer architecture) exceeds the total capacity of the node, that layer cannot be fully loaded by a single tensor parallel group, limiting the maximum scalability of the model's internal structure.

[0038] For example, cross-node scaling is inefficient. When model size forces tensor parallel groups to span multiple physical nodes, their high-frequency aggregated communication traffic is transmitted via RDMA networks. The bandwidth of inter-node networks is significantly lower than that of intra-node high-speed interconnect protocols, and the latency is also higher. This leads to a sharp increase in cross-node tensor parallel communication time, becoming a major bottleneck in the training process and significantly reducing the utilization of AI processors such as GPU computing units and overall scaling efficiency.

[0039] For example, the system configuration is rigid and optimization is complex. This architecture deeply couples hardware topology with parallel strategies, forcing developers to perform complex, manual parallel strategy mapping and tuning for specific cluster topologies to ensure that computationally intensive communication is confined to within the nodes. This increases the complexity of system deployment and optimization, and makes hardware resource configuration inflexible, making it difficult to adapt flexibly to models of different sizes or architectures.

[0040] In summary, the heterogeneous cluster architecture based on high-speed interconnection within nodes and low-speed interconnection between nodes, and the corresponding intra-node tensor parallelism technology, have become one of the factors restricting the efficient training of large-parameter-scale models.

[0041] This disclosure provides at least one embodiment of a computer system including multiple computing nodes and a first switching device. Each computing node includes multiple artificial intelligence processors, which include a first artificial intelligence processor group and a second artificial intelligence processor group. The artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are also directly connected to each other. The first and second artificial intelligence processor groups are connected via a first optical interconnect link and configured to communicate through the first optical interconnect link. The first and second artificial intelligence processor groups are connected to the first switching device via a second optical interconnect link and configured to communicate through the second optical interconnect link. Each computing node is configured to communicate through the second optical interconnect link.

[0042] In at least one embodiment of this disclosure, a hierarchical heterogeneous communication connection is achieved by directly connecting the AI ​​processors within the first AI processor group and the second AI processor group within a computing node, connecting the AI ​​processors between the first AI processor group and the second AI processor group within a computing node through a first optical interconnect link, and connecting the AI ​​processors between computing nodes through a second optical interconnect link. This enables high-bandwidth, low-latency communication both within and between computing nodes, thereby supporting the aggregated communication required for large-scale tensor parallelism, such as TP32, to accommodate models with a larger number of parameters.

[0043] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0044] Figure 2 A schematic structural diagram of a computer system model provided for at least one embodiment of this disclosure.

[0045] For example, such as Figure 2 As shown, the computer system 100 may include multiple computing nodes and a first switching device 140. Each computing node includes multiple artificial intelligence processors, and the multiple artificial intelligence processors include a first artificial intelligence processor group and a second artificial intelligence processor group.

[0046] For example, multiple computing nodes may include computing node 110, computing node 120, and computing node 130. For example, computing node 110 may include a first artificial intelligence processor group 111 and a second artificial intelligence processor group 112. For example, computing node 120 may include a first artificial intelligence processor group 121 and a second artificial intelligence processor group 122. For example, computing node 130 may include a first artificial intelligence processor group 131 and a second artificial intelligence processor group 132.

[0047] For example, AI processors in the first AI processor group can be directly interconnected (not shown in the figure), and AI processors in the second AI processor group can also be directly interconnected (not shown in the figure). For example, AI processors can be directly interconnected via a high-speed direct connection protocol. For example, an AI processor can integrate multiple high-speed differential signal lines, and its control unit can be programmed to achieve direct memory access between AI processors via the high-speed direct connection protocol and physical communication media. For example, when the AI ​​processor is a graphics processing unit (GPU), it can have ports supporting the high-speed direct connection protocol. Each port can include multiple high-speed differential signal lines, and the high-speed differential signal lines of the AI ​​processors can be connected to directly connect the ports between GPUs, enabling direct connection between AI processors in the first and second AI processor groups. For example, the control unit of the AI ​​processor can be programmed to support the aforementioned high-speed direct connection protocol and access method.

[0048] For example, in physical implementation, this high-speed direct connection protocol can be a high-speed serial point-to-point physical link, with each link including multiple differential signal lines. Data is transmitted on this physical link in the form of data blocks, implemented through address mapping in the driver. For example, when the AI ​​processor is a GPU, GPUs linked through this high-speed direct connection protocol can have a shared memory pool. One GPU writes to a shared memory address, and another GPU obtains the address of the shared memory in its local process through the driver, reading the content written by the previous GPU. For example, this high-speed direct connection protocol can directly connect GPUs.

[0049] For example, AI processors within a first or second AI processor group can be further configured to access memory via memory access semantics communication. For instance, a remote graphics processor can first allocate memory space, and then directly map the allocated memory space of the remote graphics processor to a unified virtual address visible to the local graphics processor thread, using memory access semantics communication such as load / store instructions to directly read and write the memory of another graphics processor.

[0050] For example, the first AI processor group and the second AI processor group can be connected via a first optical interconnect link, and the first AI processor group and the second AI processor group are configured to communicate via the first optical interconnect link.

[0051] For example, the first optical interconnect link may include a photoelectric conversion module on the transmitting side, a photoelectric conversion module on the receiving side, and an optical fiber transmission medium connecting the two. On the transmitting side, a high-speed electrical signal generated by an AI processor within a first or second AI processor group can be transmitted to a photoelectric conversion module, such as a silicon photonics engine transmitter chip. The modulated and multiplexed optical signal can be output to the optical fiber transmission medium. For example, the optical fiber transmission medium can be single-mode or multimode fiber. On the receiving side, the optical signal from the optical fiber transmission medium is input to a corresponding photoelectric conversion module, such as a silicon photonics engine receiver chip. Finally, the reorganized high-speed electrical signal is output to the target processor within the first or second AI processor group, completing a full data transmission. For example, this first optical interconnect link can be used to implement the aforementioned high-speed direct connection protocol.

[0052] For example, the first AI processor group and the second AI processor group are connected to the first switching device via a second optical interconnect link. The first AI processor group and the second AI processor group are configured to communicate via the second optical interconnect link, and each computing node is configured to communicate via the second optical interconnect link.

[0053] The following is a specific example of a first AI processor group sending data through a second optical interconnect link, and a second AI processor group receiving data through the same second optical interconnect link. For instance, the data to be sent by the first AI processor group can be processed by its internal or dedicated network interface controller and encapsulated into data packets conforming to the interconnect protocol. These data packets are transmitted as electrical signals to a connected photoelectric conversion module, which modulates the electrical signals onto an optical carrier of a specific wavelength to generate a modulated optical signal.

[0054] Subsequently, the optical signal is transmitted through an optical fiber medium to an input port of the first switching device. For example, the first switching device can perform all-optical processing on the optical signal internally. After entering the first switching device, the internal optical switching matrix (e.g., based on microelectromechanical systems, silicon-based optical switches, or wavelength selective switches) switches the input optical path to the output port of the second artificial processor group connected to receive data, based on the destination address information carried in the data packet header (transmitted in the out-of-band control channel) or the wavelength itself, without photoelectric conversion. Alternatively, the first switching device can perform hybrid photoelectric switching on the optical signal. The optical signal can be converted back into an electrical signal by a photoelectric conversion module at the input port of the first switching device. For example, the switching chip or routing processor of the first switching device can parse the destination address of the data packet in the electrical domain, query the routing table, and perform routing decisions, arbitration, and possible traffic management.

[0055] Subsequently, the processed electrical signal is sent to the photoelectric conversion module connected to the target port, where it is converted back into an optical signal. The optical signal output from the target port of the first switching device is transmitted via another second optical interconnect link to the photoelectric conversion module connected to the second artificial intelligence processor group. This photoelectric conversion module converts the optical signal back into an electrical signal, and after processing such as clock data recovery, it is transmitted to the second artificial intelligence processor group. For example, the optical fiber transmission medium can be single-mode or multimode optical fiber.

[0056] For example, communication between the first optical interconnect link and the second optical interconnect link includes high-frequency, synchronous, large-scale data collection communication generated by large-scale tensor parallel training, such as the All-Reduce operation.

[0057] For example, an artificial intelligence (AI) processor may include, but is not limited to, any one of a graphics processing unit (GPU), a tensor processing unit (TPU), a neural network processing unit (NPU), a deep learning processing unit (DPU), an accelerated processing unit (APU), and a general-purpose graphics processing unit (GPGPU), and this disclosure does not specifically limit the type of processor.

[0058] Figure 3 This is a schematic structural diagram of a computing node model provided in at least one embodiment of the present disclosure. Figure 2 A specific example of a computing node in a computer system model.

[0059] For example, such as Figure 3 As shown, computing node 210 includes a first artificial intelligence processor group 211 and a second artificial intelligence processor group 212. For example, the first artificial intelligence processor group 211 includes a first artificial intelligence processor subgroup 211a and a second artificial intelligence processor subgroup 211b, and the second artificial intelligence processor group 212 includes a third artificial intelligence processor subgroup 212a and a fourth artificial intelligence processor subgroup 212b.

[0060] For example, such as Figure 3As shown, the first AI processor subgroup 211a, the second AI processor subgroup 211b, the third AI processor subgroup 212a, and the fourth AI processor subgroup 212b have no overlap, and each AI processor subgroup includes the same number of AI processors. For example, as... Figure 3 As shown, within each computing node, multiple AI processors can be sequentially numbered 0, 1…N (N is a positive integer). The first AI processor subgroup 211a may include AI processor 0 and AI processor 2, the second AI processor subgroup 211b may include AI processor 1 and AI processor 3, the third AI processor subgroup 212a may include AI processor 4 and AI processor 6, and the fourth AI processor subgroup 212b may include AI processor 5 and AI processor 7.

[0061] For example, such as Figure 3 As shown, each AI processor in the first AI processor subgroup 211a and each AI processor in the third AI processor subgroup 212a are connected via at least one first optical interconnect link. For example, a one-to-one connection is formed between each AI processor in the first AI processor subgroup 211a and each AI processor in the third AI processor subgroup 212a. For example, as... Figure 3 As shown, AI processor 0 and AI processor 4 are connected via a first optical interconnect link, and AI processor 2 and AI processor 6 are connected via a first optical interconnect link. For example, the first AI processor subgroup 211a and the third AI processor subgroup 212a can also be connected via a second optical interconnect link. Figure 3 (Not shown in the image). That is, the connection between the first AI processor subgroup 211a and the third AI processor subgroup 212a can simultaneously include a first optical interconnect link and a second optical interconnect link. For example, AI processors 0 and 6 and AI processors 2 and 4 can be connected via a first optical interconnect link and a second optical interconnect link, respectively.

[0062] For example, such as Figure 3 As shown, each AI processor in the second AI processor subgroup 211b is connected to the first switching device 240 via at least one second optical interconnect link, and each AI processor in the fourth AI processor subgroup 212b is connected to the first switching device 240 via at least one second optical interconnect link. For example, as Figure 3As shown, AI processor 1 and AI processor 3 are respectively connected to the first switching device 240 through corresponding second optical interconnect links, and AI processor 5 and AI processor 7 are respectively connected to the first switching device 240 through corresponding second optical interconnect links.

[0063] It should be noted that the embodiments disclosed herein do not limit the number of artificial intelligence processors in each computing node, each artificial intelligence processor group, or each artificial intelligence processor subgroup.

[0064] For example, such as Figure 3 As shown, the first switching device 410 may include a central processing unit 242 and a bus switch 241. For example, a first artificial intelligence processor group 211 and a second artificial intelligence processor group 212 are connected to the bus switch 241 via at least one second optical interconnect link, and the bus switch 241 and the central processing unit 242 communicate with each other.

[0065] For example, bus switch 241 may have multiple high-speed optical interfaces, which can be directly connected to various artificial intelligence processors and central processing units 242 via a second optical interconnect link. For example, bus switch 241 may handle common data link layer or network layer protocols (such as high-speed direct connection protocols, Ethernet, or InfiniBand protocols). For example, bus switch 241 may perform packet parsing, buffering, queuing, congestion control, and priority management to ensure that high-priority aggregated communication traffic (such as performing All-Reduce operations) can obtain low-latency, high-bandwidth transmission guarantees and avoid performance degradation due to network congestion.

[0066] For example, the central processing unit 242 can be configured to schedule global resources, issue network configuration policies, perform performance monitoring and fault diagnosis, etc. For example, in aggregated communication, the central processing unit 242 can be configured to analyze task mapping and communication modes to dynamically configure the optimal communication path through the bus switch 241, and establish a low-latency, high-bandwidth virtual channel for a specific tensor parallel group, thereby optimizing aggregated communication performance.

[0067] For example, multiple AI processors within each computing node are highly concentrated in physical space, such as within the same chassis. Specifically, the physical form of a single computing node can be a chassis that integrates multiple (e.g., eight) AI processors. These processors are fully interconnected or grouped interconnected through high-bandwidth, low-latency board-level interconnect structures (such as high-speed buses based on silicon interposers or advanced packaging technologies), thereby constructing a tightly coupled collaborative computing unit under highly concentrated physical space conditions. The embodiments of this disclosure do not limit the type or specific implementation of the chassis; for example, it can be a box-type or cabinet-type chassis, and can be at least partially made of metal or plastic materials.

[0068] Figure 4 A schematic structural diagram of a computer system model provided in at least one embodiment of this disclosure, for example, Figure 2 Chinese computer system model and Figure 3 A specific example of a computing node.

[0069] For example, such as Figure 4 As shown, the computer system 300 may include multiple computing nodes and a first switching device 350. Each computing node includes multiple artificial intelligence processors. For example, the multiple computing nodes may include computing node 310, computing node 320, computing node 330, and computing node 340. The first switching device 350 includes a bus switch 351 and a central processing unit 352.

[0070] like Figure 4 As shown, the first AI processor group in each computing node may include AI processors locally numbered 0, 1, 2, and 3, and the second AI processor group may include AI processors locally numbered 4, 5, 6, and 7. For example, AI processors 0, 1, 2, and 3 are directly connected to each other, and AI processors 4, 5, 6, and 7 are directly connected to each other.

[0071] For example, AI processors 0 and 6, and AI processors 2 and 4 in each computing node can be connected via a first optical interconnect link and can communicate with each other. Similarly, AI processors 1, 3, 5, and 7 in each computing node can be connected to a first switching device 350 via a second optical interconnect link and can communicate with each other.

[0072] For example, computer system 300 can be configured to perform many-to-many set communication algorithms. These algorithms may include all-reduce, all-gather, and reduce-scatter algorithms, and can be implemented using, for example, a ring algorithm. Figure 4 As shown, the computer system 300 can implement a Ring All-Reduce algorithm with a parallelism of 32, also known as the TP32 Ring All-Reduce algorithm. The Ring All-Reduce algorithm logically constructs N computing nodes into a unidirectional ring, allowing each node to communicate point-to-point with its predecessor and successor nodes at any given time. This evenly distributes the communication load across all nodes and links, achieving near-linear scalability of communication time with the number of nodes and maximizing the utilization of the cluster's total distributed bandwidth.

[0073] For example, Ring All-reduce algorithms can include two communication phases: a distributed reduction phase and a full collection phase. For instance, suppose there are N participating nodes, each holding a block of data to be reduced, and the reduction operation is summation.

[0074] For example, in the distributed reduction phase, each node first divides its own data block into N equal sub-blocks (N is a positive integer). Then, the system performs N-1 rounds of communication. In the k-th round (k ranges from 0 to N-2), each node can send its currently held sub-block with index (node_id - k) mod N to its successor node in the ring. Simultaneously, it can receive a sub-block from its predecessor node, with index (node_id - k - 1) mod N. The received sub-block is then added (reduced) to the corresponding index sub-block stored locally, overwriting the locally stored sub-block. After N-1 rounds, each node will have the final value of the sub-block corresponding to its local node number in the global reduction result, while other sub-blocks are in a partially accumulated state.

[0075] For example, in the full collection phase, the final sub-blocks generated on each node in the distributed reduction phase can be propagated to all nodes on the ring. The system then performs N-1 rounds of communication. In the k-th round, each node can send its currently held sub-block with index (node_id - k + 1) mod N (this sub-block can be called the final sub-block) to the successor node. Simultaneously, it receives a new final sub-block from the predecessor node. The received final sub-block is stored in its local storage location. After N-1 rounds, each node on the ring has collected and stored N final sub-blocks from all other nodes, thus completely reconstructing the final result of the global reduction. At this point, all nodes have obtained the exact same data, and the full reduction synchronization operation is complete.

[0076] For example, such as Figure 4As shown, each AI processor has four independent physical links, theoretically capable of participating in four independent communication rings simultaneously. Each ring uses a different link from the AI ​​processor as a successor node within the ring, allowing the four rings to transmit data in parallel, increasing the total communication bandwidth by nearly four times. For example, taking AI processor 0 in each computing node (i.e., the AI ​​processor in the first AI processor group) as an example, the four independent physical links could include: a direct link between AI processor 0 and AI processor 1, a direct link between AI processor 0 and AI processor 2, a direct link between AI processor 0 and AI processor 3, and a first optical interconnect link between AI processor 0 and AI processor 6. Similarly, taking AI processor 1 in each computing node (i.e., the AI ​​processor in the second AI processor group) as an example, the four independent physical links could include: a direct link between AI processor 1 and AI processor 0, a direct link between AI processor 1 and AI processor 2, a direct link between AI processor 1 and AI processor 3, and a second optical interconnect link between AI processor 1 and the first switching device 350. Through this design, the four rings can evenly distribute the bandwidth of the direct links, the first optical interconnect links, and the first switching device.

[0077] The data processing method provided in at least one embodiment of this disclosure integrates, for example, 32 GPUs into a single logical computing unit by constructing a supernode architecture, thereby breaking through the interconnection scale limitation of 8 GPUs in a traditional single server. The supernode maintains high bandwidth and low latency interconnection characteristics comparable to a single machine. Based on the above interconnection design, TP32ring allreduce-like algorithms can be implemented and their performance potential fully utilized. Specifically, by configuring multiple (e.g., 4) communication links for each GPU, multiple parallel data transmission rings can be constructed. Under this configuration, the theoretical communication bandwidth of the system can reach the level described by the formula. Theoretical bandwidth formula: Total bandwidth = M × B × N / 2 × (N−1). Where, M = 4, representing the number of parallel rings (consistent with the number of links connected to each GPU); B, representing the physical bandwidth of a single link (all links have the same bandwidth); N, representing the total number of processors (e.g., GPUs) participating in the computation. The above design tightly integrates hardware interconnection capabilities with algorithm characteristics, improving the communication efficiency of large-scale distributed training.

[0078] Figure 5 This is a schematic flowchart illustrating a node operation method provided in at least one embodiment of the present disclosure.

[0079] like Figure 5As shown, the node operation method provided in at least one embodiment of this disclosure includes at least steps S410-S420. For example, this node operation method can be applied to a computer system provided in any of the above embodiments of this disclosure. The computer system includes multiple computing nodes, each computing node includes multiple artificial intelligence processors, and the multiple artificial intelligence processors include a first artificial intelligence processor group and a second artificial intelligence processor group. The artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are directly connected to each other.

[0080] Step S410: In response to the connection between the first artificial intelligence processor group and the second artificial intelligence processor group via the first optical interconnect link, the first artificial intelligence processor group and the second artificial intelligence processor group communicate via the first optical interconnect link.

[0081] Step S420: In response to the first artificial intelligence processor group and the second artificial intelligence processor group being connected to the first switching device through the second optical interconnect link, the first artificial intelligence processor group and the second artificial intelligence processor group communicate through the second optical interconnect link, and the computing nodes communicate with each other through the second optical interconnect link.

[0082] For example, in at least one embodiment of the node operation method, the first artificial intelligence processor group includes a first artificial intelligence processor subgroup and a second artificial intelligence processor subgroup, the second artificial intelligence processor group includes a third artificial intelligence processor subgroup and a fourth artificial intelligence processor subgroup; the first artificial intelligence processor subgroup, the second artificial intelligence processor subgroup, the third artificial intelligence processor subgroup and the fourth artificial intelligence processor subgroup do not overlap with each other, and each artificial intelligence processor subgroup includes the same number of artificial intelligence processors.

[0083] For example, in at least one embodiment of the node operation method, step S410 may include: each AI processor in the first AI processor subgroup and each AI processor in the third AI processor subgroup are connected through at least one first optical interconnect link, and a one-to-one connection is formed between each AI processor in the first AI processor subgroup and each AI processor in the third AI processor subgroup.

[0084] For example, in at least one embodiment of the node operation method, step S420 may include: each AI processor in the second AI processor subgroup and the first switching device are connected via at least one second optical interconnect link, and each AI processor in the fourth AI processor subgroup and the first switching device are connected via at least one second optical interconnect link.

[0085] For example, in at least one embodiment of the node operation method, the first switching device includes a central processing unit and a bus switch, a first artificial intelligence processor group and a second artificial intelligence processor group are connected to the bus switch through at least one second optical interconnect link, and the bus switch and the central processing unit communicate with each other.

[0086] For example, in at least one embodiment of the node operation method, the computer system is configured to perform a many-to-many set communication algorithm.

[0087] For example, in at least one embodiment of the node operation method, multiple artificial intelligence processors within each computing node are housed in the same chassis.

[0088] It should be noted that the functions or beneficial effects of each step in the node operation method provided in any embodiment of this disclosure can be found in the description of the computer system provided in any embodiment of this disclosure, and will not be repeated here.

[0089] Figure 6 This is a schematic block diagram of an electronic device provided for at least one embodiment of the present disclosure; for example, the electronic device can be used to implement the computer system provided for at least one embodiment of the present disclosure.

[0090] For example, such as Figure 6 As shown, the electronic device 500 includes at least one processor 501 and at least one memory 502. For example, the at least one memory 502 includes one or more computer program modules. For example, the one or more computer program modules are stored in the memory 502 and configured to be executed by the at least one processor 501. The one or more computer program modules include instructions for performing the node operation method described above. When executed by the at least one processor 501, they can perform one or more steps of the node operation method provided in at least one embodiment of this disclosure. The memory 502 and the processor 501 can be interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0091] For example, processor 501 can be a central processing unit (CPU), digital signal processor (DSP), graphics processing unit (GPU), general-purpose graphics processing unit (GPGPU), artificial intelligence (AI) accelerator, or other form of processing unit with data processing and / or program execution capabilities, such as a field-programmable gate array (FPGA); for example, the central processing unit (CPU) can be an x86, ARM, or RISC-V architecture. Processor 501 can be a general-purpose processor or a special-purpose processor, capable of controlling other components in electronic device 500 to perform desired functions.

[0092] For example, memory 502 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB memory, flash memory, etc.

[0093] Figure 7 This is a schematic block diagram of another electronic device provided for at least one embodiment of the present disclosure.

[0094] The electronic devices in at least one embodiment of this disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and fixed terminals such as digital TVs and desktop computers. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0095] The electronic device includes at least one processor and a memory. The processor may be referred to as processing device 601 as described below, and the memory may include at least one of ROM 602, RAM 603, and storage device 608 as described below. The memory is used to store programs for performing the methods described in the various method embodiments above; the processor is configured to execute the programs stored in the memory. The processor may include a central processing unit (CPU) or other forms of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0096] like Figure 7 As shown, the electronic device 600 may include a processing unit 601 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in ROM 602 or a program loaded from storage device 608 into RAM 603. RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interfaces are also connected to bus 604.

[0097] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, displays, speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0098] In particular, according to at least one embodiment of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, at least one embodiment of this disclosure includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the node operation method of at least one embodiment of this disclosure.

[0099] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In at least one embodiment of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In at least one embodiment of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0100] The aforementioned computer-readable medium may be included in the aforementioned electronic device 600; or it may exist independently and not assembled into the electronic device 600.

[0101] Figure 8 This is a schematic block diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure.

[0102] For example, such as Figure 8 As shown, a non-transitory computer-readable storage medium 700 stores computer-readable instructions 701, which, when executed by at least one processor, perform one or more steps of the node operation method described above.

[0103] For example, the storage medium may include a memory card for a smartphone, a storage component for a tablet computer, a hard drive for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media. For example, the readable storage medium may also be... Figure 6 The memory 502 in the memory is described in the foregoing content and will not be repeated here.

[0104] Although the present disclosure has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to the embodiments of the present disclosure, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present disclosure are within the scope of protection claimed by the present disclosure.

[0105] The following points should be noted regarding this disclosure:

[0106] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0107] (2) For clarity, the thickness of layers or regions in the drawings used to describe embodiments of the present disclosure is enlarged or reduced, i.e., these drawings are not drawn to actual scale.

[0108] (3) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0109] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. The scope of protection of this disclosure should be determined by the scope of protection of the claims.

Claims

1. A computer system, characterized by The computer system comprises a plurality of computing nodes and a first switching device, wherein, Each of the computing nodes comprises a plurality of artificial intelligence processors, the plurality of artificial intelligence processors comprising a first artificial intelligence processor group and a second artificial intelligence processor group; The artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are directly connected to each other; The first artificial intelligence processor group and the second artificial intelligence processor group are connected through a first optical interconnection link, and the first artificial intelligence processor group and the second artificial intelligence processor group are configured to communicate through the first optical interconnection link; The first artificial intelligence processor group and the second artificial intelligence processor group are connected through a second optical interconnection link and the first switching device, and the first artificial intelligence processor group and the second artificial intelligence processor group are configured to communicate through the second optical interconnection link, Each of the computing nodes is configured to communicate through the second optical interconnection link.

2. The computer system of claim 1, wherein, The first artificial intelligence processor group comprises a first artificial intelligence processor subgroup and a second artificial intelligence processor subgroup, and the second artificial intelligence processor group comprises a third artificial intelligence processor subgroup and a fourth artificial intelligence processor subgroup; The first artificial intelligence processor subgroup, the second artificial intelligence processor subgroup, the third artificial intelligence processor subgroup and the fourth artificial intelligence processor subgroup are mutually exclusive, and each artificial intelligence processor subgroup comprises the same number of artificial intelligence processors.

3. The computer system of claim 2, wherein, Each artificial intelligence processor in the first artificial intelligence processor subgroup and each artificial intelligence processor in the third artificial intelligence processor subgroup are connected through at least one first optical interconnection link, and form a one-to-one corresponding connection between each artificial intelligence processor in the first artificial intelligence processor subgroup and each artificial intelligence processor in the third artificial intelligence processor subgroup.

4. The computer system of claim 2, wherein, Each of the artificial intelligence processors in the second artificial intelligence processor subgroup and the first switching device are connected through at least one second optical interconnection link, and Each of the artificial intelligence processors in the fourth artificial intelligence processor subgroup and the first switching device are connected through at least one second optical interconnection link.

5. The computer system of claim 1, wherein, The first switching device comprises a central processing unit and a bus switch, The first artificial intelligence processor group and the second artificial intelligence processor group are connected to the bus switch through at least one second optical interconnection link, and the bus switch and the central processing unit are in communication.

6. The computer system of any of claims 1-5, wherein, The computer system is configured to perform a many-to-many set communication algorithm.

7. The computer system of any of claims 1-5, wherein, The plurality of artificial intelligence processors in each of the computing nodes are arranged in the same chassis.

8. A method of operating a node of a computer system, characterized by, The computer system comprises a plurality of computing nodes, each of the computing nodes comprises a plurality of artificial intelligence processors, the plurality of artificial intelligence processors comprises a first artificial intelligence processor group and a second artificial intelligence processor group; wherein the artificial intelligence processors in the first artificial intelligence processor group are directly connected to each other, and the artificial intelligence processors in the second artificial intelligence processor group are directly connected to each other; The node operation method comprises: In response to the first artificial intelligence processor group and the second artificial intelligence processor group being connected through a first optical interconnection link, the first artificial intelligence processor group and the second artificial intelligence processor group are enabled to communicate through the first optical interconnection link; In response to the first artificial intelligence processor group and the second artificial intelligence processor group being connected through a second optical interconnection link and the first switching device, the first artificial intelligence processor group and the second artificial intelligence processor group are enabled to communicate through the second optical interconnection link, and the computing nodes are enabled to communicate through the second optical interconnection link.

9. An electronic device, comprising: The electronic device comprises at least one processor; At least one memory comprising one or more computer program modules; The one or more computer program modules are stored in the at least one memory and configured to be executed by the at least one processor, and the one or more computer program modules are used to implement the node operation method in claim 8.

10. A non-transitory computer readable storage medium, comprising: The non-volatile computer readable storage medium is used to non-transiently store computer readable instructions, and when the computer readable instructions are executed by a computer, the node operation method in claim 8 is implemented.

Images