Parallel processor clusters, topology switching methods, electronic devices and storage media

Abstract

A parallel processor cluster, topology switching method, electronic device, and storage medium are disclosed, applicable to the field of multiprocessor systems. The parallel processor cluster includes M parallel processors and N optical circuit switches. Each parallel processor has N ports that are one-to-one coupled to the N optical circuit switches. The total number of ports on each parallel processor is greater than M, and N is less than M. The parallel processor cluster has a first topology and a second topology, and supports switching between the two topologies. In the first topology, the M parallel processors are divided into L groups, each group containing M / L parallel processors, and the M / L parallel processors are fully interconnected. In the second topology, the M parallel processors are fully interconnected. M, N, and L are positive integers. This parallel processor cluster can adapt to different network models and achieve good performance in various network models, fully utilizing hardware resources.

Description

Technical Field

[0001] The embodiments of this disclosure are applied to the field of multiprocessor systems, specifically relating to a parallel processor cluster, a topology switching method for a parallel processor cluster, an electronic device, and a non-transitory computer-readable storage medium. Background Technology

[0002] With the rapid development of cloud computing, artificial intelligence, and big data applications, modern data centers are constantly expanding in scale, and the requirements for network connectivity are becoming increasingly stringent. For different network models, such as the Transformer model or the MOE model, the bandwidth, latency, and algorithm design in the processor cluster are strongly correlated with the topology. When the cluster is applied to different network models, fixed hardware connections mean that the topology is only applicable to the corresponding model, resulting in poor performance in other models. Furthermore, using traditional electrical switching methods, the buffering and routing modules in the electrical switching method introduce latency, affecting communication bandwidth and thus impacting the performance of large-scale models. Summary of the Invention

[0003] This invention application provides at least one embodiment of a parallel processor cluster, the parallel processor cluster including M parallel processors and N optical circuit switches, each parallel processor having N ports that are one-to-one coupled to the N optical circuit switches, the total number of ports of each parallel processor being greater than M and N being less than M, the parallel processor cluster supporting switching between a first topology and a second topology, the parallel processor cluster being configured to switch between the first topology and the second topology through the N optical circuit switches according to the computing task executed by the parallel processor cluster, wherein, in the first topology, the M parallel processors are divided into L groups, each group including M / L parallel processors, the M / L parallel processors are fully interconnected, and the M / L parallel processors are communicatively connected to the M / L parallel processors included in each other group, the M / L parallel processors are respectively connected to different M / L parallel processors in each other group, in the second topology, the M parallel processors are fully interconnected, and M, N, and L are positive integers.

[0004] For example, in the parallel processor cluster provided in at least one embodiment of this application, the ports of the parallel processors connected to the N ports of each parallel processor are different in the first topology and the second topology. The N optical circuit switches are configured to establish communication connection relationships between the N ports of each of the M parallel processors according to the topology configuration file, so as to form the first topology or the second topology.

[0005] For example, in the parallel processor cluster provided in at least one embodiment of this application, the connection relationship of the other ports in each parallel processor other than the N ports is the same in the first topology and the second topology, and the other ports of each parallel processor establish a communication connection relationship using communication cables.

[0006] For example, in the parallel processor cluster provided in at least one embodiment of this application, each optical circuit switch is coupled to a port with the same port number on the M parallel processors via a communication cable.

[0007] For example, in the parallel processor cluster provided in at least one embodiment of this invention, each parallel processor includes A chip cores, where A is a positive integer greater than 1. In the first topology, the M / L parallel processors are fully interconnected, including the i-th chip cores of the M / L parallel processors being fully interconnected, where i is a positive integer ranging from 1 to A. The M / L parallel processors are also communicatively connected to the M / L parallel processors in each of the other groups, and the i-th chip core of the M / L parallel processors is communicatively connected to the i-th chip core of the M / L parallel processors in each of the other groups. In the second topology, the M parallel processors are fully interconnected, including one chip core of each parallel processor being communicatively connected to the corresponding chip core in the other M-1 parallel processors.

[0008] For example, in the parallel processor cluster provided in at least one embodiment of this invention, at least one of the A chip cores of each parallel processor in each group has a communication connection established through a communication cable with the corresponding chip cores of the A chip cores of all other parallel processors in the group. Each of the M / L parallel processors also has a communication connection through the communication cable with one of the M / L parallel processors included in each other group. The communication connection includes at least one chip core of the A chip cores of the parallel processor and a corresponding chip core of the A chip cores of the parallel processor.

[0009] For example, in the parallel processor cluster provided in at least one embodiment of this invention application, in the second topology, each parallel processor establishes a communication connection with at least one parallel processor through the N optical circuit switches, wherein the at least one parallel processor is a parallel processor in each of the other groups that does not have a communication connection with the A chip cores of the parallel processor. In the first topology, the chip core in each parallel processor establishes a communication connection with a target chip core through the N optical circuit switches, wherein the target chip core is a chip core that, according to the first topology, has not yet established a communication connection with the A chip cores of the parallel processor.

[0010] For example, in the parallel processor cluster provided in at least one embodiment of this application, in response to A=2, L=2, each parallel processor includes two chip cores, the two chip cores being a first chip core and a second chip core. The first chip core of each parallel processor is communicatively connected to the first chip cores of T1 parallel processors in the group via a communication cable. The second chip core of each parallel processor is communicatively connected to the second chip cores of T2 parallel processors in the group via a communication cable. The first chip core of each parallel processor is also communicatively connected to the first chip core of a corresponding parallel processor in another group via a communication cable. Here, T1 and T2 are positive integers, T1+T2=M / L-1, and the T1 parallel processors are different from the T2 parallel processors.

[0011] For example, in the parallel processor cluster provided in at least one embodiment of this invention application, in the first topology, the first chip core of each parallel processor establishes a communication connection with the first chip cores of the M / 2-T1-1 parallel processors in the group (excluding the T1 parallel processors) through the N optical circuit switches. The second chip core of each parallel processor establishes a communication connection with the second chip cores of other parallel processors in the group (excluding the T2 parallel processors) through the N optical circuit switches. The second chip core of each parallel processor also establishes a communication connection with the second chip core of the corresponding parallel processor in the other group through the N optical circuit switches.

[0012] For example, in the parallel processor cluster provided in at least one embodiment of the present invention, in the second topology, each parallel processor establishes a communication connection with the first chip core or the second chip core of at least one parallel processor in the other group through the N optical circuit switches, wherein the at least one parallel processor is a parallel processor in the other group that does not have a communication connection with the first chip core or the second chip core of the parallel processor.

[0013] For example, in a parallel processor cluster provided in at least one embodiment of this application, the parallel processor cluster executes a computational task according to the computational task performed by the parallel processor cluster. Switching between the first topology and the second topology is performed through the N optical circuit switches, including the following operations: In response to the computational task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein during inference or training of the first model, data is split dimensionally and distributed to different parallel processors for distributed computation, and the computation results are aggregated; In response to the computational task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein during inference or training of the second model, the M parallel processors exchange data bidirectionally in pairs.

[0014] For example, in the parallel processor cluster provided in at least one embodiment of this application, the communication operator of the first model includes operators that communicate through tensor parallelism or sequence parallelism, and the communication operator includes a ring reduction operator, a full collection operator, and a scattering operator.

[0015] For example, in the parallel processor cluster provided in at least one embodiment of this application, the communication operator of the second model includes an operator that sends data to all other parallel processors and receives data from all other parallel processors, and the communication operator includes an all-to-all operator.

[0016] For example, in the parallel processor cluster provided in at least one embodiment of this application, the hardware communication bandwidth between the N ports on each parallel processor and the N optical circuit switches is the same as the bandwidth of the communication cable, the N ports on each parallel processor are coupled to the N optical circuit switches through the communication cable, and the data transmission bandwidth of the communication connection between the M parallel processors is the same.

[0017] This invention application provides at least one embodiment of a topology switching method for a parallel processor cluster. The parallel processor cluster includes M parallel processors and N optical circuit switches. Each parallel processor has N ports that are connected one-to-one with the N optical circuit switches. The total number of ports for each parallel processor is greater than M, and N is less than M. The topology switching method includes: switching the topology of the parallel processor cluster through the N optical circuit switches according to the computing task executed by the parallel processor cluster. The topology of the parallel processor cluster includes a first topology and a second topology. In the first topology, the M parallel processors are divided into L groups, each group including M / L parallel processors. The M / L parallel processors are fully interconnected, and each M / L parallel processor is communicatively connected to the M / L parallel processors in each other group. The M / L parallel processors are also connected to different M / L parallel processors within each other group. In the second topology, the M parallel processors are fully interconnected, where M, N, and L are positive integers.

[0018] For example, in at least one embodiment of the topology switching method provided in this invention application, the topology of the parallel processor cluster is switched through the N optical circuit switches according to the computing task executed by the parallel processor cluster. This includes: in response to the computing task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein during inference or training of the first model, data is split dimensionally and distributed to different parallel processors for distributed computing, and the computing results are aggregated; in response to the computing task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein during inference or training of the second model, the M parallel processors exchange data bidirectionally in pairs.

[0019] This application provides at least one embodiment of an electronic device, including: a memory that non-transitoryly stores computer-executable instructions; and a processor configured to run the computer-executable instructions, wherein the computer-executable instructions are executed by the processor to implement the topology switching method according to any embodiment of this application.

[0020] This application provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the topology switching method according to any embodiment of this application.

[0021] The parallel processor cluster provided in this invention application can switch topologies at the driver layer without changing the hardware wiring, enabling the parallel processor cluster to adapt to different computing tasks and achieve good performance in the network models corresponding to different computing tasks, thus making full use of hardware resources. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings of the embodiments will be briefly described below. Obviously, the drawings described below only relate to some embodiments of this disclosure and are not intended to limit this disclosure.

[0023] Figure 1 This is a schematic diagram of a MoE layer structure;

[0024] Figure 2 A schematic block diagram of a cluster topology provided in at least one embodiment of this disclosure;

[0025] Figure 3 This is a schematic structural diagram of a general-purpose graphics processor;

[0026] Figure 4A This is a schematic diagram of a first topology provided in an embodiment of the present disclosure;

[0027] Figure 4B This is a schematic diagram of a second topology provided in an embodiment of the present disclosure;

[0028] Figure 5 A schematic diagram of the communication cables for a parallel processor cluster provided in at least one embodiment of this disclosure;

[0029] Figure 6 This is a schematic diagram of the connection relationship of a first topology provided in at least one embodiment of the present disclosure;

[0030] Figure 7 This is a schematic diagram of the connection relationship of the second topology provided in at least one embodiment of the present disclosure;

[0031] Figure 8 A schematic flowchart illustrating a topology switching method for a parallel processor cluster provided in at least one embodiment of this disclosure;

[0032] Figure 9 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of the present disclosure;

[0033] Figure 10 This is a schematic block diagram of another electronic device provided in one embodiment of the present disclosure. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the described embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.

[0035] Unless otherwise defined, the technical or scientific terms used in this disclosure shall have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar terms used in this disclosure do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships, and these relative positional relationships may change accordingly when the absolute position of the described object changes. To keep the following description of the embodiments of this disclosure clear and concise, detailed descriptions of some known functions and components are omitted.

[0036] The Transformer model is a classic neural network-based text processing model that has spurred the development of important models that have greatly advanced the field of NLP (Natural Language Processing), such as BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-trained Transformer), and LLM (Large Language Model).

[0037] Represented by GPT (Generative Pre-trained Transformer) and LLAMA (Large Language Model Meta AI), the core structure of these models consists of multi-layered stacked Transformer decoder blocks (generative models). Each decoder block contains two core components: a multi-head attention mechanism used to compute dependencies between tokens, and a feed-forward network (FFN) used to perform non-linear transformations on the features of each token, enhancing the model's expressive power. The computation of both components relies on enormous tensors; for example, the weight matrix dimension is typically over a thousand dimensions, such as the 4096-dimensional hidden layer dimension of LLAMA 27b.

[0038] When training network models such as GPT and LLAMA, tensor parallelism is usually used to split the tensors, which splits the model's tensors (such as weight matrices, input and output feature maps, and other multi-dimensional data) onto multiple devices (GPUs / TPUs). Each device holds only a portion of the tensors and performs local computations. Finally, the results are aggregated through cross-device communication to achieve parallel acceleration.

[0039] For network models like GPT and LLAMA, there is a significant amount of tensor parallel communication internally. This includes communication operators such as the ring-full-reduction operator for tensor synchronization between devices, and the use of the full-collection operator to collect and summarize tensors from various devices. Tensor parallelism has a clear structured characteristic: the communication data consists of fragmented tensors that need to be summarized or split according to dimensions, rather than random pairwise data exchange. Furthermore, the communication scenarios are concentrated on intra-layer global collaboration, such as the All-Gather operator obtaining complete tensors and the Reduce-Scatter operator splitting results, rather than point-to-point independent communication.

[0040] Therefore, network models like GPT and LLAMA are better suited to network topologies where groups are fully interconnected within each group, groups are paired, and the communication bandwidth on the links within and between groups is consistent. In this case, GPT and LLAMA networks achieve optimal performance. This is because tensor parallel communication is essentially global collaboration; all GPUs converge data to the same GPU, or GPUs distribute data to all GPUs. For example, the ring full reduction operator enables data to be transmitted and concatenated segment by segment in the ring link. The throughput of the entire process is determined by the slowest link in the link. This topology allows the communication operator to fully utilize all link bandwidth, achieving theoretically maximum performance.

[0041] The MOE structure is another very important structure in the Transformer model. It improves model performance by assigning different tokens (also known as token feature vectors) to appropriate experts.

[0042] Figure 1 This is a schematic diagram of a MoE layer structure.

[0043] like Figure 1 As shown, the MoE layer includes multiple expert networks and routing modules.

[0044] Expert networks are the core computational units in the MoE layers. Each expert network is an independent neural network, typically in the form of a feedforward network (FFN). Each expert network is designed to focus on processing specific tasks or data features. For example, in natural language processing, some expert networks may focus on syntactic analysis, while others may focus on semantic understanding.

[0045] The routing module determines which expert networks each input term should be sent to for processing. For example, the number of expert networks to which each term is sent can be preset, such as 2, 4, 6, 8, etc.

[0046] In the MoE model, input tokens are distributed to a small number of suitable experts via a routing module (e.g., only 8 out of 256 experts are activated). These experts are often distributed across different GPUs. In this case, the alltoall operator is needed to distribute the tokens from their respective GPUs to the corresponding expert's GPU, and after computation, the results are aggregated back to the original GPU.

[0047] Therefore, when training the MoE model with full interconnection between GPUs, the scheduling time for lexical units can be significantly reduced.

[0048] The two network models described above are commonly used large-scale models, but due to their different structures, their requirements for the processor cluster topology are completely different. However, once the hardware connections are fixed, this topology is only suitable for the corresponding model and performs poorly in other models.

[0049] For example, if a fully interconnected topology between processors is applied to network models such as GPT and LLAMA, due to the large amount of tensor parallel communication in these networks, a large number of GPUs may initiate communication simultaneously, causing severe link conflicts. Moreover, the essence of tensor parallel communication is local tensor aggregation or splitting, and the redundancy of fully interconnected links cannot improve the efficiency of this type of global communication. On the contrary, it will lead to bandwidth dispersion and performance reduction due to too many links. Tensor parallel communication has structured characteristics. For example, after splitting by tensor dimensions, only specific GPUs need to transmit data of the corresponding dimension. The large number of redundant links in the fully interconnected network are idle most of the time, resulting in a waste of bandwidth resources.

[0050] For example, if a topology suitable for network models like GPT and LLAMA is applied to the MoE model, the communication latency will increase significantly when data transmission needs to be relayed through a processor. In this topology, the parallel processors are not fully connected, resulting in high communication latency for non-directly connected links when using operators such as alltoall, forming a low-bandwidth bottleneck. This means that the high bandwidth of directly connected links cannot be utilized, leading to a decrease in overall performance and affecting the inference and training efficiency of the MoE model, thus failing to fully utilize the hardware performance.

[0051] This disclosure provides at least one embodiment of a parallel processor cluster capable of topology switching, and a method for switching the topology of the parallel processor cluster. The parallel processor cluster includes M parallel processors and N optical circuit switches. Each parallel processor has N ports that are one-to-one coupled to the N optical circuit switches. The total number of ports for each parallel processor is greater than M, and N is less than M. The topology of the parallel processor cluster supports switching between a first topology and a second topology. The parallel processor cluster is configured to switch between the first and second topologies via the N optical circuit switches according to the computational task executed by the parallel processor cluster. In the first topology, the M parallel processors are divided into L groups, each group including M / L parallel processors. The M / L parallel processors are fully interconnected, and each M / L parallel processor is communicatively connected to the M / L parallel processors in each other group. Each M / L parallel processor is connected to different M / L parallel processors within each other group. In the second topology, the M parallel processors are fully interconnected. M, N, and L are positive integers.

[0052] The parallel processor cluster provided in at least one embodiment of this disclosure can perform topology switching at the driver layer without changing the hardware wiring, enabling the parallel processor cluster to adapt to different computing tasks and achieve good performance in the network models corresponding to different computing tasks, thus giving full play to the performance of hardware resources.

[0053] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings, but this disclosure is not limited to these specific embodiments.

[0054] Figure 2 This is a schematic block diagram of a cluster topology provided for at least one embodiment of the present disclosure.

[0055] like Figure 2 As shown, the parallel processor cluster 100 includes M parallel processors 101 and N optical circuit switches 102, with each parallel processor having N ports coupled to N optical circuit switches. It should be noted that, for clarity, Figure 2 The diagram shows the connection between the parallel processor and the optical circuit switch, as well as the connection between the parallel processor and the electrical switch and the central processing unit. However, the connection between parallel processors is not shown. For a detailed description of the connection between parallel processors, please refer to the following text.

[0056] For example, M parallel processors 101 can belong to one node in a parallel processor cluster, which can also provide other nodes with the same structure.

[0057] An optical circuit switch (OCS) is a dedicated network switching device based on optical layer transmission. Its core function is to directly establish or disconnect the physical optical path connection between optical fiber ports through optical switching devices, so as to realize the transparent transmission of optical signals without going through photoelectric or electro-optical conversion links.

[0058] In this disclosure, coupling refers to the establishment of a relationship between two or more components that allows the transfer of energy or signals. Coupling includes, for example, electrical connections or optical coupling. Communication connection refers to the establishment of a direct communication relationship between two parallel processors through a direct communication cable or a communication cable + optical circuit switch, enabling them to exchange data directly without the need for forwarding through other parallel processors.

[0059] For example, a parallel processor can establish a communication connection with another parallel processor through an optical circuit switch. This communication connection includes both optical signal transmission based on the optical circuit switch and signal transmission via communication cables. Alternatively, a parallel processor can directly establish a communication connection with another parallel processor via a communication cable, which can be achieved by directly connecting the ports of the two parallel processors through the communication cable. For example, the communication cable can be made of optical fiber, copper cable, etc.

[0060] For example, the parallel processors in the parallel processor cluster provided in at least one embodiment of this disclosure include graphics processing units (GPUs), general-purpose graphics processing units (GPGPUs), tensor processing units (TPUs), data processing units (DPUs), neural processing units (NPUs), etc.

[0061] For example, the parallel processor cluster can be a multi-GPU system, which integrates multiple GPU cards and works together to complete large-scale parallel computing tasks. It is commonly used for tasks such as artificial intelligence training, scientific computing, and high-performance graphics rendering.

[0062] For example, in some embodiments, the parallel processor cluster provided in at least one embodiment of this disclosure can be a single-machine multi-card form, where multiple GPU cards are integrated in a single server or workstation, and inter-card communication is achieved through a bus (such as PCIe bus, AMD bus, etc.). For example, the parallel processor cluster can also be a multi-machine multi-card form, where multiple single-machine multi-card servers are interconnected through a high-speed network to form a cluster, and each card can not only communicate with other cards on its own machine, but also interact with cards in other groups through the network. For example, the parallel processor cluster can also be a TPU cluster, a multi-FPGA accelerator card system, etc., and this disclosure does not impose specific limitations on it.

[0063] For example, this parallel processor cluster overcomes the limitations of single-card computing power or memory through parallel computing, and is therefore widely used in scenarios requiring large-scale data processing, high-throughput computing, or low-latency parallel tasks. Applications include deep learning and large model training, high-performance computing, large-scale data processing and AI inference, real-time rendering and visualization, and other fields such as large model training, scientific computing, high-concurrency AI services, and real-time rendering.

[0064] Figure 3 This is a schematic diagram of a general-purpose graphics processor. For example, when the parallel processor is a graphics processor or a general-purpose graphics processor, its specific structure can be found in [reference needed]. Figure 3 As shown.

[0065] like Figure 3 As shown, a general-purpose graphics processor is actually an array of programmable multiprocessors. For example, a programmable multiprocessor can be a streaming processor cluster (SPC), such as including... Figure 3The diagram shows multiple streaming processor clusters, all with identical structures. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0066] like Figure 3 As shown, a streaming processor cluster comprises multiple compute units (CUs), all with identical structures. Each CU performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A CU contains multiple cores, each including an arithmetic logic unit (ALU), a floating-point unit, etc., which execute specific computational tasks. Furthermore, a CU also includes registers (e.g.,...). Figure 3 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0067] like Figure 3 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0068] like Figure 3 As shown, each streaming processor cluster also provides a buffer for data caching of multiple computing units within the streaming processor cluster.

[0069] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 3 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0070] In each computing unit, the thread beam scheduling / distribution module ( Figure 3(Not shown in the diagram) Thread bundles are scheduled and allocated so that multiple computing cores within the computing unit can run thread bundles. Depending on the number of computing cores in the computing unit, multiple thread bundles within a thread block can execute concurrently or in a time-sharing manner. Multiple threads within each thread bundle execute the same instructions. Memory-executed instructions are issued to shared memory within the computing unit or further issued to intermediate-level caches, global caches, or global memory (e.g., ...). Figure 3 High Bandwidth Memory (HBM) is used for read and write operations.

[0071] A parallel processor 101 includes multiple ports, the total number of which is greater than M, to implement the second topology described below, and N of these ports are connected one-to-one with N optical circuit switches 102. For example... Figure 2 As shown, each parallel processor's N ports are connected to N optical circuit switches.

[0072] The connection between the parallel processor's port and the optical circuit switch can be achieved via a communication cable, which is a physical communication connection. This communication cable is fixed and can use hardware-level interconnect technologies specifically designed for, for example, GPU clusters. It can be a communication cable optimized for high bandwidth and adapted to the deployment requirements of parallel processor clusters in large data centers; this disclosure does not impose any specific limitations in this regard.

[0073] The topology of the parallel processor cluster 100 includes a first topology and a second topology, and the parallel processor cluster 100 supports switching between the first topology and the second topology. Of course, the topology of the parallel processor cluster 100 can also include more topologies, and the parallel processor cluster 100 can also support switching between these topologies. The logic is similar to that of the first topology and the second topology, and will not be elaborated here.

[0074] When switching topologies, the parallel processor cluster 100 is configured to switch between a first topology and a second topology through N optical circuit switches, depending on the computing tasks executed by the parallel processor cluster.

[0075] In addition, such as Figure 2 As shown, the parallel processor cluster 100 also includes multiple electrical switches and multiple central processing units. Each central processing unit is communicatively connected to P electrical switches, and each electrical switch is communicatively connected to Q parallel processors.

[0076] Each power switch is configured to forward data streams between the central processing unit (CPU) and the Q parallel processors. Each CPU is configured to schedule, coordinate, and control the P*T parallel processors connected to it via the P power switches. Here, P, Q, and T are positive integers and can be set according to the cluster deployment requirements.

[0077] For example, the electrical switch can be a PCIe switch. Of course, this disclosure is not limited to this, and electrical switches with other structures can also be used.

[0078] In the first topology, it adopts a fully interconnected structure within groups and pairwise interconnected structures between groups. Specifically, in the first topology, M parallel processors are divided into L groups, each group containing M / L parallel processors. The M / L parallel processors are fully interconnected, and each M / L parallel processor in each of the other groups has a pairwise communication connection. The M / L parallel processors are also connected to different M / L parallel processors within each of the other groups. M, N, and L are positive integers.

[0079] Figure 4A This is a schematic diagram of a first topology provided in an embodiment of the present disclosure.

[0080] exist Figure 4A In the example, M=8, L=2. It should be noted, however, that this disclosure is not limited to this; it is merely an example. Figure 4A The first topology of a possible embodiment is described using an example.

[0081] exist Figure 4A The first topology shown includes two groups, each containing four parallel processors. These four parallel processors are fully interconnected, meaning that every two parallel processors within a group have a direct communication connection, which can be achieved via communication cables or a combination of communication cables and optical circuit switches. Furthermore, each of the four parallel processors in one group is connected to a corresponding parallel processor in the other group, with each of the four parallel processors in the first group corresponding to one of the four parallel processors in the second group.

[0082] In addition, Figure 4A The diagram also shows a central processing unit (CPU) and electrical switches. As previously described, the CPU is used to schedule, coordinate, and control four parallel processors connected to it via two electrical switches. Each electrical switch is used to forward data flows between the two connected parallel processors and the CPU.

[0083] In the second topology, all M parallel processors are interconnected.

[0084] Figure 4BThis is a schematic diagram of a second topology provided in an embodiment of the present disclosure.

[0085] exist Figure 4B In the example, M=4. Of course, it should be noted that this disclosure is not limited to this.

[0086] like Figure 4B As shown, the four parallel processors are fully interconnected, meaning that there is a direct communication connection between every two parallel processors.

[0087] For example, the number of N is the same as the number of ports whose connectivity changes in both topologies. Here, connectivity refers to the connections between ports of the parallel processors, but physically they are still all connected to the optical circuit switch; the port objects connected through the optical circuit switch change.

[0088] For example, the ports of each parallel processor that need to be connected to other processors through the optical circuit switches are different. That is, the port connection relationship of the N ports changes in the first topology and the second topology. The number of N is determined according to the first topology and the second topology. The N optical circuit switches are configured to establish communication connection relationships between the N ports of each of the M parallel processors according to the topology configuration file, so as to form the first topology or the second topology.

[0089] For example, in the first topology, the N ports of each parallel processor are connected to the ports of other parallel processors according to a first pairing relationship. In the second topology, the N ports of each parallel processor are connected to the ports of other parallel processors according to a second pairing relationship. The first pairing relationship and the second pairing relationship are completely different.

[0090] For example, the connection relationships of the ports other than N ports in each parallel processor are the same in the first and second topologies, and the other ports of each parallel processor are connected by communication cables.

[0091] Specifically, the port connections of each parallel processor in the first and second topologies are compared. For ports that remain unchanged in both topologies, communication cables are used to establish communication connections. For ports whose connections change, N optical circuit switches are used to establish communication connections. Therefore, when a topology switch occurs, communication cables are used for ports whose connections remain unchanged, while optical circuit switches are used for ports whose connections change, thus enabling topology switching.

[0092] When the parallel processor cluster supports more topologies, a similar logic can be used: common connections in the topology are established using communication cables, while other ports are established using optical circuit switches. This allows for topology switching through optical circuit switches, minimizing the number of optical circuit switches required and reducing cabling costs.

[0093] For example, optical circuit switches configure paths through topology configuration files, transforming user-defined optical path connection rules into port mapping instructions for the switch hardware, thereby enabling the establishment or release of physical paths at the optical layer.

[0094] Therefore, by using optical circuit switches to perform topology switching at the driver layer, software-defined dynamic reconstruction of the communication topology of the parallel processor cluster is achieved. This eliminates the need for any physical hardware rewiring and allows for flexible switching of the cluster topology mode based on the communication characteristics of different computing tasks, thus achieving excellent performance across diverse computing power models. This dynamic topology switching mechanism breaks the limitation of traditional hardware topologies being fixed once deployed, enabling the parallel processor cluster to adaptively meet the differentiated needs of various models such as training, inference, and scientific computing. This achieves optimal performance across all scenarios while ensuring controllable hardware deployment costs.

[0095] For example, in some embodiments, a parallel processor cluster executes computational tasks based on the parallel processor cluster, switching between a first topology and a second topology via N optical circuit switches, including performing the following operations: in response to a computational task being an inference or training task of a first model, configuring N optical circuit switches according to a first topology profile to switch the parallel processor cluster to the first topology, wherein during inference or training of the first model, data is split by dimension and distributed to different parallel processors for distributed computation, and the computation results are aggregated; in response to a computational task being an inference or training task of a second model, configuring N optical circuit switches according to a second topology profile to switch the parallel processor cluster to the second topology, wherein during inference or training of the second model, M parallel processors exchange data bidirectionally in pairs.

[0096] For example, the communication operators in the first model include operators that communicate through tensor parallelism or sequence parallelism. These communication operators include ring reduce operators, all-gather operators, and scattering operators. Ring reduce operators, all-gather operators, and scattering operators belong to set communication operators. Ring reduce operators (Ring Reduce) involve distributed parallel processors sequentially passing and aggregating data in a ring topology, ultimately resulting in all parallel processors obtaining a global aggregation result (such as summation or averaging). All-gather operators (All-gather) involve each parallel processor broadcasting its local data to all other parallel processors, ultimately resulting in all parallel processors holding the complete global data set. Scattering operators (Scatter) involve the root node partitioning the global data and distributing it to each child node according to preset rules. Scattering operators include reduce scattering (ReduceScatter), etc.

[0097] For example, the first model can be a network model that includes a large number of communication methods, such as tensor parallel or sequence parallel communication, like traditional Transformer models such as GPT and LLAMA. Of course, this disclosure does not limit the specific network model. For any network model with similar communication requirements to traditional Transformer models such as GPT and LLAMA, the first topology provided in at least one embodiment of this disclosure can be applied.

[0098] In this first topology, network models with communication requirements such as tensor parallelism or sequence parallelism can fully leverage hardware performance advantages during inference or training. As mentioned earlier, this communication mode, in which data is split dimensionally and distributed to different parallel processors for distributed computation, and the computation results are aggregated, is essentially global collaboration. All GPUs converge data to the same GPU, or GPUs distribute data to all GPUs. The communication has structured characteristics, with specific GPUs transmitting data of corresponding dimensions. Operators under this first topology can fully utilize all bandwidth on the link to achieve optimal performance.

[0099] For example, the hardware communication bandwidth and communication cable bandwidth between the N ports on each parallel processor and the N optical circuit switches are consistent. The N ports on each parallel processor are coupled to the N optical circuit switches via communication cables, and the data transmission bandwidth of the communication connections between the M parallel processors is consistent. The latency of optical path transmission is extremely small and negligible. The bandwidth of the communication cables connecting the parallel processors to the optical circuit switches is the same as the bandwidth of the communication cables between the parallel processors. This ensures that the link bandwidth remains consistent for the parallel processors regardless of whether it is in the first or second topology, and the entire link speed is matched. During operation, the operator can fully utilize all the bandwidth on the link, avoiding bottlenecks caused by low-bandwidth links and achieving theoretically maximum performance.

[0100] For example, the communication operators in the second model include operators that send data to all other parallel processors and receive data from all other parallel processors at the same time; the communication operators include all-to-all operators.

[0101] For example, the second model could be the MoE model, or other network models with a large number of, for example, all-to-all operator communications. These models require sending data from the GPU to any possible GPU, and then aggregating the results back to the original GPU after computation. Because there is a possibility that any two parallel processors may need to communicate, these models can significantly shorten the scheduling time of tokens when the parallel processors are fully interconnected.

[0102] For example, each optical circuit switch is connected to the same port number on M parallel processors via a communication cable. For instance, one optical circuit switch is connected to port 2 on the M parallel processors via a communication cable, and another optical circuit switch is connected to port 4 on the M parallel processors via a communication cable.

[0103] When cabling parallel processor clusters, a symmetrical cabling method is often used to ensure that the links from each parallel processor to the network core, load, or other parallel processors have consistent physical and transmission characteristics. Each optical circuit switch establishes communication connections with ports of the same port number on M parallel processors. Ports with the same port number can be connected through the optical circuit switch, thereby maintaining symmetrical cabling and simplifying the topology design of the parallel processor cluster.

[0104] For example, in some embodiments, the parallel processor includes a chip core.

[0105] For example, in other embodiments, each parallel processor includes A chip cores, where A is a positive integer greater than 1.

[0106] A chip core can be a die, which is an independent silicon wafer containing complete, specific functional circuitry and serves as the core carrier of the computing power and functionality of a parallel processor. Parallel processors, such as graphics processing units (GPUs), can utilize chiplet technology to package multiple chip cores into a single unit, significantly increasing computing power and improving processor performance.

[0107] In a scenario where each parallel processor comprises multiple chip cores, for the first topology, all M / L parallel processors are fully interconnected, including all i-th chip cores of the M / L parallel processors, where i is a positive integer ranging from 1 to A. For example, assuming each parallel processor comprises two chip cores, namely the first chip core and the second chip core, the full interconnection of the i-th chip cores of the M / L parallel processors means that the first chip cores of the M / L parallel processors within the same group are fully interconnected, and the second chip cores of the M / L parallel processors within the same group are also fully interconnected.

[0108] For example, the first and second chip cores are structurally or functionally independent and can be used as two independent chip cores to perform computing tasks.

[0109] For the first topology, M / L parallel processors are communicatively connected to each of the other M / L parallel processors in each group. The i-th chip core of each of the M / L parallel processors is also communicatively connected to the i-th chip core of each of the other M / L parallel processors in each group. For example, assuming each parallel processor includes two chip cores, namely a first chip core and a second chip core, for each of the M / L parallel processors in a certain group, the first chip core of that parallel processor is communicatively connected to the first chip core of a corresponding parallel processor in each of the other M / L parallel processors, and the second chip core of that parallel processor is communicatively connected to the second chip core of that corresponding parallel processor in each of the other M / L parallel processors.

[0110] For example, in the first topology, multiple parallel processors in one group are connected in pairs to multiple parallel processors in another group. Each parallel processor is connected to one parallel processor in the other group. Different parallel processors are connected to different parallel processors in the other group. In the first topology, the communication connection between two parallel processors includes a corresponding communication connection between each chip core of the two parallel processors.

[0111] In the case where each parallel processor comprises multiple chip cores, for the second topology, all M parallel processors are fully interconnected, including one chip core of each parallel processor communicating with the corresponding chip cores of the other M-1 parallel processors. In other words, for the second topology, two parallel processors are considered to be communicating if at least one chip core of each processor has a direct communication connection.

[0112] The following section details the hardware connectivity of the parallel processor clusters under the two topologies described above.

[0113] For example, in some embodiments, for the first topology and the second topology, communication cables are used to connect ports that do not change during switching, while communication connections are established through optical circuit switches when the port connection changes. This allows for topology switching to be achieved with a minimum number of optical circuit switch modules, reducing cluster deployment costs.

[0114] For example, regarding fixed communication cables, in some embodiments, at least one of the A chip cores of each parallel processor in each group has a communication connection with the corresponding chip cores of the A chip cores of all other parallel processors in the group. Each of the M / L parallel processors also has a communication connection via a communication cable with one of the M / L parallel processors included in each other group. The communication connection includes at least one of the A chip cores of the parallel processor and a corresponding chip core of the A chip cores of the parallel processor.

[0115] It should be noted that in the above description, groups are used to describe the parallel processors and connected objects for easy location. The group represents the relationship in the first topology and does not constitute a physical limitation for the second topology. For example, in the second topology, each parallel processor is still regarded as an independent individual, resulting in a fully connected relationship.

[0116] Figure 5 This is a schematic diagram of the communication cables for a parallel processor cluster provided in at least one embodiment of the present disclosure.

[0117] exist Figure 5 In the example, the parallel processor includes two chip cores, namely a first chip core and a second chip core. Each chip core includes five ports: the first chip core includes ports 1 to 5, and the second chip core includes ports 6 to 10. Of course, this disclosure is not limited to this, and it does not impose specific limitations on the number of chip cores included in the parallel processor or the number of ports included in each chip core.

[0118] Figure 5 The diagram shows eight parallel processors, which are referred to as parallel processors 101_0 to 101_7 for clarity. Of course, this disclosure is not limited to these, and the parallel processor cluster may include more parallel processors.

[0119] For example, Figure 5 An example could be the connection relationship of a node in a parallel processor cluster. A parallel processor cluster can also have more nodes; the connection relationships within a node can be found in [reference needed]. Figures 5-7 Related descriptions.

[0120] For example, in Figure 5In the example, for the first topology, parallel processors 101_0, 101_1, 101_2, and 101_3 belong to one group, while parallel processors 101_4, 101_5, 101_6, and 101_7 belong to another group. Of course, the parallel processor cluster can include more groups, and this disclosure does not impose specific limitations.

[0121] In addition, parallel processor clusters may also include a central processing unit and an electrical switch; the specific connection relationships can be found in the foregoing. Figure 2 and Figure 4A The relevant description is in Figure 5 This is not shown again. For example, port 1 of each parallel processor is connected to an electrical switch for data exchange with the central processing unit.

[0122] For example, the first chip core of each parallel processor is connected to the first chip cores of T1 parallel processors in the same group via a communication cable. The second chip core of each parallel processor is connected to the second chip cores of T2 parallel processors in the same group via a communication cable. The first chip core of each parallel processor is also connected to the first chip core of a corresponding parallel processor in another group via a communication cable. Here, T1 and T2 are positive integers, T1 + T2 = M / L - 1, and the T1 and T2 parallel processors are different. The communication cables can be copper cables (for electrical signals) or optical cables (for optical signals).

[0123] For multiple parallel processors within a group, such as Figure 5 As shown, taking parallel processor 101_0 as an example, port 3 of the first chip core of parallel processor 101_0 and port 3 of the first chip core of parallel processor 101_2 are connected by a communication cable to establish a direct communication connection.

[0124] Port 7 of the second chip core of parallel processor 101_0 is connected to port 7 of the second chip core of parallel processor 101_3 via a communication cable to establish a direct communication connection; port 9 of the second chip core of parallel processor 101_0 is connected to port 9 of the second chip core of parallel processor 101_1 via a communication cable to establish a direct communication connection.

[0125] Of course, when the group includes more parallel processors, communication connections can be established according to the above logic, which will not be elaborated here.

[0126] The connection relationships between other parallel processors and parallel processors within the same group are similar and will not be elaborated here.

[0127] In addition, such as Figure 5As shown, port 5 of the first chip core of parallel processor 101_0 and port 5 of parallel processor 101_7 are connected by a communication cable to establish a direct communication connection; port 5 of the first chip core of parallel processor 101_3 and port 5 of parallel processor 101_4 are connected by a communication cable to establish a direct communication connection; port 5 of the first chip core of parallel processor 101_1 and port 5 of parallel processor 101_6 are connected by a communication cable to establish a direct communication connection; port 5 of the first chip core of parallel processor 101_2 and port 5 of parallel processor 101_5 are connected by a communication cable to establish a direct communication connection.

[0128] When the computational task performed by the parallel processor cluster is the inference or training task of the first model, N optical circuit switches are configured according to the first topology configuration file to switch the parallel processor cluster to the first topology.

[0129] For example, in the first topology, each chip core in the parallel processor establishes a communication connection with a target chip core through N optical circuit switches. The target chip core is the chip core that, according to the first topology, has not yet established a communication connection with A other chip cores in the parallel processor. In other words, for the connection relationships required by the first topology, those not yet established via communication cables are established through N optical circuit switches.

[0130] Figure 6 A schematic diagram of the connection relationship of a first topology provided for at least one embodiment of this disclosure.

[0131] exist Figure 6 In the middle, solid lines represent Figure 5 The communication cables in the diagram are not repeated here. Dashed lines represent communication connections established through optical circuit switches, and the boxes on each dashed line indicate the switch number.

[0132] For example, in this embodiment, four optical circuit switches are shown, which are referred to as optical circuit switch 102_0, optical circuit switch 102_1, optical circuit switch 102_2 and optical circuit switch 102_3 respectively for distinction. Optical circuit switch 102_0 is connected to port 2 of each parallel processor through a communication cable, optical circuit switch 102_1 is connected to port 4 of each parallel processor through a communication cable, optical circuit switch 102_2 is connected to port 6 of each parallel processor through a communication cable, and optical circuit switch 102_3 is connected to port 8 of each parallel processor through a communication cable.

[0133] In the first topology, the first chip core of each parallel processor establishes a communication connection with the first chip cores of the M / 2-T1-1 parallel processors in the group (excluding the T1 parallel processor) through N optical circuit switches. The second chip core of each parallel processor establishes a communication connection with the second chip cores of other parallel processors in the group (excluding the T2 parallel processor) through N optical circuit switches. The second chip core of each parallel processor also establishes a communication connection with the second chip core of the corresponding parallel processor in another group through N optical circuit switches.

[0134] Taking the parallel processor 101_0 as an example, according to the definition of the first topology, the first chip core of the parallel processor 101_0 also needs to establish a communication connection with the first chip core of the parallel processor 101_1 and the first chip core of the parallel processor 101_3.

[0135] Using the first topology configuration file, the optical circuit switch 102_0 is configured to connect port 2 of parallel processor 101_0 and port 2 of parallel processor 101_1, thereby establishing a communication connection between port 2 of parallel processor 101_0 and port 2 of parallel processor 101_1. Furthermore, using the first topology configuration file, the optical circuit switch 102_1 is configured to connect port 4 of parallel processor 101_0 and port 4 of parallel processor 101_3, thereby establishing a communication connection between port 4 of parallel processor 101_0 and port 4 of parallel processor 101_3. This achieves communication between the first chip core of parallel processor 101_0 and the first chip cores of all other parallel processors in the group.

[0136] According to the definition of the first topology, the second chip core of the parallel processor 101_0 also needs to establish a communication connection with the second chip core of the parallel processor 101_2.

[0137] Using the first topology configuration file, the optical circuit switch 102_3 is configured to connect port 8 of parallel processor 101_0 and port 8 of parallel processor 101_2, thereby establishing a communication connection between port 8 of parallel processor 101_0 and port 8 of parallel processor 101_2. This enables communication between the second chip core of parallel processor 101_0 and the second chip cores of all other parallel processors in the group.

[0138] Furthermore, according to the definition of the first topology, the second chip core of the parallel processor 101_0 also needs to establish a communication connection with the second chip core of another set of parallel processors 101_7.

[0139] By configuring the optical circuit switch 102_2 to connect port 6 of parallel processor 101_0 and port 6 of parallel processor 101_7 through the first topology configuration file, a communication connection is established between port 6 of parallel processor 101_0 and port 6 of parallel processor 101_7. This enables communication between the second chip core of parallel processor 101_0 and the second chip core of another corresponding set of parallel processors.

[0140] Let's take parallel processor 101_4 as an example. According to the definition of the first topology, the first chip core of parallel processor 101_4 also needs to establish communication connections with the first chip cores of parallel processor 101_5 and parallel processor 101_7.

[0141] Using the first topology configuration file, optical circuit switch 102_0 is configured to connect port 2 of parallel processor 101_4 and port 2 of parallel processor 101_5, thereby establishing a communication connection between port 2 of parallel processor 101_4 and port 2 of parallel processor 101_5. Furthermore, using the first topology configuration file, optical circuit switch 102_1 is configured to connect port 4 of parallel processor 101_4 and port 4 of parallel processor 101_7, thereby establishing a communication connection between port 4 of parallel processor 101_4 and port 4 of parallel processor 101_7. Thus, communication connections are achieved between the first chip core of parallel processor 101_4 and the first chip cores of all other parallel processors in the group.

[0142] According to the definition of the first topology, the second chip core of the parallel processor 101_4 also needs to establish a communication connection with the second chip core of the parallel processor 101_6.

[0143] By configuring the optical circuit switch 102_3 to connect port 8 of parallel processor 101_4 and port 8 of parallel processor 101_6 through the first topology configuration file, a communication connection is established between port 8 of parallel processor 101_4 and port 8 of parallel processor 101_6. This enables communication between the second chip core of parallel processor 101_4 and the second chip cores of all other parallel processors in the group.

[0144] Furthermore, according to the definition of the first topology, the second chip core of the parallel processor 101_4 also needs to establish a communication connection with the second chip core of another set of parallel processors 101_3.

[0145] By configuring the optical circuit switch 102_2 to connect port 6 of parallel processor 101_4 and port 6 of parallel processor 101_3 through the first topology configuration file, a communication connection is established between port 6 of parallel processor 101_4 and port 6 of parallel processor 101_3. This enables communication between the second chip core of parallel processor 101_4 and the second chip core of another corresponding set of parallel processors.

[0146] The connection relationships for other parallel processors are similar and will not be elaborated here.

[0147] Thus, the first topology is achieved through communication cables and optical circuit switches.

[0148] In the second topology, each parallel processor establishes a communication connection with at least one parallel processor through N optical circuit switches. The at least one parallel processor is a parallel processor in each of the other groups in the first topology that does not have a communication connection with any of the A chip cores of the parallel processor.

[0149] In the second topology, as long as a communication connection is established between two chip cores (communication cable or communication cable + optical circuit switch), it is considered that there is a connection between the two parallel processors. In the second topology, each processor is required to have a communication connection with other processors.

[0150] Figure 7 This is a schematic diagram of the connection relationship of a second topology provided for at least one embodiment of the present disclosure.

[0151] exist Figure 7 In the middle, solid lines represent Figure 5 The communication cables in the diagram are not repeated here. Dashed lines represent communication connections established through optical circuit switches, and the boxes on the dashed lines indicate the numbers of the optical circuit switches.

[0152] For example, in this embodiment, three optical circuit switches are shown, which are referred to as optical circuit switch 102_0, optical circuit switch 102_2, and optical circuit switch 102_3 for distinction. Optical circuit switch 102_0 is connected to port 2 of each parallel processor via a communication cable, optical circuit switch 102_2 is connected to port 6 of each parallel processor via a communication cable, and optical circuit switch 102_3 is connected to port 8 of each parallel processor via a communication cable.

[0153] The parallel processor cluster may also include an optical circuit switch 102_1. The optical circuit switch 102_1 is connected to port 4 of each parallel processor via communication cables. In the second topology, this interface can serve as an expansion interface. For example, ports 4 and 10 can be used as expansion ports to form a "2-machine supernode". For instance, assuming eight parallel processors are connected in a topology to form a node, a "2-machine supernode" means that one supernode includes two nodes. The parallel processors within a supernode can communicate with each other. For example, a parallel processor in one node can connect to a corresponding parallel processor in another node through an expansion port to establish a communication relationship.

[0154] For example, in other embodiments, port 1 connected to the electrical switch can also be reused to connect to and via the electrical switch. Figure 7 Other parallel processors can establish communication connections through port 6 of the parallel processor, and can then be expanded using ports 4, 6, and 10 of the parallel processor to form a "4-machine supernode". A "4-machine supernode" means that one supernode includes four nodes. For example, a parallel processor in one node, such as parallel processor 101_0, can establish communication relationships by expanding ports 4, 6, and 10 to connect with the corresponding parallel processors in the other three nodes (e.g., parallel processor 101_0 in the other three nodes).

[0155] For example, in the second topology, each parallel processor establishes a communication connection with the first or second chip core of at least one parallel processor in another group through N optical circuit switches, wherein at least one parallel processor is a parallel processor in the other group that does not have a communication connection with either the first or second chip core of the parallel processor.

[0156] Taking parallel processor 101_0 as an example, according to the definition of the second topology, parallel processor 101_0 also needs to establish communication connections with parallel processors 101_4, 101_5 and 101_6.

[0157] like Figure 7As shown, using the second topology configuration file, optical circuit switch 102_0 is configured to connect port 2 of parallel processor 101_0 and port 2 of parallel processor 101_5, thereby establishing a communication connection between port 2 of parallel processor 101_0 and port 2 of parallel processor 101_5. Furthermore, using the second topology configuration file, optical circuit switch 102_3 is configured to connect port 8 of parallel processor 101_0 and port 8 of parallel processor 101_6, thereby establishing a communication connection between port 8 of parallel processor 101_0 and port 8 of parallel processor 101_6. Also, using the second topology configuration file, optical circuit switch 102_2 is configured to connect port 6 of parallel processor 101_0 and port 6 of parallel processor 101_4, thereby establishing a communication connection between port 6 of parallel processor 101_0 and port 6 of parallel processor 101_4.

[0158] This enables communication between the parallel processor 101_0 and all other parallel processors.

[0159] Taking parallel processor 101_5 as another example, according to the definition of the second topology, parallel processor 101_5 also needs to establish communication connections with parallel processor 101_0, parallel processor 101_1 and parallel processor 101_3.

[0160] like Figure 7 As shown, through the second topology configuration file, optical circuit switch 102_0 is configured to connect port 2 of parallel processor 101_5 and port 2 of parallel processor 101_0, thereby establishing a communication connection between port 2 of parallel processor 101_5 and port 2 of parallel processor 101_0. Furthermore, through the second topology configuration file, optical circuit switch 102_3 is configured to connect port 8 of parallel processor 101_5 and port 8 of parallel processor 101_3, thereby establishing a communication connection between port 8 of parallel processor 101_5 and port 8 of parallel processor 101_3. And, through the second topology configuration file, optical circuit switch 102_2 is configured to connect port 6 of parallel processor 101_5 and port 6 of parallel processor 101_1, thereby establishing a communication connection between port 6 of parallel processor 101_5 and port 6 of parallel processor 101_1.

[0161] This enables communication between the parallel processor 101_5 and all other parallel processors.

[0162] The connection relationships for other parallel processors are similar and will not be elaborated here.

[0163] Thus, a second topology is achieved through communication cables and optical circuit switches.

[0164] comprehensive Figure 2,exist Figures 5-7 In the example, all parallel processors' ports 2, 4, 6, and 8 are connected to the same optical circuit switch, and the communication cables have consistent bandwidth. This maintains bandwidth consistency between chip cores or parallel processors, fully utilizing high bandwidth. The parallel processor's port 1 is connected to the optical circuit switch, and ports 3, 5, 7, and 9 of each parallel processor are connected according to... Figure 5 The diagram shows a fixed connection using communication cables. Ports 2, 4, 6, and 8 are switched via an optical circuit switch to form a first topology and a second topology. Port 10 of each parallel processor serves as an expansion port. The first topology is used to perform computational tasks related to traditional Transformer-type models such as GPT and LLAMA, while the second topology is used to perform computational tasks related to models such as MoE.

[0165] In the above embodiments, some communication connections that do not change the port connection relationship during topology switching are completed through communication cables, and some communication connections that change the port connection relationship during topology switching are completed through optical circuit switches. Without any physical hardware rewiring operations, the parallel processor cluster can be flexibly and efficiently switched between the first topology and the second topology. Thus, the topology can be switched according to different computing tasks, and the corresponding topology can be adapted to the communication requirements of computing tasks to maximize performance advantages and improve hardware generalization and flexibility.

[0166] Specifically, each parallel processor's N ports are fixedly coupled to their corresponding switches, while the optical circuit switch can dynamically configure the on / off states of optical switches and link mapping relationships to change the communication paths between the switch and other processor ports. This topology dynamic switching mechanism based on optical circuit switches avoids the problems of high operational complexity, long system downtime, and increased link failure rates caused by hardware rewiring. Furthermore, it can accurately match the optimal processor interconnection topology according to the computational characteristics and communication requirements of different artificial intelligence models, effectively reducing cross-processor data transmission latency and improving the utilization of computing resources. Whether for long-sequence parallel computation of large language models like Transformers or for the pairwise interactive data communication requirements of MoE models, this disclosure can adapt to the performance requirements of different models through flexible topology switching, ensuring system stability while achieving efficient operation and performance optimization under various models.

[0167] Of course, it should be noted that the above embodiments only show one possible cluster configuration and corresponding hardware wiring method, but this disclosure is not limited thereto.

[0168] It should be noted that, in at least one embodiment of this disclosure, the parallel processor cluster may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0169] For example, parallel processor clusters can be implemented in hardware, software, or a combination of both, and this disclosure does not impose any specific restrictions on this.

[0170] At least one embodiment of this disclosure also provides a method for topology switching of a parallel processor cluster.

[0171] Figure 8 This is a schematic flowchart illustrating a method for switching the topology of a parallel processor cluster provided in at least one embodiment of the present disclosure.

[0172] The parallel processor cluster consists of M parallel processors and N optical circuit switches. The N ports on each parallel processor are connected one-to-one with the N optical circuit switches. The total number of ports on each parallel processor is greater than M and N is less than M.

[0173] For details on the specific architecture of the parallel processor cluster, please refer to the aforementioned description of the parallel processor cluster; it will not be repeated here.

[0174] For example, the ports of the parallel processors connected to the N ports of each parallel processor are different in the first topology and the second topology. The N optical circuit switches are configured to establish communication connections between the N ports of each of the M parallel processors according to the topology configuration file, so as to form the first topology or the second topology.

[0175] For example, the connection relationships of the ports other than N ports in each parallel processor are the same in the first and second topologies, and the other ports of each parallel processor are connected by communication cables.

[0176] Therefore, during topology switching, the ports of the optical circuit switch can be configured to be open through the topology configuration file to switch the topology structure.

[0177] like Figure 8 As shown, the topology switching method includes step S10: according to the computing task executed by the parallel processor cluster, the topology of the parallel processor cluster is switched through N optical circuit switches, wherein the topology of the parallel processor cluster includes a first topology and a second topology.

[0178] In the first topology, M parallel processors are divided into L groups, each group includes M / L parallel processors, all M / L parallel processors are interconnected, and each M / L parallel processor communicates with the M / L parallel processors in each other group. Each M / L parallel processor is connected to a different M / L parallel processor in each other group. In the second topology, M parallel processors are all interconnected, and M, N, and L are positive integers.

[0179] For a detailed description of the first and second topologies, please refer to the description of the foregoing embodiments; it will not be repeated here.

[0180] For example, in some embodiments, step S10 may include: in response to a computing task being an inference or training task of a first model, configuring N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein during inference or training of the first model, data is split by dimension and distributed to different parallel processors for distributed computing, and the computing results are aggregated; in response to a computing task being an inference or training task of a second model, configuring N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein during inference or training of the second model, M parallel processors exchange data bidirectionally in pairs.

[0181] For example, the communication operators in the first model include operators that communicate via tensor parallelism or sequence parallelism, such as ring reduction operators, full collection operators, and scattering operators. For example, the first model can be a network model that includes a large number of network models with tensor parallelism or sequence parallelism communication, such as traditional Transformer-type models like GPT and LLAMA.

[0182] For example, the communication operators in the second model include operators that send data to all other parallel processors and simultaneously receive data from all other parallel processors; these communication operators include all-to-all operators. For instance, the second model could be a MoE model, or other network models with a large number of, for example, all-to-all operator communications.

[0183] Therefore, based on the communication requirements of the computing tasks executed by the parallel processor cluster, the optical circuit switches are configured through configuration files, and the connection relationships of the optical circuit switches are switched. This allows for software-defined dynamic reconstruction of the communication topology of the parallel processor cluster, without any physical hardware rewiring. The cluster topology mode can be flexibly switched according to the communication characteristics of different computing tasks, thus achieving excellent performance in diverse computing power models. This dynamic topology switching mechanism breaks the limitation of traditional hardware topologies being fixed once deployed, enabling the parallel processor cluster to adaptively meet the differentiated needs of various models such as training, inference, and scientific computing. It achieves optimal performance across all scenarios while ensuring controllable hardware deployment costs.

[0184] Regarding the circuit connection relationship between the first topology and the second topology during the switching process, please refer to the relevant description in the foregoing embodiments, which will not be repeated here.

[0185] For example, each parallel processor includes multiple chip cores. The topology switching and connection relationships in the case of multiple chip cores can be referred to the relevant description in the foregoing embodiments, and will not be repeated here.

[0186] The topology switching method for a parallel processor cluster provided in at least one embodiment of this disclosure can achieve similar technical effects to the aforementioned parallel processor cluster, and will not be described in detail here.

[0187] Figure 9 A schematic diagram of a non-transitory computer-readable storage medium provided for at least one embodiment of this disclosure. For example, such as Figure 9 As shown, storage medium 200 can be a non-transitory computer-readable storage medium on which one or more computer-readable instructions 201 can be stored non-transitory. For example, when the computer-readable instructions 201 are executed by a processor, one or more steps in the topology switching method described above can be performed.

[0188] For example, the storage medium 200 can be used in an electronic device 300, such as the storage medium 200 including the storage device 308 in the electronic device 300.

[0189] For example, a storage device may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. One or more computer-readable instructions may be stored on the computer-readable storage medium, and a processor may execute these instructions to perform various functions of the processor. Various application programs and various data may also be stored in the storage medium.

[0190] For example, the storage medium may include a memory card for a smartphone, a cache component for a tablet computer, a hard disk for a personal computer, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), flash memory, or any combination of the above storage media, or other suitable storage media.

[0191] Figure 10 This is a schematic block diagram of another electronic device provided as an embodiment of the present disclosure. Figure 10 As shown, the electronic device 300 is, for example, suitable for implementing the topology switching method provided in the embodiments of this disclosure. It should be noted that... Figure 10 The components of the electronic device 300 shown are merely exemplary and not limiting. The electronic device 300 may have other components as needed for the actual application.

[0192] like Figure 10 As shown, the electronic device 300 may include a processing device 301 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in memory to achieve various functions.

[0193] For example, when the computer-readable instruction processing device 301 executes, it can perform one or more steps of the topology switching method according to any of the above embodiments. It should be noted that a detailed description of the processing procedure of the topology switching method can be found in the relevant descriptions in the embodiments of the above topology switching method.

[0194] For example, the memory may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) 303 and / or cache memory, etc., whereby computer-readable instructions can be loaded from storage device 308 into RAM 303 to execute. Non-volatile memory may include, for example, read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), portable compact disc read-only memory (CD-ROM), USB storage, flash memory, etc. Various applications and various data, such as various data used and / or generated by applications, may also be stored in the computer-readable storage medium.

[0195] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are interconnected via a bus 304. The input / output (I / O) interface 305 is also connected to the bus 304.

[0196] Typically, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touchscreen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 308 including, for example, magnetic tape, hard disk, flash memory, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other electronic devices to exchange data. Although Figure 10 An electronic device 300 with various devices is shown, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 300 may alternatively implement or possess more or fewer devices. For example, a processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 may be a device with data processing capabilities and / or program execution capabilities, such as a central processing unit (CPU), a tensor processor (TPU), or a graphics processing unit (GPU). The central processing unit (CPU) may be an x86, ARM, RISC-V architecture, etc. The GPU may be directly integrated into the SOC, directly integrated onto the motherboard, or built into the northbridge chip of the motherboard.

[0197] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0198] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.

[0199] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0200] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0201] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0202] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

[0203] The following points should be noted regarding this disclosure:

[0204] (1) The accompanying drawings of the embodiments of this disclosure only involve the structures involved in the embodiments of this disclosure. Other structures can be referred to the general design.

[0205] (2) Where there is no conflict, the embodiments of this disclosure and the features in the embodiments can be combined with each other to obtain new embodiments.

[0206] The above description is only a specific embodiment of this disclosure, but the protection scope of this disclosure is not limited thereto. The protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A cluster of parallel processors, characterized in that, The parallel processor cluster comprises M parallel processors, N optical circuit switches, and N ports on each parallel processor are one-to-one coupled with the N optical circuit switches, the total number of ports of each parallel processor is greater than M, and N is less than M, The parallel processor cluster supports switching between a first topology and a second topology, and when switching the topologies, the parallel processor cluster is configured to switch between the first topology and the second topology through the N optical circuit switches according to a computing task performed by the parallel processor cluster, In the first topology, the M parallel processors are divided into L groups, each group comprising M / L parallel processors, the M / L parallel processors are fully connected, and the M / L parallel processors are respectively communicatively connected with the M / L parallel processors included in each other group, and the M / L parallel processors are respectively connected with different M / L parallel processors in each other group, In the second topology, the M parallel processors are fully connected, M, N, and L are positive integers.

2. The cluster of parallel processors of claim 1, wherein, The ports of the parallel processors connected by the N ports of each parallel processor are different in the first topology and the second topology, The N optical circuit switches are configured to establish a communicatively connected relationship between the N ports of the M parallel processors according to a topology configuration file to form the first topology or the second topology.

3. The cluster of parallel processors of claim 1, wherein, The connection relationship of other ports of each parallel processor except the N ports is the same in the first topology and the second topology, The other ports of each parallel processor establish a communicatively connected relationship using a communication cable.

4. The cluster of parallel processors of claim 1, wherein, Each optical circuit switch is coupled with a port of the same port number on the M parallel processors through a communication cable.

5. The cluster of parallel processors of claim 1, wherein, Each parallel processor comprises A chip cores, and A is a positive integer greater than 1, In the first topology, the M / L parallel processors are fully connected, including fully connecting the i-th chip cores of the M / L parallel processors, i is a positive integer and sequentially takes 1 to A, and The M / L parallel processors are respectively communicatively connected with the M / L parallel processors included in each other group, including the i-th chip cores of the M / L parallel processors and the i-th chip cores of the M / L parallel processors included in each other group are respectively communicatively connected; In the second topology, the M parallel processors are fully connected, including one chip core of each parallel processor is communicatively connected with the corresponding chip core in the other M-1 parallel processors.

6. The cluster of parallel processors of claim 5, wherein, At least one chip core of the A chip cores of each parallel processor in each group has a communicatively connected relationship with the corresponding chip core of the A chip cores of all other parallel processors in the group where the parallel processor is located through a communication cable, Each of the M / L parallel processors is further communicatively connected with one of the M / L parallel processors included in each of the other groups through the communication cable, the communication connection including at least one of the A chip cores of the parallel processor and a corresponding one of the A chip cores of the one of the parallel processors.

7. The cluster of parallel processors of claim 6, wherein, In the second topology, each parallel processor is communicatively connected with at least one parallel processor included in each of the other groups through the N optical circuit switches, wherein the at least one parallel processor is one of the M / L parallel processors included in each of the other groups that does not have the communication connection relationship with any of the A chip cores of the parallel processor. In the first topology, each chip core in each parallel processor is communicatively connected with a target chip core through the N optical circuit switches, wherein the target chip core is one that has not established the communication connection relationship with any of the A chip cores of the parallel processor according to the first topology.

8. The cluster of parallel processors of claim 5, wherein, In response to A=2, L=2, each parallel processor includes two chip cores, including a first chip core and a second chip core, The first chip core of each parallel processor is communicatively connected with the first chip cores of T1 parallel processors in the group through the communication cable, and the second chip core of each parallel processor is communicatively connected with the second chip cores of T2 parallel processors in the group through the communication cable, and the first chip core of each parallel processor is further communicatively connected with the first chip core of a corresponding parallel processor in another group through the communication cable, wherein T1 and T2 are positive integers, T1+T2=M / L-1, and the T1 parallel processors are different from the T2 parallel processors.

9. The parallel processor cluster of claim 8, wherein, In the first topology, the first chip core of each parallel processor is communicatively connected with the first chip cores of M / 2-T1-1 parallel processors in the group other than the T1 parallel processors through the N optical circuit switches, The second chip core of each parallel processor is communicatively connected with the second chip cores of other parallel processors in the group other than the T2 parallel processors through the N optical circuit switches, The second chip core of each parallel processor is further communicatively connected with the second chip core of the corresponding parallel processor in the other group through the N optical circuit switches.

10. The parallel processor cluster of claim 8, wherein, In the second topology, each parallel processor is communicatively connected with the first chip core or the second chip core of at least one parallel processor in the other group through the N optical circuit switches, wherein the at least one parallel processor is one of the parallel processors in the other group that does not have the communication connection relationship with the first chip core and the second chip core of the parallel processor.

11. The cluster of parallel processors of any of claims 1-10, wherein, The parallel processor cluster performs a computing task according to the parallel processor cluster, and switches the first topology and the second topology through the N optical circuit switches, including performing the following operations: In response to the computing task being an inference or training task of a first model, the N optical circuit switches are configured according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is in inference or training, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated. In response to the computing task being an inference or training task of a second model, the N optical circuit switches are configured according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein when the second model is in inference or training, data is exchanged between the M parallel processors in a two-way manner.

12. The cluster of parallel processors of claim 11, wherein, The communication operator of the first model includes an operator that communicates through tensor parallel or sequence parallel, and the communication operator includes a ring reduction operator, a full collection operator, and a scattering operator.

13. The cluster of parallel processors of claim 11, wherein, Wherein, The communication operator of the second model includes an operator that sends data to all other parallel processors and receives data from all other parallel processors, and the communication operator includes a full-to-full operator.

14. The cluster of parallel processors of any of claims 1-10, wherein, The N ports on each parallel processor are coupled to the N optical circuit switches through communication cables, and the data transmission bandwidth of the communication connection between the M parallel processors is consistent.

15. A method of topology switching of a cluster of parallel processors, characterized in that, The parallel processor cluster includes M parallel processors and N optical circuit switches, the N ports on each parallel processor are one-to-one connected to the N optical circuit switches, the total number of ports on each parallel processor is greater than M, and N is less than M, The topology switching method includes: According to the computing task performed by the parallel processor cluster, the topology structure of the parallel processor cluster is switched through the N optical circuit switches, wherein the topology structure of the parallel processor cluster includes a first topology and a second topology; In the first topology, the M parallel processors are divided into L groups, each group includes M / L parallel processors, the M / L parallel processors are fully connected, and the M / L parallel processors are respectively connected to the M / L parallel processors included in each other group, and the M / L parallel processors are respectively connected to different M / L parallel processors in each other group, In the second topology, the M parallel processors are fully connected, M, N, and L are positive integers.

16. The topology switching method of claim 15, wherein, According to the computing task performed by the parallel processor cluster, the topology structure of the parallel processor cluster is switched through the N optical circuit switches, including: In response to the computing task being an inference or training task of a first model, the N optical circuit switches are configured according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is in inference or training, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated. In response to the computing task being an inference or training task of a second model, the N optical circuit switches are configured according to a second topology configuration file to switch the parallel processor cluster to the second topology structure, wherein the second model exchanges data between the M parallel processors in a two-by-two bidirectional manner when inferring or training.

17. An electronic device, comprising: Comprise: a memory, which stores computer executable instructions non-transiently; a processor, which is configured to run the computer executable instructions, wherein the computer executable instructions, when run by the processor, implement the topology switching method according to claim 15 or 16.

18. A non-transitory computer-readable storage medium, comprising: The non-transient computer readable storage medium stores computer executable instructions, The computer executable instructions, when executed by the processor, implement the topology switching method according to claim 15 or 16.

Images