Parallel processor cluster, topology switching method, electronic equipment and storage medium

Abstract

The invention discloses a parallel processor cluster, a topology switching method, electronic equipment and a storage medium, and is applied to the field of multiprocessor systems. The parallel processor cluster comprises M parallel processors and N optical circuit switches, N ports on each parallel processor are coupled with the N optical circuit switches in a one-to-one correspondence mode, the total number of the ports of each parallel processor is larger than M, N is smaller than M. A topological structure of the parallel processor cluster comprises a first topological structure and a second topological structure, the parallel processor cluster supports switching of a first topological structure and a second topological structure, in the first topological structure, M parallel processors are divided into L groups, each group comprises M / L parallel processors, the M / L parallel processors are fully connected, in the second topological structure, the M parallel processors are fully connected, and M, N and L are positive integers. The parallel processor cluster can adapt to different network models, can achieve better performance in the different network models, and gives full play to hardware resource performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure apply to the field of multi-processor systems, and particularly relate to a parallel processor cluster, a topology switching method of the parallel processor cluster, an electronic device, and a non-transitory computer-readable storage medium. BACKGROUND

[0002] With the rapid development of cloud computing, artificial intelligence and big data applications, modern data centers are expanding in scale, and the requirements for network connection are also increasing. For different network models, such as Transformer models or MOE models, the bandwidth, delay, and algorithm design in the processor cluster are strongly related to the topology. When the cluster is applied to different network models, the fixed hardware connection makes the topology only applicable to the corresponding model, and in other models, the performance is poor. Using traditional electrical switching, the cache, routing and other modules in the electrical switching mode will bring delay, affect the communication bandwidth, and further affect the running performance of large models. SUMMARY

[0003] At least one embodiment of the present application provides a parallel processor cluster, which includes M parallel processors and N optical circuit switches. The N ports on each parallel processor are one-to-one coupled to the N optical circuit switches. The total number of ports of each parallel processor is greater than M, and N is less than M. The parallel processor cluster supports switching between a first topology structure and a second topology structure. When switching the topology structure, the parallel processor cluster is configured to switch between the first topology structure and the second topology structure through the N optical circuit switches according to the computing task performed by the parallel processor cluster. In the first topology structure, the M parallel processors are divided into L groups, each group including M / L parallel processors. The M / L parallel processors are fully connected, and each group includes M / L parallel processors that are respectively connected to different M / L parallel processors in other groups. In the second topology structure, the M parallel processors are fully connected. M, N, and L are positive integers.

[0004] For example, in the parallel processor cluster provided by at least one embodiment of the present application, the ports of the parallel processors connected by the N ports of each parallel processor are different in the first topology structure and the second topology structure. The N optical circuit switches are configured to establish a communication connection relationship between the N ports of the M parallel processors according to a topology configuration file to form the first topology structure or the second topology structure.

[0005] For example, in the parallel processor cluster provided in at least one embodiment of the present application, the connection relationship of other ports of each parallel processor except the N ports is the same in the first topology structure and the second topology structure, and the other ports of each parallel processor establish a communication connection relationship by using a communication cable.

[0006] For example, in the parallel processor cluster provided in at least one embodiment of the present application, each optical circuit switch is coupled with the ports of the same port number on the M parallel processors by using a communication cable.

[0007] For example, in the parallel processor cluster provided in at least one embodiment of the present application, each parallel processor includes A chip cores, A is a positive integer greater than 1, in the first topology structure, the M / L parallel processors are fully connected, including the i-th chip core of the M / L parallel processors are fully connected, i is a positive integer and sequentially takes 1 to A, and the M / L parallel processors and the M / L parallel processors included in each of the other groups are respectively connected, including the i-th chip core of the M / L parallel processors and the i-th chip core of the M / L parallel processors included in each of the other groups are respectively connected; in the second topology structure, the M parallel processors are fully connected, including one chip core of each parallel processor and the corresponding chip core in the other M-1 parallel processors are connected.

[0008] For example, in the parallel processor cluster provided in at least one embodiment of the present application, at least one of the A chip cores of each parallel processor in each group has a communication connection relationship with the corresponding chip core of the A chip cores of all other parallel processors in the group through a communication cable, and each of the M / L parallel processors is also connected with one parallel processor included in the M / L parallel processors of each of the other groups through the communication cable, and the communication connection includes at least one of the A chip cores of the parallel processor and the corresponding chip core of the A chip cores of the one parallel processor.

[0009] For example, in the parallel processor cluster provided by at least one embodiment of the present application, in the second topological structure, each parallel processor establishes a communication connection with at least one parallel processor in the other group through the N optical circuit switches, wherein the at least one parallel processor is a parallel processor in the other group that does not have a communication connection relationship with the first chip core and the second chip core of the parallel processor.

[0010] For example, in the parallel processor cluster provided by at least one embodiment of the present application, in response to A=2, L=2, each parallel processor includes two chip cores, the two chip cores include a first chip core and a second chip core, the first chip core of each parallel processor is in communication connection with the first chip core of T1 parallel processors in the group through a communication cable, the second chip core of each parallel processor is in communication connection with the second chip core of T2 parallel processors in the group through a communication cable, and the first chip core of each parallel processor is also in communication connection with the first chip core of a corresponding parallel processor in another group through a communication cable, wherein T1 and T2 are positive integers, T1+T2=M / L-1, and the T1 parallel processors are different from the T2 parallel processors.

[0011] For example, in the parallel processor cluster provided by at least one embodiment of the present application, in the first topological structure, the first chip core of each parallel processor establishes a communication connection with the first chip core of M / 2-T1-1 parallel processors in the group except the T1 parallel processors through the N optical circuit switches, the second chip core of each parallel processor establishes a communication connection with the second chip core of other parallel processors in the group except the T2 parallel processors through the N optical circuit switches, and the second chip core of each parallel processor also establishes a communication connection with the second chip core of the corresponding parallel processor in the other group through the N optical circuit switches.

[0012] For example, in the parallel processor cluster provided by at least one embodiment of the present application, in the second topological structure, each parallel processor establishes a communication connection with at least one parallel processor in the other group through the N optical circuit switches, wherein the at least one parallel processor is a parallel processor in the other group that does not have a communication connection relationship with the first chip core and the second chip core of the parallel processor.

[0013] For example, in the parallel processor cluster provided in at least one embodiment of the present application, the parallel processor cluster performs switching between the first topology and the second topology according to a computing task performed by the parallel processor cluster through the N optical circuit switches, including performing the following operations: in response to the computing task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is inferred or trained, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated; in response to the computing task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein when the second model is inferred or trained, data is exchanged between the M parallel processors in a two-way manner.

[0014] For example, in the parallel processor cluster provided in at least one embodiment of the present application, the communication operator of the first model includes an operator that communicates through tensor parallelism or sequence parallelism, and the communication operator includes a ring reduction operator, a full collection operator, and a scatter operator.

[0015] For example, in the parallel processor cluster provided in at least one embodiment of the present application, wherein the communication operator of the second model includes an operator that sends data to all other parallel processors while receiving data from all other parallel processors, and the communication operator includes a full-to-full operator.

[0016] For example, in the parallel processor cluster provided in at least one embodiment of the present application, the hardware communication bandwidth between the N ports on each parallel processor and the N optical circuit switches is consistent with the bandwidth of the communication cable, the N ports on each parallel processor are coupled to the N optical circuit switches through the communication cable, and the data transmission bandwidth of the communication connection between the M parallel processors is consistent.

[0017] The topology switching method of the parallel processor cluster provided in at least one embodiment of the present application includes: switching a topology structure of the parallel processor cluster through the N optical circuit switches according to a computing task performed by the parallel processor cluster, wherein the topology structure of the parallel processor cluster includes a first topology structure and a second topology structure; in the first topology structure, the M parallel processors are divided into L groups, each group includes M / L parallel processors, the M / L parallel processors are fully connected, and the M / L parallel processors are respectively connected to the M / L parallel processors included in each other group, the M / L parallel processors are respectively connected to different M / L parallel processors in each other group, and in the second topology structure, the M parallel processors are fully connected, and M, N, and L are positive integers.

[0018] For example, in the topology switching method provided in at least one embodiment of the present application, the switching of the topology structure of the parallel processor cluster through the N optical circuit switches according to the computing task performed by the parallel processor cluster includes: in response to the computing task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology structure, wherein when the first model is inferred or trained, data is split by dimension to different parallel processors for distributed computing and the computing results are aggregated; and in response to the computing task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology structure, wherein when the second model is inferred or trained, the M parallel processors exchange data bidirectionally in pairs.

[0019] The electronic device provided in at least one embodiment of the present application includes: a memory that non-transiently stores computer executable instructions; and a processor configured to run the computer executable instructions, wherein the computer executable instructions, when run by the processor, implement the topology switching method according to any embodiment of the present application.

[0020] The present application provides a non-transitory computer readable storage medium, wherein the non-transitory computer readable storage medium stores computer executable instructions, and the computer executable instructions, when executed by a processor, implement the topology switching method according to any embodiment of the present application.

[0021] The parallel processor cluster provided by the application can perform topology switching at a driver layer, without changing hardware wiring, so that the parallel processor cluster can adapt to different computing tasks, and good performance can be achieved in network models corresponding to different computing tasks, and hardware resource performance can be fully utilized. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only related to some embodiments of the present disclosure, and not limit the present disclosure.

[0023] Figure 1 A structural schematic diagram of a MoE layer;

[0024] Figure 2 A schematic block diagram of a cluster topology structure provided by at least one embodiment of the present disclosure;

[0025] Figure 3 A schematic structural diagram of a general-purpose graphics processor;

[0026] Figure 4A A schematic diagram of a first topology structure provided by an embodiment of the present disclosure;

[0027] Figure 4B A schematic diagram of a second topology structure provided by an embodiment of the present disclosure;

[0028] Figure 5 A schematic diagram of a communication cable of a parallel processor cluster provided by at least one embodiment of the present disclosure;

[0029] Figure 6 A connection relationship schematic diagram of a first topology structure provided by at least one embodiment of the present disclosure;

[0030] Figure 7 A connection relationship schematic diagram of a second topology structure provided by at least one embodiment of the present disclosure;

[0031] Figure 8 A schematic flowchart of a topology switching method of a parallel processor cluster provided by at least one embodiment of the present disclosure;

[0032] Figure 9 A schematic diagram of a non-transitory computer readable storage medium provided by at least one embodiment of the present disclosure;

[0033] Figure 10 A schematic block diagram of another electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0034] In order to make the purposes, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions of the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings of the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all the embodiments. Based on the described embodiments of the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without any creative effort fall within the protection scope of the present disclosure.

[0035] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning understood by a person of ordinary skill in the art to which the present disclosure pertains. The terms “first”, “second” and similar terms used in the present disclosure do not denote any order, quantity or importance, but are only used to distinguish different constituent parts. The terms “include” or “contain” and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, without excluding other elements or objects. The terms “connect” or “connected” and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms “up”, “down”, “left”, “right” and the like only represent relative positional relationships, which can change when the absolute positions of the described objects change. In order to keep the following description of the embodiments of the present disclosure clear and concise, the present disclosure omits the detailed description of some known functions and known components.

[0036] The Transformer model is a classic neural network-based text processing model, which has given birth to important models such as BERT (Bidirectional Encoder Representation from Transformers), GPT (Generative Pre-trained Transformer), LLM (Large Language Model) that have greatly promoted the development of the NLP (Natural Language Processing) field.

[0037] With GPT (Generative Pre-trained Transformer) and LLAMA (Large Language Model Meta AI) as representatives, the core structure is a multi-layer stacked Transformer decoder (Decoder) block (generative model), each decoder block contains two major components: multi-head self-attention mechanism (Multi-Head Attention) for calculating the dependency between tokens, and feed-forward neural network (Feed-Forward Network, FFN) for nonlinear transformation of each token's features to enhance the model's expression ability. The calculation of these two components relies on huge tensors, such as weight matrices with dimensions often reaching thousands of dimensions, for example, the hidden layer dimension of LLAMA 27b is 4096.

[0038] When training network models such as GPT and LLAMA, tensor parallelism (TensorParallel) is usually used to split tensors, and the model's tensors (such as weight matrices, input and output feature maps, and other multi-dimensional data) are split onto multiple devices (GPU / TPU), each device only holds part of the tensor and performs local computation, and finally the results are aggregated through cross-device communication to achieve parallel acceleration.

[0039] For network models such as GPT and LLAMA, there are a large number of tensor parallel communications, such as using ring all-reduce operators and other communication operators for tensor synchronization between devices, using all-gather operators to collect and aggregate tensors from each device, etc. Tensor parallelism has clear structured features, communication data is split tensor fragments, which need to be aggregated or split by dimensions, rather than random pairwise data exchange, in addition, communication scenarios focus on layer-wide collaboration, such as All-Gather (all-gather) operators to obtain complete tensors, Reduce-Scatter (reduce-scatter) operators to split results, etc., rather than point-to-point independent communication.

[0040] Therefore, for network models such as GPT and LLAMA, a network topology that is fully connected within a group, connected two by two between groups, and has consistent communication bandwidth on the links within and between groups is more suitable. In this case, the performance of GPT and LLAMA is best. This is because the communication essence of tensor parallelism is global cooperation, that is, all GPUs converge data to the same GPU, or GPUs distribute data to all GPUs. For example, the role of the ring all-reduce operator is to make data transmit and splice in the ring link segment by segment, and the throughput of the whole process is determined by the slowest link in the ring. This topology can make the communication operator fully utilize all link bandwidth and achieve the highest theoretical performance.

[0041] The MOE structure is another very important structure in the Transformer model, which makes the model perform better by assigning different tokens (also known as token feature vectors) to appropriate experts.

[0042] Figure 1 A schematic diagram of a MoE layer.

[0043] As shown in Figure 1 , the MoE layer includes multiple expert networks and a routing module.

[0044] Expert networks are the core computing units in the MoE layer. Each expert network is an independent neural network, usually in the form of a feedforward network (FFN). Each expert network is designed to focus on processing specific tasks or data features. For example, in natural language processing, some expert networks may focus on syntax analysis, while others focus on semantic understanding.

[0045] The role of the routing module is to determine which expert networks each input token should be sent to for processing. For example, the number of expert networks to which each token is sent can be pre-set, such as 2, 4, 6, 8, etc.

[0046] In the MoE model, input tokens are assigned to a small number of suitable experts (such as 256 experts activating only 8) by the routing module, and these experts are often distributed on different GPUs. At this time, the alltoall operator is needed to distribute the tokens from the GPU where they are located to the GPU of the corresponding expert, and after the calculation is completed, the results are aggregated back to the original GPU.

[0047] Therefore, for the training of MoE models, in the case of full connection between GPUs, the scheduling time of tokens can be greatly shortened.

[0048] The two network models above are currently commonly used large models, but due to the different structures of the two models, their requirements for the topology of the processor cluster are completely different. However, when the hardware connection is fixed, the topology is only applicable to the corresponding model and has poor performance in other models.

[0049] For example, if the full-connection topology between processors is applied to network models such as GPT and LLAMA, due to the existence of a large number of tensor parallel communications in network models such as GPT and LLAMA, a large number of GPUs may simultaneously initiate communication, causing serious link conflicts. Moreover, the essence of tensor parallel communication is local tensor aggregation or splitting, and the full-connection redundant links cannot improve the efficiency of such global communication, but rather cause bandwidth dispersion and reduce performance due to excessive links. Tensor parallel communication has a structured feature, for example, after splitting according to the tensor dimension, only specific GPU-to-GPU communication of the corresponding dimension data is required. The majority of time, the large number of redundant links in full-connection are in an idle state, resulting in waste of bandwidth resources.

[0050] For example, if the topology suitable for network models such as GPT and LLAMA is applied to the MoE model, due to the need for data transmission to pass through a certain processor during the data transmission process, the communication delay will be greatly increased. In this topology, the parallel processors are not fully connected, resulting in a high communication delay for non-directly connected links when using, for example, an alltoall operator, forming a low-bandwidth bottleneck, so that the high-bandwidth direct connection cannot be utilized, and the overall performance is reduced, affecting the inference and training efficiency of the MoE model and failing to fully utilize the hardware performance.

[0051] At least one embodiment of the present disclosure provides a parallel processor cluster capable of topology switching and a topology switching method of the parallel processor cluster. The parallel processor cluster includes M parallel processors and N optical circuit switches. The N ports on each parallel processor are one-to-one coupled to the N optical circuit switches. The total number of ports of each parallel processor is greater than M, N is less than M, and the topology structure of the parallel processor cluster supports switching between a first topology structure and a second topology structure. The parallel processor cluster is configured to switch between the first topology structure and the second topology structure through the N optical circuit switches according to a computing task performed by the parallel processor cluster. In the first topology structure, the M parallel processors are divided into L groups, each group including M / L parallel processors. The M / L parallel processors are fully connected, and each group of M / L parallel processors is respectively connected to each group of M / L parallel processors. Each group of M / L parallel processors is connected to different groups of M / L parallel processors. In the second topology structure, the M parallel processors are fully connected. M, N, and L are positive integers.

[0052] The parallel processor cluster provided by at least one embodiment of the present disclosure can perform topology switching at a driver layer, without changing hardware wiring, so that the parallel processor cluster can adapt to different computing tasks and achieve good performance in network models corresponding to different computing tasks, and hardware resource performance is fully utilized.

[0053] The embodiments of the present disclosure are described in detail below with reference to the drawings, but the present disclosure is not limited to these specific embodiments.

[0054] Figure 2 A schematic block diagram of the cluster topology structure provided by at least one embodiment of the present disclosure is shown.

[0055] As Figure 2 shown, the parallel processor cluster 100 includes M parallel processors 101 and N optical circuit switches 102, and N ports on each parallel processor are coupled to N optical circuit switches. It should be noted that, for the sake of clarity, Figure 2 the connection relationship between the parallel processor and the optical circuit switch is shown, and the connection relationship between the parallel processor and the electrical switch and the central processor is shown, but the connection relationship between the parallel processors is not shown. For the connection relationship between the parallel processors, refer to the detailed description below.

[0056] For example, the M parallel processors 101 can belong to one node in the parallel processor cluster, and other nodes with the same structure can also be provided in the parallel processor cluster.

[0057] The optical circuit switch (OCS) is a dedicated network switching device based on optical layer transmission, and the core function is to directly establish or disconnect the physical optical path connection between the fiber ports through the optical switch device to realize the transparent transmission of optical signals without the optical-electric or electric-optical conversion link.

[0058] In the present disclosure, coupling refers to the establishment of an association relationship between two or more components that can transfer energy and signals, and coupling includes electrical connection or optical coupling, for example. Communication connection refers to the establishment of a direct communication relationship between two parallel processors through a direct communication cable or a communication cable + optical circuit switch, without the need to pass through other parallel processors to forward data.

[0059] For example, the parallel processor establishes a communication connection with another parallel processor through the optical circuit switch, and the communication connection includes both optical signal transmission based on the optical circuit switch and signal transmission through the communication cable; the parallel processor can also directly establish a communication connection with another parallel processor through the communication cable, and the communication connection can be achieved by directly connecting the ports of the two parallel processors through the communication cable. For example, the communication cable can be implemented by using an optical cable, a copper cable, etc.

[0060] For example, the parallel processor in the parallel processor cluster provided by at least one embodiment of the present disclosure includes a graphic processing unit (GPU), a general-purpose graphic processing unit (GPGPU), a tensor processing unit (TPU), a data processing unit (DPU), a neural processing unit (NPU), etc.

[0061] For example, the parallel processor cluster can be a multi-card (multi-GPU) system that integrates multiple GPU cards, an architecture that cooperates to complete large-scale parallel computing tasks, and is commonly used in, for example, artificial intelligence training, scientific computing, high-performance graphics rendering, etc.

[0062] For example, in some embodiments, the parallel processor cluster provided by at least one embodiment of the present disclosure can be in the form of a single machine multi-card, integrating multiple GPU cards in a single server or workstation, and realizing card-to-card communication through a bus (such as a PCIe bus, an AMD bus, etc.). For example, the parallel processor cluster can also be a multi-machine multi-card, multiple single-machine multi-card servers interconnected through a high-speed network to form a cluster, and each card can not only communicate with other cards on the local machine, but also interact with cards in other groups through the network. For example, the parallel processor cluster can also be a TPU cluster, a multi-FPGA acceleration card system, etc., which is not specifically limited by the present disclosure.

[0063] For example, the parallel processor cluster breaks through the single-card computing power or memory limit through parallel computing, and is therefore widely used in scenarios that require large-scale data processing, high-throughput computing, or low-latency parallel tasks. For example, deep learning and large model training, high-performance computing, large-scale data processing and AI inference, real-time rendering and visualization, etc., can be applied to large model training, scientific computing, high-concurrency AI services, real-time rendering, etc.

[0064] Figure 3 A schematic structural diagram of a general-purpose graphic processing unit. For example, when the parallel processor is a graphic processing unit or a general-purpose graphic processing unit, its specific structure can refer to the diagram shown in Figure 3 .

[0065] As shown in Figure 3 , the general-purpose graphic processing unit is actually an array of programmable multi-processors, for example, the programmable multi-processors can be a streaming processor cluster (SPC), which includes, for example, Figure 3The diagram shows multiple streaming processor clusters, all with identical structures. In a general-purpose graphics processor, one streaming processor cluster handles one computational task, or multiple streaming processor clusters handle one computational task. Multiple streaming processor clusters share data through a global cache or global memory.

[0066] like Figure 3 As shown, a streaming processor cluster comprises multiple compute units (CUs), all with identical structures. Each CU performs arithmetic and logical operations, such as accumulation, reduction, and standard addition, subtraction, multiplication, and division. A CU contains multiple cores, each including an arithmetic logic unit (ALU), a floating-point unit, etc., which execute specific computational tasks. Furthermore, a CU also includes registers (e.g.,...). Figure 3 The register file and shared memory in a computing unit are used to store source and destination data related to computing tasks in a hierarchical manner. The shared memory in a computing unit is used to share data between the cores of that computing unit.

[0067] like Figure 3 As shown, each computing unit also provides a tensor core for performing tensor-related computations, such as tensor shrinking operations. Tensor cores can accelerate tensor operations such as matrix multiplication. Tensor cores in multiple computing units can be scheduled and controlled uniformly.

[0068] like Figure 3 As shown, each streaming processor cluster also provides a buffer for data caching of multiple computing units within the streaming processor cluster.

[0069] In parallel computing, computational tasks are typically executed by multiple threads. These threads are divided into multiple thread blocks before execution in a general-purpose graphics processor (or parallel computing processor), and then dispatched via a thread block distribution module. Figure 3 (Not shown in the image) Multiple thread blocks are distributed to various computation units. All threads in a thread block must be assigned to the same computation unit for execution. Simultaneously, thread blocks are broken down into minimum execution thread bundles (or simply warps), each containing a fixed number (or less than this fixed number) of threads, for example, 32 threads. Multiple thread blocks can execute in the same computation unit or in different computation units.

[0070] In each computing unit, the thread beam scheduling / distribution module ( Figure 3The thread bundles are scheduled and allocated to the plurality of computing cores of the computing unit to run the thread bundles. According to the number of computing cores in the computing unit, the plurality of thread bundles in one thread block can be executed simultaneously or in time-sharing manner. The plurality of threads in each thread bundle execute the same instruction. The memory execution instruction is transmitted to the shared memory in the computing unit or further transmitted to the intermediate level cache or the global cache or the global memory (for example, High Bandwidth Memory, HBM) to perform read / write operation, etc. Figure 2

[0071] For one parallel processor 101, it includes a plurality of ports, the total number of the plurality of ports is greater than M, so as to realize the second topology structure as described below, and N ports of the plurality of ports are connected in one-to-one correspondence with N optical circuit switches 102. As shown in Figure 2 , the N ports of each parallel processor are connected with the N optical circuit switches respectively.

[0072] The connection between the port of the parallel processor and the optical circuit switch can be realized through a communication cable, which is a physical communication connection. The communication cable is fixed and unchangeable. The communication cable can use hardware level interconnection technology dedicated to, for example, GPU cluster, and can use a communication cable optimized for high bandwidth and adapted to the deployment requirements of the parallel processor cluster of large data center, and the present disclosure does not make specific limitation thereon.

[0073] The topology structure of the parallel processor cluster 100 includes the first topology structure and the second topology structure, and the parallel processor cluster 100 supports switching between the first topology structure and the second topology structure. Of course, the topology structure of the parallel processor cluster 100 can also include more topology structures, and the parallel processor cluster 100 can also support switching of these topology structures, the logic of which is similar to that of the first topology structure and the second topology structure, which will not be described here.

[0074] When switching the topology structure, the parallel processor cluster 100 is configured to switch between the first topology structure and the second topology structure through the N optical circuit switches according to the computing task performed by the parallel processor cluster.

[0075] In addition, as shown in Figure 4A , the parallel processor cluster 100 further includes a plurality of electrical switches and a plurality of central processors, each central processor is in communication connection with P electrical switches, and each electrical switch is in communication connection with Q parallel processors.

[0076] ​Each of the electrical switches is configured to forward data flow between the central processor and the Q parallel processors, and each of the central processors is configured to schedule, coordinate and control P*T parallel processors connected to the central processor through the P electrical switches. Here, P, Q and T are positive integers, and can be set according to the needs of cluster deployment.

[0077] For example, the electrical switches can be PCIe switches, and of course, the present disclosure is not limited thereto, and other structures of electrical switches can also be used.

[0078] In the first topology, M parallel processors are divided into L groups, each group including M / L parallel processors, and the M / L parallel processors in each group are fully connected, and the M / L parallel processors in each group are connected to the M / L parallel processors in each of the other groups in pairs, and the M / L parallel processors in each group are connected to different M / L parallel processors in each of the other groups. M, N and L are positive integers.

[0079] Figure 4A A schematic diagram of the first topology provided by an embodiment of the present disclosure is shown.

[0080] In the example shown in Figure 4A , M=8 and L=2, and it should be noted that the present disclosure is not limited thereto, and only the first topology in a possible embodiment is described by way of example. Figure 4A

[0081] In the first topology shown in Figure 4A , two groups are included, each group including four parallel processors, and the four parallel processors in each group are fully connected, that is, each two parallel processors in the group have a communication connection relationship, which is a direct communication relationship and can be realized through a communication cable or a communication cable + an optical electrical switch. In addition, for the four parallel processors in each group, each parallel processor has a connection relationship with a corresponding parallel processor in the other group, and the four parallel processors are one-to-one connected to the four parallel processors in the other group.

[0082] In addition, in Figure 4B , a central processor and electrical switches are also shown, and as described above, the central processor is used to schedule, coordinate and control four parallel processors connected to the central processor through two electrical switches, and each electrical switch is used to forward data flow between the connected two parallel processors and the central processor.

[0083] In the second topology, the M parallel processors are fully connected.

[0084] Figure 4B ​A schematic diagram of the second topology provided by an embodiment of the present disclosure.

[0085] In Figure 4B In an example, M=4, it should be noted that the present disclosure is not limited thereto.

[0086] As Figure 5 indicated, all the 4 parallel processors are fully connected, that is, each two parallel processors have a direct communication connection relationship.

[0087] For example, the number of N is the same as the number of ports whose connection relationship is changed in the two topologies. Here, the connection relationship refers to the connection relationship between the ports of the parallel processors, but physically, they are still connected to the optical circuit switch, and the port object connected through the optical circuit switch changes.

[0088] For example, the N ports of each parallel processor are connected to different ports of other processors through the optical circuit switch, that is, the N ports whose connection relationship is changed in the first topology and the second topology, and the number of N is determined according to the first topology and the second topology. The N optical circuit switches are configured to establish a communication connection relationship between the N ports of each of the M parallel processors according to the topology configuration file, so as to form the first topology or the second topology.

[0089] For example, in the first topology, the N ports of each parallel processor are connected to the ports of other parallel processors according to a first pairing relationship, and in the second topology, the N ports of each parallel processor are connected to the ports of other parallel processors according to a second pairing relationship, and the first pairing relationship and the second pairing relationship are completely different.

[0090] For example, the connection relationship of the other ports of each parallel processor except the N ports is the same in the first topology and the second topology, and the other ports of each parallel processor establish a communication connection relationship by using a communication cable.

[0091] Specifically, by comparing the port connection relationship of each parallel processor in the first topology and the second topology, for the ports that remain unchanged in the two topologies, a communication cable is used to establish a communication connection relationship, and for the ports whose connection relationship changes in the two topologies, an N optical circuit switch is used to establish a communication connection relationship. Therefore, when topology switching occurs, for the ports whose connection relationship does not change, a communication cable is used for communication, and for the ports whose connection relationship changes, an optical circuit switch is used to realize the topology switching.

[0092] Similar logic can also be used when the parallel processor cluster supports more topologies. The connections shared by the topologies are connected by communication cables, and other ports are connected by optical circuit switches to realize topology switching through optical circuit switches and reduce the number of optical circuit switches as much as possible to reduce the wiring cost.

[0093] For example, the optical circuit switch sets up a path through the topology configuration file, converts the user-defined optical path connection rule into a port mapping instruction of the switch hardware, and realizes physical path establishment or release on the optical layer.

[0094] Thus, the optical circuit switch is used to perform topology switching in the driver layer to realize software-defined dynamic reconstruction of the communication topology of the parallel processor cluster. Without any physical hardware rewiring operation, the cluster topology mode can be flexibly switched according to the communication characteristics of different computing tasks, so that excellent performance can be achieved in diversified computing models. This topology dynamic switching mechanism breaks the limitation of the traditional hardware topology that is fixed after deployment, so that the parallel processor cluster can adapt to the differentiated needs of various models such as training, inference, and scientific computing. Under the premise of controllable hardware deployment cost, the optimal solution for all scenarios is realized.

[0095] For example, in some embodiments, the parallel processor cluster switches between the first topology and the second topology through N optical circuit switches according to the computing task performed by the parallel processor cluster, including performing the following operations: in response to the computing task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is inferred or trained, data is split by dimension to different parallel processors for distributed computing and the computing results are aggregated; in response to the computing task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein when the second model is inferred or trained, two-way data exchange is performed between M parallel processors.

[0096] For example, the communication operator of the first model includes an operator that communicates through tensor parallelism or sequence parallelism, and the communication operator includes a ring reduction operator, an all-gather operator, a scatter operator, and the like. The ring reduction operator, the all-gather operator, and the scatter operator are set communication operators, the ring reduction operator (Ring Reduce) is a distributed parallel processor that sequentially transmits and aggregates data in a ring topology, and finally all parallel processors obtain a global aggregation result (such as summation, average calculation), the all-gather operator (All-gather) is an operator that broadcasts local data to all other parallel processors, and finally all parallel processors hold a complete global data set, and the scatter operator (Scatter) is an operator that distributes global data to each child node according to a predetermined rule after the global data is fragmented by the root node, and the scatter operator includes a reduce scatter operator (ReduceScatter) and the like.

[0097] For example, the first model can be a network model that includes more or a large number of tensor parallelism or sequence parallelism communications, such as a traditional Transformer model such as GPT, LLAMA, and the like. Of course, the present disclosure does not limit the specific network model, and for a network model that has similar communication requirements as the traditional Transformer model such as GPT, LLAMA, and the like, the first topology structure provided by at least one embodiment of the present disclosure can be applied.

[0098] In the first topology structure, the network model that has a communication requirement such as tensor parallelism or sequence parallelism can fully exert the performance advantage of the hardware during inference or training. As described above, the communication mode of splitting data by dimension to different parallel processors for distributed calculation and aggregating the calculation results is essentially global collaboration, and all GPUs converge data to the same GPU or distribute data to all GPUs, and the communication has a structured feature, and the transmission of corresponding dimension data between specific GPUs can fully utilize all bandwidths on the link and exert the best performance advantage.

[0099] For example, the hardware communication bandwidth between the N ports on each parallel processor and the N optical circuit switches and the bandwidth of the communication cable are consistent. The N ports on each parallel processor are coupled to the N optical circuit switches through the communication cable, and the data transmission bandwidth of the communication connection between the M parallel processors is consistent. The delay of the optical path transmission is extremely small and can be ignored. The bandwidth of the communication cable connected between the parallel processor and the optical circuit switch is the same as the bandwidth of the communication cable between the parallel processors, so that the link bandwidth of the parallel processor can be guaranteed to be consistent whether in the first topology structure or in the second topology structure, the entire link speed is matched, and the operator can fully utilize all bandwidths on the link when executing, avoid some low-bandwidth links causing bottlenecks, and achieve the theoretical highest performance.

[0100] For example, the communication operator of the second model includes an operator that sends data to all other parallel processors while receiving data from all other parallel processors, and the communication operator includes an all-to-all operator.

[0101] For example, the second model can be a MoE model, or other network models that have more or a large number of, for example, all-to-all operator communications. Such models need to send data from the GPU where they are located to any possible GPU, and then aggregate the results back to the original GPU after the calculation is completed. Such models can greatly shorten the scheduling time of tokens because there is a possibility that any two parallel processors need to communicate, and in the case of full connection between parallel processors, the scheduling time of tokens can be greatly shortened.

[0102] For example, each optical circuit switch is in communication connection with the ports of the same port number on the M parallel processors through a communication cable, for example, one optical circuit switch is in communication connection with the 2nd port on the M parallel processors through a communication cable, and another optical circuit switch is in communication connection with the 4th port on the M parallel processors through a communication cable.

[0103] When wiring the parallel processor cluster, a symmetrical wiring method is often used, so that the links of each parallel processor to the network core, load, or other parallel processors have consistent physical and transmission characteristics. Each optical circuit switch is in communication connection with the ports of the same port number on the M parallel processors, and the same port number ports can be turned on through the optical circuit switch, thereby maintaining symmetrical wiring and simplifying the topology design difficulty of the parallel processor cluster.

[0104] For example, in some embodiments, the parallel processor includes one chip core.

[0105] For example, in some other embodiments, each parallel processor includes A chip cores, and A is a positive integer greater than 1.

[0106] The chip core can be a die, which is an independent silicon chip containing complete specific functional circuits, and is the core carrier of the parallel processor computing power and functions. The parallel processor, for example, a graphics processor, can package multiple chip cores into one whole through Chiplet technology, greatly improving computing power and significantly improving processor performance.

[0107] In the case where each parallel processor comprises a plurality of chip cores, for the first topology, the M / L parallel processors are fully connected among each other, and the i-th chip core of the M / L parallel processors is fully connected among each other, i being a positive integer and sequentially taking 1 to A. For example, assuming that each parallel processor comprises two chip cores, i.e. a first chip core and a second chip core, the i-th chip core of the M / L parallel processors means that the first chip cores of the M / L parallel processors in the same group are fully connected among each other, and the second chip cores of the M / L parallel processors in the same group are also fully connected among each other.

[0108] For example, the first chip core and the second chip core are independent in structure or function, and can perform a computing task as two independent chip cores.

[0109] For the first topology, the M / L parallel processors in each group are respectively communicatively connected with the M / L parallel processors in each other group, and the i-th chip core of the M / L parallel processors in each group is respectively communicatively connected with the i-th chip core of the M / L parallel processors in each other group. For example, assuming that each parallel processor comprises two chip cores, i.e. a first chip core and a second chip core, for each parallel processor in the M / L parallel processors of a certain group, the first chip core of the parallel processor is communicatively connected with the first chip core of a corresponding parallel processor in the M / L parallel processors of each other group, and the second chip core of the parallel processor is communicatively connected with the second chip core of the corresponding parallel processor in the M / L parallel processors of each other group.

[0110] For example, for the first topology, the plurality of parallel processors in any one group are communicatively connected with the plurality of parallel processors in another group in a one-to-one correspondence, each parallel processor in a group is communicatively connected with a parallel processor in another group, different parallel processors in a group are communicatively connected with different parallel processors in another group, and in the first topology, the communicative connection between two parallel processors comprises that each chip core of the two parallel processors is communicatively connected.

[0111] In the case where each parallel processor comprises a plurality of chip cores, for the second topology, the M parallel processors are fully connected among each other, and one chip core of each parallel processor is communicatively connected with a corresponding chip core in the other M-1 parallel processors. That is, for the case of the second topology, as long as a certain chip core of two parallel processors has a direct communicative connection relationship, the two parallel processors are considered to be communicatively connected.

[0112] The hardware connection relationship of the parallel processor cluster under the above two topologies will be described in detail below.

[0113] For example, in some embodiments, for the first topology and the second topology, the communication cable connections whose port connection relationship does not change at the time of switching are adopted, and the communication connections are established through the optical circuit switch whose port connection relationship changes, so that the least number of optical circuit switch modules can be arranged to realize topology switching, and the cluster deployment cost is reduced.

[0114] For example, with respect to the fixed communication cable, in some embodiments, at least one of the A chip cores of each parallel processor in each group has a communication connection relationship with the corresponding chip core of all other parallel processors in the group, and each of the M / L parallel processors is also in communication connection with one parallel processor included in the other group through the communication cable, and the communication connection includes that at least one of the A chip cores of the parallel processor is in communication connection with the corresponding chip core of the A chip cores of the one parallel processor.

[0115] It should be noted that in the above description, groups are used for description for the purpose of positioning parallel processors and connected objects, and the groups represent the relationship in the first topology, and do not constitute a physical limitation for the scenario of the second topology, for example, in the second topology, each parallel processor is still regarded as an independent individual, and a full connection relationship is generated.

[0116] Figure 5 A schematic diagram of the communication cable of the parallel processor cluster provided by at least one embodiment of the present disclosure.

[0117] In the example of Figure 5 The parallel processor includes two chip cores, which are a first chip core and a second chip core, respectively, and each chip core includes five ports. The first chip core includes ports 1 to 5, and the second chip core includes ports 6 to 10. Of course, the present disclosure is not limited thereto, and the present disclosure does not specifically limit the number of chip cores included in the parallel processor and the number of ports included in the chip core.

[0118] Figure 5 Eight parallel processors are shown in the example of

[0119] For example, Figure 5-7 The example of Figure 5 The example of

[0120] For example, in the example of Figure 2In the example shown in FIG. 1, for the first topology, parallel processor 101_0, parallel processor 101_1, parallel processor 101_2, and parallel processor 101_3 belong to one group, and parallel processor 101_4, parallel processor 101_5, parallel processor 101_6, and parallel processor 101_7 belong to another group. Of course, the parallel processor cluster can include more groups, and the present disclosure does not make specific limitations.

[0121] In addition, the parallel processor cluster can also include a central processor and an electrical switch, and the specific connection relationship can refer to the foregoing Figure 4A and Figure 5 related descriptions, which are not shown in the Figure 5 any more. For example, the port 1 of each parallel processor is connected with the electrical switch for data exchange with the central processor.

[0122] For example, the first chip core of each parallel processor is in communication connection with the first chip core of T1 parallel processors in the group through a communication cable, the second chip core of each parallel processor is in communication connection with the second chip core of T2 parallel processors in the group through a communication cable, and the first chip core of each parallel processor is also in communication connection with the first chip core of the corresponding parallel processor in another group through a communication cable, wherein T1 and T2 are positive integers, T1+T2=M / L-1, and the T1 parallel processors are different from the T2 parallel processors. The communication cable can be a copper cable (electrical signal) or an optical cable (optical signal).

[0123] For multiple parallel processors in a group, as shown in Figure 5 , for example, the port 3 of the first chip core of parallel processor 101_0 is connected with the port 3 of the first chip core of parallel processor 101_2 through a communication cable to establish a direct communication connection relationship.

[0124] The port 7 of the second chip core of parallel processor 101_0 is connected with the port 7 of the second chip core of parallel processor 101_3 through a communication cable to establish a direct communication connection relationship; and the port 9 of the second chip core of parallel processor 101_0 is connected with the port 9 of the second chip core of parallel processor 101_1 through a communication cable to establish a direct communication connection relationship.

[0125] Of course, when more parallel processors are included in the group, communication connection relationships can also be established according to the above logic, which is not described herein again.

[0126] The connection relationship of other parallel processors with the parallel processors in the group is the same, which is not described herein again.

[0127] In addition, as shown in Figure 6The port 5 of the first chip core of the parallel processor 101_0 and the port 5 of the parallel processor 101_7 are connected through a communication cable to establish a direct communication connection relationship; the port 5 of the first chip core of the parallel processor 101_3 and the port 5 of the parallel processor 101_4 are connected through a communication cable to establish a direct communication connection relationship; the port 5 of the first chip core of the parallel processor 101_1 and the port 5 of the parallel processor 101_6 are connected through a communication cable to establish a direct communication connection relationship; and the port 5 of the first chip core of the parallel processor 101_2 and the port 5 of the parallel processor 101_5 are connected through a communication cable to establish a direct communication connection relationship.

[0128] When the computing task performed by the parallel processor cluster is an inference or training task of the first model, the N optical circuit switches are configured according to the first topology configuration file to switch the parallel processor cluster to the first topology structure.

[0129] For example, in the first topology structure, the chip cores in each parallel processor establish a communication connection with the target chip core through the N optical circuit switches, where the target chip core is a chip core that has not yet established a communication connection relationship with the A chip cores of the parallel processor according to the first topology structure. That is, for the connection relationship required by the first topology structure, those that have not established a communication connection through a communication cable establish a communication connection through the N optical circuit switches.

[0130] Figure 6 A connection relationship diagram of the first topology structure provided by at least one embodiment of the present disclosure is shown.

[0131] In the first topology structure, the parallel processor cluster includes N parallel processors, and each parallel processor includes A chip cores. Figure 5 In the first topology structure, the solid line represents the communication cable in the Figure 7 The dashed line represents the communication connection established through the optical circuit switch, and the box on each dashed line represents the number of the optical circuit switch.

[0132] For example, in this embodiment, four optical circuit switches are shown, which are respectively represented as optical circuit switch 102_0, optical circuit switch 102_1, optical circuit switch 102_2, and optical circuit switch 102_3 for description. The optical circuit switch 102_0 is connected to the port 2 of each parallel processor through a communication cable, the optical circuit switch 102_1 is connected to the port 4 of each parallel processor through a communication cable, the optical circuit switch 102_2 is connected to the port 6 of each parallel processor through a communication cable, and the optical circuit switch 102_3 is connected to the port 8 of each parallel processor through a communication cable.

[0133] In the first topology, the first chip core of each parallel processor establishes a communication connection with the first chip core of M / 2-T1-1 parallel processors in the group except for T1 parallel processors through N optical circuit switches, the second chip core of each parallel processor establishes a communication connection with the second chip core of other parallel processors in the group except for T2 parallel processors through N optical circuit switches, and the second chip core of each parallel processor also establishes a communication connection with the second chip core of the corresponding parallel processor in another group through N optical circuit switches.

[0134] Taking the parallel processor 101_0 as an example, according to the definition of the first topology, the first chip core of the parallel processor 101_0 also needs to establish a communication connection with the first chip core of the parallel processor 101_1 and the first chip core of the parallel processor 101_3.

[0135] Through the first topology configuration file, the optical circuit switch 102_0 is configured to turn on the port 2 of the parallel processor 101_0 and the port 2 of the parallel processor 101_1, thereby establishing a communication connection between the port 2 of the parallel processor 101_0 and the port 2 of the parallel processor 101_1. Moreover, through the first topology configuration file, the optical circuit switch 102_1 is configured to turn on the port 4 of the parallel processor 101_0 and the port 4 of the parallel processor 101_3, thereby establishing a communication connection between the port 4 of the parallel processor 101_0 and the port 4 of the parallel processor 101_3. In this way, the communication connection between the first chip core of the parallel processor 101_0 and the first chip core of all other parallel processors in the group is realized.

[0136] According to the definition of the first topology, the second chip core of the parallel processor 101_0 also needs to establish a communication connection with the second chip core of the parallel processor 101_2.

[0137] Through the first topology configuration file, the optical circuit switch 102_3 is configured to turn on the port 8 of the parallel processor 101_0 and the port 8 of the parallel processor 101_2, thereby establishing a communication connection between the port 8 of the parallel processor 101_0 and the port 8 of the parallel processor 101_2. In this way, the communication connection between the second chip core of the parallel processor 101_0 and the second chip core of all other parallel processors in the group is realized.

[0138] In addition, according to the definition of the first topology, the second chip core of the parallel processor 101_0 also needs to establish a communication connection with the second chip core of the parallel processor 101_7 in another group.

[0139] The first topology configuration file configures the optical circuit switch 102_2 to turn on the port 6 of the parallel processor 101_0 and the port 6 of the parallel processor 101_7, thereby establishing a communication connection between the port 6 of the parallel processor 101_0 and the port 6 of the parallel processor 101_7. Thus, the second chip core of the parallel processor 101_0 is connected to the second chip core of the corresponding parallel processor in the other group.

[0140] The parallel processor 101_4 is taken as an example for description. According to the definition of the first topology structure, the first chip core of the parallel processor 101_4 needs to be connected to the first chip core of the parallel processor 101_5 and the first chip core of the parallel processor 101_7.

[0141] The first topology configuration file configures the optical circuit switch 102_0 to turn on the port 2 of the parallel processor 101_4 and the port 2 of the parallel processor 101_5, thereby establishing a communication connection between the port 2 of the parallel processor 101_4 and the port 2 of the parallel processor 101_5. Moreover, the first topology configuration file configures the optical circuit switch 102_1 to turn on the port 4 of the parallel processor 101_4 and the port 4 of the parallel processor 101_7, thereby establishing a communication connection between the port 4 of the parallel processor 101_4 and the port 4 of the parallel processor 101_7. Thus, the first chip core of the parallel processor 101_4 is connected to the first chip core of all the parallel processors in the group.

[0142] According to the definition of the first topology structure, the second chip core of the parallel processor 101_4 needs to be connected to the second chip core of the parallel processor 101_6.

[0143] The first topology configuration file configures the optical circuit switch 102_3 to turn on the port 8 of the parallel processor 101_4 and the port 8 of the parallel processor 101_6, thereby establishing a communication connection between the port 8 of the parallel processor 101_4 and the port 8 of the parallel processor 101_6. Thus, the second chip core of the parallel processor 101_4 is connected to the second chip core of all the parallel processors in the group.

[0144] Moreover, according to the definition of the first topology structure, the second chip core of the parallel processor 101_4 needs to be connected to the second chip core of the parallel processor 101_3 in the other group.

[0145] The first topology configuration file is used to configure the optical circuit switch 102_2 to connect port 6 of the parallel processor 101_4 and port 6 of the parallel processor 101_3, thereby establishing a communication connection between port 6 of the parallel processor 101_4 and port 6 of the parallel processor 101_3. In this way, the second chip core of the parallel processor 101_4 is connected to the second chip core of the corresponding parallel processor in the other group.

[0146] The connection relationship of other parallel processors is the same as above, and will not be described here.

[0147] In this way, the first topology structure is realized by the communication cable and the optical circuit switch.

[0148] In the second topology structure, each parallel processor is connected to at least one parallel processor by N optical circuit switches, wherein the at least one parallel processor is the parallel processor in each group in the first topology structure which does not have a communication connection relationship with the A chip cores of the parallel processor.

[0149] In the second topology structure, as long as a communication connection (communication cable or communication cable + optical circuit switch) is established between one chip core, it is considered that the two parallel processors have a connection relationship, and in the second topology structure, it is required that each processor has a communication connection relationship with other processors.

[0150] Figure 7 The connection relationship diagram of the second topology structure provided by at least one embodiment of the present disclosure is shown.

[0151] In the second topology structure, Figure 5 In the second topology structure, Figure 7 In the second topology structure, the solid line represents the communication cable in the first topology structure, and the repeated part will not be described here. The dashed line represents the communication connection established by the optical circuit switch, and the box on the dashed line represents the number of the optical circuit switch.

[0152] For example, in this embodiment, three optical circuit switches are shown, which are respectively denoted as optical circuit switch 102_0, optical circuit switch 102_2 and optical circuit switch 102_3 for description. The optical circuit switch 102_0 is connected to port 2 of each parallel processor by a communication cable, the optical circuit switch 102_2 is connected to port 6 of each parallel processor by a communication cable, and the optical circuit switch 102_3 is connected to port 8 of each parallel processor by a communication cable.

[0153] The cluster of parallel processors can also include an optical circuit switch 102_1 connected to port 4 of each parallel processor via a communication cable, which can serve as an expansion interface in the second topology. For example, port 4 and port 10 can serve as expansion ports, forming a "2-machine supernode". For example, assuming 8 parallel processors are connected in a topology to form a node, a "2-machine supernode" means that 1 supernode includes 2 nodes, and the parallel processors in the supernode can communicate with each other, for example, a parallel processor in one node can be connected to a corresponding parallel processor in another node via an expansion port to establish a communication relationship.

[0154] For example, in other embodiments, port 1 connected to the electrical switch can also be multiplexed to connect to and Figure 7 Other parallel processors connected to parallel processor 101_0 via port 6 in the second topology can be expanded using ports 4, 6, and 10 of the parallel processors to form a "4-machine supernode", which means that 1 supernode includes 4 nodes. For example, a parallel processor in one node, such as parallel processor 101_0, can be connected to corresponding parallel processors (e.g., parallel processors 101_0) in three other nodes via expansion ports 4, 6, and 10 to establish a communication relationship.

[0155] For example, in the second topology, each parallel processor is connected to at least one parallel processor of another group via N optical circuit switches and the first chip core or the second chip core of the parallel processor, where the at least one parallel processor is a parallel processor in the other group that does not have a communication connection relationship with the first chip core and the second chip core of the parallel processor.

[0156] For example, parallel processor 101_0 needs to be connected to parallel processor 101_4, parallel processor 101_5, and parallel processor 101_6 according to the definition of the second topology.

[0157] For example, in the second topology, each parallel processor is connected to at least one parallel processor of another group via N optical circuit switches and the first chip core or the second chip core of the parallel processor, where the at least one parallel processor is a parallel processor in the other group that does not have a communication connection relationship with the first chip core and the second chip core of the parallel processor. Figure 7As shown, the optical circuit switch 102_0 is configured to connect port 2 of the parallel processor 101_0 and port 2 of the parallel processor 101_5 by the second topology configuration file, so as to establish a communication connection between the port 2 of the parallel processor 101_0 and the port 2 of the parallel processor 101_5. Also, the optical circuit switch 102_3 is configured to connect port 8 of the parallel processor 101_0 and port 8 of the parallel processor 101_6 by the second topology configuration file, so as to establish a communication connection between the port 8 of the parallel processor 101_0 and the port 8 of the parallel processor 101_6. Also, the optical circuit switch 102_2 is configured to connect port 6 of the parallel processor 101_0 and port 6 of the parallel processor 101_4 by the second topology configuration file, so as to establish a communication connection between the port 6 of the parallel processor 101_0 and the port 6 of the parallel processor 101_4.

[0158] Thus, the communication connection between the parallel processor 101_0 and all other parallel processors is realized.

[0159] Taking the parallel processor 101_5 as an example again, according to the definition of the second topology structure, the parallel processor 101_5 also needs to establish a communication connection with the parallel processor 101_0, the parallel processor 101_1 and the parallel processor 101_3.

[0160] As shown, Figure 2 the optical circuit switch 102_0 is configured to connect port 2 of the parallel processor 101_5 and port 2 of the parallel processor 101_0 by the second topology configuration file, so as to establish a communication connection between the port 2 of the parallel processor 101_5 and the port 2 of the parallel processor 101_0. Also, the optical circuit switch 102_3 is configured to connect port 8 of the parallel processor 101_5 and port 8 of the parallel processor 101_3 by the second topology configuration file, so as to establish a communication connection between the port 8 of the parallel processor 101_5 and the port 8 of the parallel processor 101_3. Also, the optical circuit switch 102_2 is configured to connect port 6 of the parallel processor 101_5 and port 6 of the parallel processor 101_1 by the second topology configuration file, so as to establish a communication connection between the port 6 of the parallel processor 101_5 and the port 6 of the parallel processor 101_1.

[0161] Thus, the communication connection between the parallel processor 101_5 and all other parallel processors is realized.

[0162] The connection relationship of other parallel processors is the same, which will not be described here.

[0163] Thus, the second topology structure is realized by the communication cable and the optical circuit switch.

[0164] In summary Figure 5-7In Figure 5 the example, port 2 of all parallel processors is connected to the same optical circuit switch, port 4 of all parallel processors is connected to the same optical circuit switch, port 6 of all parallel processors is connected to the same optical circuit switch, port 8 of all parallel processors is connected to the same optical circuit switch, and the bandwidth of the communication cable is consistent, thereby keeping the bandwidth between chip cores or between parallel processors consistent and fully utilizing the high bandwidth. Port 1 of the parallel processor is connected to an electrical switch, and ports 3, 5, 7, and 9 of each parallel processor are fixedly connected in a communication cable mode as shown in Figure 8 , and ports 2, 4, 6, and 8 are switched in connection by the optical circuit switch to form a first topology structure and a second topology structure, and port 10 of each parallel processor is used as an expansion port. The first topology structure is used to perform a computing task related to a traditional Transformer model such as GPT, LLAMA, etc., and the second topology structure is used to perform a computing task related to a MoE model.

[0165] In the above embodiment, some communication connections that do not change the port connection relationship in the topology switching process are completed by the communication cable, and some communication connections that change the port connection relationship in the topology switching process are completed by the optical circuit switch, without the need for any physical hardware rewiring operation, so that flexible and efficient switching of the parallel processor cluster between the first topology structure and the second topology structure can be achieved, thereby switching the topology structure according to different computing tasks and adapting the corresponding topology structure to the communication demand of the computing task to maximize the performance advantage and improve the hardware generalization and flexibility.

[0166] Specifically, the N ports of each parallel processor are fixedly coupled with the corresponding switch, and the optical circuit switch can change the communication path between the switch and the other processor ports by dynamically configuring the on-off state of the optical switch and the link mapping relationship. This topology dynamic switching mechanism based on the optical circuit switch not only avoids the problems of high operation complexity, long system downtime, and rising link failure rate caused by hardware rewiring, but also accurately matches the optimal processor interconnection topology according to the computing characteristics and communication demand of different artificial intelligence models, effectively reduces the cross-processor data transmission delay, and improves the utilization rate of computing power resources. Whether it is a long sequence parallel computation for a Transformer type large language model or a data communication demand for two-way interaction of a MoE model, the present disclosure can adapt to the performance demand of different models through flexible switching of the topology, while ensuring the stability of the system, and realizing efficient operation and performance optimization under multiple models.

[0167] Of course, it should be noted that the above embodiment only shows one possible cluster configuration and the corresponding hardware wiring mode, but the present disclosure is not limited thereto.

[0168] It should be noted that, in at least one embodiment of this disclosure, the parallel processor cluster may include more or fewer circuits or units, and the connection relationship between the various circuits or units is not limited and can be determined according to actual needs. The specific configuration of each circuit or unit is not limited; it can be constructed from analog devices, digital chips, or other suitable methods according to circuit principles.

[0169] For example, parallel processor clusters can be implemented in hardware, software, or a combination of both, and this disclosure does not impose any specific restrictions on this.

[0170] At least one embodiment of this disclosure also provides a method for topology switching of a parallel processor cluster.

[0171] Figure 8 This is a schematic flowchart illustrating a method for switching the topology of a parallel processor cluster provided in at least one embodiment of the present disclosure.

[0172] The parallel processor cluster consists of M parallel processors and N optical circuit switches. The N ports on each parallel processor are connected one-to-one with the N optical circuit switches. The total number of ports on each parallel processor is greater than M and N is less than M.

[0173] For details on the specific architecture of the parallel processor cluster, please refer to the aforementioned description of the parallel processor cluster; it will not be repeated here.

[0174] For example, the ports of the parallel processors connected to the N ports of each parallel processor are different in the first topology and the second topology. The N optical circuit switches are configured to establish communication connections between the N ports of each of the M parallel processors according to the topology configuration file, so as to form the first topology or the second topology.

[0175] For example, the connection relationships of the ports other than N ports in each parallel processor are the same in the first and second topologies, and the other ports of each parallel processor are connected by communication cables.

[0176] Therefore, during topology switching, the ports of the optical circuit switch can be configured to be open through the topology configuration file to switch the topology structure.

[0177] like Figure 9 As shown, the topology switching method includes step S10: according to the computing task executed by the parallel processor cluster, the topology of the parallel processor cluster is switched through N optical circuit switches, wherein the topology of the parallel processor cluster includes a first topology and a second topology.

[0178] In the first topology, the M parallel processors are divided into L groups, each group including M / L parallel processors, the M / L parallel processors being fully connected, and the M / L parallel processors being respectively connected in communication with the M / L parallel processors included in each other group, the M / L parallel processors being respectively connected with different M / L parallel processors in each other group, and in the second topology, the M parallel processors are fully connected, M, N and L being positive integers.

[0179] For specific descriptions of the first topology and the second topology, reference can be made to the foregoing descriptions of the embodiments, which will not be repeated here.

[0180] For example, in some embodiments, step S10 can include: in response to the computing task being an inference or training task of a first model, configuring the N optical circuit switches according to a first topology configuration file to switch the cluster of parallel processors to the first topology, wherein when the first model is inferred or trained, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated; and in response to the computing task being an inference or training task of a second model, configuring the N optical circuit switches according to a second topology configuration file to switch the cluster of parallel processors to the second topology, wherein when the second model is inferred or trained, the M parallel processors exchange data bidirectionally in pairs.

[0181] For example, the communication operators of the first model include operators that communicate through tensor parallelism or sequence parallelism, and the communication operators include ring reduction operators, all-gather operators, scatter operators, etc. For example, the first model can be a network model that includes more or a large number of, for example, tensor parallelism or sequence parallelism communications, such as traditional Transformer class models such as GPT, LLAMA, etc.

[0182] For example, the communication operators of the second model include operators that send data to all other parallel processors while receiving data from all other parallel processors, and the communication operators include all-to-all operators. For example, the second model can be a MoE model, or other network models that have more or a large number of, for example, all-to-all communication operators.

[0183] Thus, according to the communication requirements of the computing task executed by the parallel processor cluster, the optical circuit switch is configured by the configuration file, the connection relationship of the optical circuit switch is switched, and thus the topology structure of the parallel processor cluster is realized. The software-defined dynamic reconstruction of the communication topology of the parallel processor cluster can be realized without any physical hardware operation. The cluster topology mode can be flexibly switched according to the communication characteristics of different computing tasks, so as to achieve excellent performance in diversified computing models. This dynamic topology switching mechanism breaks the limitation that the traditional hardware topology is fixed after deployment, so that the parallel processor cluster can adapt to the differentiated needs of various models such as training, inference and scientific computing. Under the premise of controllable hardware deployment cost, the optimal solution of performance in all scenarios is realized.

[0184] As to the circuit connection relationship of the first topology structure and the second topology structure during the switching process, reference can be made to the related description of the foregoing embodiments, which will not be repeated here.

[0185] For example, each parallel processor includes a plurality of chip cores, and as to the topology switching and connection relationship in the case of a plurality of chip cores, reference can be made to the related description of the foregoing embodiments, which will not be repeated here.

[0186] The topology switching method of the parallel processor cluster provided by at least one embodiment of the present disclosure can achieve similar technical effects as the foregoing parallel processor cluster, which will not be repeated here.

[0187] Figure 9 A schematic diagram of the non-transitory computer-readable storage medium provided by at least one embodiment of the present disclosure is shown. For example, as shown in Figure 10 The storage medium 200 can be a non-transitory computer-readable storage medium, and one or more computer readable instructions 201 can be non-transitorily stored on the storage medium 200. For example, when the computer readable instructions 201 are executed by a processor, one or more steps of the topology switching method described above can be performed.

[0188] For example, the storage medium 200 can be applied in an electronic device 300, for example, the storage medium 200 can include a storage device 308 in the electronic device 300.

[0189] For example, the storage device can include any combination of one or more computer program products, which can include various forms of computer-readable storage media, such as volatile and / or non-volatile computer-readable media. For example, volatile computer-readable media can include random access memory (RAM), and / or cache memory, etc. Non-volatile computer-readable media can include read only memory (ROM), hard disks, erasable programmable read only memory (EPROM), portable compact disc read only memory (CD-ROM), USB memory, flash memory, etc. One or more computer-readable instructions can be stored on the computer-readable storage media, and the processor can execute the computer-readable instructions to implement various functions of the processor. Various application programs and various data, etc. can also be stored in the storage media.

[0190] For example, the storage media can include a memory card of a smart phone, a cache component of a tablet computer, a hard disk of a personal computer, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), portable compact disc read only memory (CD-ROM), flash memory, or any combination of the above storage media, or other applicable storage media.

[0191] Figure 10 Another schematic block diagram of an electronic device is provided for an embodiment of the present disclosure. As shown in Figure 10 The electronic device 300 is suitable for implementing the topology switching method provided by the embodiments of the present disclosure, for example. It should be noted that Figure 10 The components of the electronic device 300 shown are only exemplary and not limiting, and the electronic device 300 can also have other components according to actual application needs.

[0192] As shown in Figure 10 The electronic device 300 can include a processing device 301 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to non-transitory computer-readable instructions stored in the memory to implement various functions.

[0193] For example, the computer-readable instructions executed by the processing device 301 can perform one or more steps of the topology switching method according to any of the above embodiments. It should be noted that the detailed description of the processing process of the topology switching method can refer to the related description in the above embodiments of the topology switching method.

[0194] For example, the memory can include any combination of one or more computer program products, which can include various forms of computer-readable storage media, such as volatile and / or non-volatile memory. Volatile memory, for example, can include random access memory (RAM) 303 and / or cache, etc., from which computer-readable instructions can be loaded and executed, for example, from storage 308 into random access memory (RAM) 303. Non-volatile memory, for example, can include read-only memory (ROM) 302, hard disk, erasable programmable read-only memory (EPROM), compact disc read-only memory (CD-ROM), USB memory, flash memory, etc. Various applications and various types of data can also be stored in the computer-readable storage media, such as various data used by the applications, etc.

[0195] For example, the processing device 301, the read-only memory (ROM) 302, and the random access memory (RAM) 303 are connected to each other through a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0196] Generally, the following devices can be connected to the input / output (I / O) interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, a flash memory, etc.; and communication devices 309. The communication devices 309 can allow the electronic device 300 to communicate wirelessly or wired with other electronic devices to exchange data. Although ​ The electronic device 300 is shown with various devices, but it should be understood that not all of the shown devices are required to be implemented or provided, and the electronic device 300 can instead be implemented or provided with more or fewer devices. For example, the processing device 301 can control other components in the electronic device 300 to perform desired functions. The processing device 301 can be a central processing unit (CPU), a tensor processing unit (TPU), or a graphics processing unit (GPU), etc., which has data processing ability and / or program execution ability. The central processing unit (CPU) can be X86, ARM, RISC-V architecture, etc. The GPU can be directly integrated into the SOC, directly integrated into the mainboard, or built into the north bridge chip of the mainboard.

[0197] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other processing devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other processing devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0198] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware. In some cases, the names of the units do not constitute a limitation on the units themselves.

[0199] The functions described in this specification can be performed by one or more hardware logic components. For example, non-limiting examples of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0200] The above description is merely illustrative of the exemplary embodiments of the present disclosure and the principles of the technology involved. It should be understood by those skilled in the art that the scope of the disclosure involved is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by the combinations of the above technical features or their equivalent features without departing from the above disclosed concept. For example, technical solutions formed by replacing the above features with technical features disclosed in the present disclosure (but not limited to) having similar functions.

[0201] Moreover, while operations are depicted in a particular, chronological sequence in this disclosure, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, at all. To the extent that specific steps are described in a particular order, alternate implementations can perform the operations in different orders, or employ the steps in parallel or concurrently. Additionally, certain features can also be implemented as processes - in places other than those that are described. For example, multiple processing or manufacturing systems can be employed and / or particular operations can be performed by a processing system other than the one that performs other operations, or operations can be distributed across multiple processing systems. Furthermore, to the extent that the terms "including", "includes", "having", "has", "contain", "contains", "indicate", "indicates", "comprise", "comprises", or "comprising" are used in this disclosure and / or claims, these are taken to be open-ended, allowing for items, components, or processes not specifically named or processes not specifically described. Additionally, certain features that are already known in the design, manufacture, and / or use of devices of the type disclosed herein can not have been described in particular detail herein. Instead, it is contemplated that suitable conventional components or processes will be used as desired. Finally, the terms "first", "second", "third", etc., are used herein to describe various steps in a process and various elements in a device, and these designations can be interchanged under appropriate circumstances. For example, a "first" element can become a "second" element, and a "second" element can become a "first" element, when two elements are discussed in as much detail. In general, the terms "first", "second", "third", etc., are used to distinguish between two or more elements or steps in a process where the elements or steps have particular characteristics or functions as described in the specification. The terms "first", "second", "third", etc., are not necessarily used to distinguish between two or more related elements or steps, unless the context clearly indicates this is the intended usage.

[0202] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0203] For the disclosure, the following points need to be explained:

[0204] (1) The drawings of the embodiments of the disclosure only involve the structures related to the embodiments of the disclosure, and other structures can be referred to the general design.

[0205] (2) In the case of no conflict, the embodiments of the disclosure and the features in the embodiments can be combined to obtain new embodiments.

[0206] The above description is merely illustrative of the disclosure and is not intended to limit the scope of the disclosure. The scope of the disclosure should be determined by the claims.

Claims

1. A cluster of parallel processors, characterized in that, The parallel processor cluster comprises M parallel processors, N optical circuit switches, and N ports on each parallel processor are one-to-one coupled with the N optical circuit switches, the total number of ports of each parallel processor is greater than M, and N is less than M, The parallel processor cluster supports switching between a first topology and a second topology, and when switching the topologies, the parallel processor cluster is configured to switch between the first topology and the second topology through the N optical circuit switches according to a computing task performed by the parallel processor cluster, In the first topology, the M parallel processors are divided into L groups, each group comprising M / L parallel processors, the M / L parallel processors are fully connected, and the M / L parallel processors are respectively communicatively connected with the M / L parallel processors included in each other group, and the M / L parallel processors are respectively connected with different M / L parallel processors in each other group, In the second topology, the M parallel processors are fully connected, M, N, and L are positive integers.

2. The cluster of parallel processors of claim 1, wherein, The ports of the parallel processors connected by the N ports of each parallel processor are different in the first topology and the second topology, The N optical circuit switches are configured to establish a communicatively connected relationship between the N ports of the M parallel processors according to a topology configuration file to form the first topology or the second topology.

3. The cluster of parallel processors of claim 1, wherein, The connection relationship of other ports of each parallel processor except the N ports is the same in the first topology and the second topology, The other ports of each parallel processor establish a communicatively connected relationship using a communication cable.

4. The cluster of parallel processors of claim 1, wherein, Each optical circuit switch is coupled with a port of the same port number on the M parallel processors through a communication cable.

5. The cluster of parallel processors of claim 1, wherein, Each parallel processor comprises A chip cores, and A is a positive integer greater than 1, In the first topology, the M / L parallel processors are fully connected, including fully connecting the i-th chip cores of the M / L parallel processors, i is a positive integer and sequentially takes 1 to A, and The M / L parallel processors are respectively communicatively connected with the M / L parallel processors included in each other group, including the i-th chip cores of the M / L parallel processors and the i-th chip cores of the M / L parallel processors included in each other group are respectively communicatively connected; In the second topology, the M parallel processors are fully connected, including one chip core of each parallel processor is communicatively connected with the corresponding chip core in the other M-1 parallel processors.

6. The cluster of parallel processors of claim 5, wherein, At least one chip core of the A chip cores of each parallel processor in each group has a communicatively connected relationship with the corresponding chip core of the A chip cores of all other parallel processors in the group where the parallel processor is located through a communication cable, Each of the M / L parallel processors is further communicatively connected to one of the M / L parallel processors included in each of the other groups via the communication cable, the communicatively connection including at least one of the A chip cores of the parallel processor and a corresponding one of the A chip cores of the one of the parallel processors.

7. The cluster of parallel processors of claim 6, wherein, In the second topology, each parallel processor is communicatively connected to at least one parallel processor included in each of the other groups via the N optical circuit switches, wherein the at least one parallel processor is one of the M / L parallel processors included in each of the other groups that does not have the communicatively connection relationship with any of the A chip cores of the parallel processor. In the first topology, each chip core in each parallel processor is communicatively connected to a target chip core via the N optical circuit switches, wherein the target chip core is one that has not yet established the communicatively connection relationship with any of the A chip cores of the parallel processor according to the first topology.

8. The cluster of parallel processors of claim 5, wherein, In response to A=2 and L=2, each parallel processor includes two chip cores, including a first chip core and a second chip core, The first chip core of each parallel processor is communicatively connected to the first chip cores of T1 parallel processors in the group via the communication cable, and the second chip core of each parallel processor is communicatively connected to the second chip cores of T2 parallel processors in the group via the communication cable, and the first chip core of each parallel processor is further communicatively connected to the first chip core of a corresponding parallel processor in another group via the communication cable, wherein T1 and T2 are positive integers, T1+T2=M / L-1, and the T1 parallel processors are different from the T2 parallel processors.

9. The parallel processor cluster of claim 8, wherein, In the first topology, the first chip core of each parallel processor is communicatively connected to the first chip cores of M / 2-T1-1 parallel processors in the group other than the T1 parallel processors via the N optical circuit switches, the second chip core of each parallel processor is communicatively connected to the second chip cores of the other parallel processors in the group other than the T2 parallel processors via the N optical circuit switches, the second chip core of each parallel processor is further communicatively connected to the second chip core of the corresponding parallel processor in the other group via the N optical circuit switches.

10. The parallel processor cluster of claim 8, wherein, In the second topology, each parallel processor is communicatively connected to the first chip core or the second chip core of at least one parallel processor in the other group via the N optical circuit switches, wherein the at least one parallel processor is one of the parallel processors in the other group that does not have the communicatively connection relationship with the first chip core and the second chip core of the parallel processor.

11. The cluster of parallel processors of any of claims 1-10, wherein, The parallel processor cluster performs a computing task according to the parallel processor cluster, and switches the first topology and the second topology through the N optical circuit switches, including performing the following operations: In response to the computing task being an inference or training task of a first model, the N optical circuit switches are configured according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is in inference or training, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated. In response to the computing task being an inference or training task of a second model, the N optical circuit switches are configured according to a second topology configuration file to switch the parallel processor cluster to the second topology, wherein when the second model is in inference or training, data is exchanged between the M parallel processors in a two-way manner.

12. The cluster of parallel processors of claim 11, wherein, The communication operator of the first model includes an operator that communicates through tensor parallel or sequence parallel, and the communication operator includes a ring reduction operator, a full collection operator, and a scattering operator.

13. The cluster of parallel processors of claim 11, wherein, Wherein, The communication operator of the second model includes an operator that sends data to all other parallel processors and receives data from all other parallel processors, and the communication operator includes a full-to-full operator.

14. The cluster of parallel processors of any of claims 1-10, wherein, The N ports on each parallel processor are coupled to the N optical circuit switches through communication cables, and the data transmission bandwidth of the communication connection between the M parallel processors is consistent.

15. A method of topology switching of a cluster of parallel processors, characterized in that, The parallel processor cluster includes M parallel processors and N optical circuit switches, the N ports on each parallel processor are one-to-one connected to the N optical circuit switches, the total number of ports on each parallel processor is greater than M, and N is less than M, The topology switching method includes: According to the computing task performed by the parallel processor cluster, the topology structure of the parallel processor cluster is switched through the N optical circuit switches, wherein the topology structure of the parallel processor cluster includes a first topology and a second topology; In the first topology, the M parallel processors are divided into L groups, each group includes M / L parallel processors, the M / L parallel processors are fully connected, and the M / L parallel processors are respectively connected to the M / L parallel processors included in each other group, and the M / L parallel processors are respectively connected to different M / L parallel processors in each other group, In the second topology, the M parallel processors are fully connected, M, N, and L are positive integers.

16. The topology switching method of claim 15, wherein, According to the computing task performed by the parallel processor cluster, the topology structure of the parallel processor cluster is switched through the N optical circuit switches, including: In response to the computing task being an inference or training task of a first model, the N optical circuit switches are configured according to a first topology configuration file to switch the parallel processor cluster to the first topology, wherein when the first model is in inference or training, data is split by dimension to different parallel processors for distributed computing, and the computing results are aggregated. In response to the computing task being an inference or training task of a second model, the N optical circuit switches are configured according to a second topology configuration file to switch the parallel processor cluster to the second topology structure, wherein the second model exchanges data between the M parallel processors in a two-by-two bidirectional manner when inferring or training.

17. An electronic device, comprising: Comprise: a memory, which stores computer executable instructions non-transiently; a processor, which is configured to run the computer executable instructions, wherein the computer executable instructions, when run by the processor, implement the topology switching method according to claim 15 or 16.

18. A non-transitory computer-readable storage medium, comprising: The non-transient computer readable storage medium stores computer executable instructions, The computer executable instructions, when executed by the processor, implement the topology switching method according to claim 15 or 16.

Images