Aggregation communication method and device based on tensor parallelization, equipment and medium

Abstract

The invention discloses an aggregation communication method and device based on tensor parallelization, equipment and a medium. The method comprises the following steps: determining a target topological logic based on a parallel output tensor scale corresponding to a target model and a preset scale threshold; determining a target main computing card and a target auxiliary computing card based on the target topological logic and the computing card distribution information corresponding to the target model; determining a first tensor fragment on the target main computing card and a second tensor fragment on the target auxiliary computing card in the total parallel output tensor corresponding to the target model based on the current number of the target main computing card; and generating a target full-protocol link between the first tensor fragment and each second tensor fragment based on a preset protocol logic, so that the target main computing card realizes aggregation communication with the target auxiliary computing card according to the target full-protocol link. Through the technical scheme of the invention, aggregation communication can be realized according to different requirements of models with different sizes on parallel splitting, and the efficiency of aggregation communication is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a tensor parallel-based aggregate communication method, device, equipment and medium. Background Art

[0002] With the widespread application and development of large language models (LLMs), the scale of models is getting larger and larger, making it a common practice to build LLMs through distributed systems. However, if LLMs are built using distributed systems, different parallel splitting strategies need to be adopted at the algorithm level, especially tensor parallelism, which takes a long time in collective communication.

[0003] When generating parallel splitting strategies for tensor parallelism, the prior art usually focuses on parallel splitting strategies for small models. However, as the LLM increases, the values ​​of different models are different, such as 4 cards, 8 cards, 16 cards, or 32 cards. If the parallel splitting strategy of small models is used for different models, the needs of different models cannot be taken into account, which greatly reduces the communication efficiency. Therefore, how to realize tensor parallel aggregation communication and improve the efficiency of aggregation communication according to the different needs of models of different sizes for parallel splitting is a problem that needs to be solved urgently. Summary of the invention

[0004] The present invention provides a tensor-parallel based aggregate communication method, device, equipment and medium, which can solve the problem of low efficiency of tensor-parallel aggregate communication.

[0005] According to one aspect of the present invention, there is provided a tensor parallel-based aggregate communication method, comprising:

[0006] Obtain the computing card distribution information, parallel output tensor scale, and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model;

[0007] Determine the target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold;

[0008] Determine a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information;

[0009] Determine, based on the current serial number of the target main computing card, a first tensor slice on the target main computing card and a second tensor slice on the target secondary computing card in the full parallel output tensor;

[0010] A target full protocol link is generated between the first tensor slice and each second tensor slice based on the preset protocol logic, so that the target main computing card can realize aggregate communication with the target secondary computing card according to the target full protocol link.

[0011] According to another aspect of the present invention, there is provided a tensor parallel based aggregate communication device, comprising:

[0012] A data acquisition module is used to obtain the computing card distribution information, parallel output tensor scale and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model;

[0013] A first determination module, configured to determine a target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold;

[0014] A second determination module is used to determine a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information;

[0015] A third determination module is used to determine the first tensor slice on the target main computing card and the second tensor slice on the target secondary computing card in the full parallel output tensor based on the current serial number of the target main computing card;

[0016] The aggregate communication module is used to generate a target full protocol link between the first tensor slice and each second tensor slice based on a preset protocol logic, so that the target main computing card can realize aggregate communication with the target secondary computing card based on the target full protocol link.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the tensor parallel-based aggregate communication method described in any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tensor-parallel-based aggregate communication method described in any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the tensor-parallel-based aggregate communication method according to any embodiment of the present invention is implemented.

[0023] The technical solution of the embodiment of the present invention determines the target topology logic through the parallel output tensor scale corresponding to the target model and the preset scale threshold; then, the target main computing card and the target secondary computing card are determined based on the target topology logic and the computing card distribution information corresponding to the target model, and the first tensor shard on the target main computing card and the second tensor shard on the target secondary computing card in the full parallel output tensor corresponding to the target model are determined based on the current number of the target main computing card; finally, the target full protocol link between the first tensor shard and each second tensor shard is generated based on the preset protocol logic, so that the target main computing card can realize the aggregate communication with the target secondary computing card based on the target full protocol link. Due to the different requirements for parallel splitting according to models of different sizes, different topology logics can be used according to different communication ranges of aggregate communication. Therefore, the problem of low efficiency of tensor parallel aggregate communication is solved, tensor parallel aggregate communication can be realized, and the efficiency of aggregate communication is improved.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flow chart of a tensor parallel-based aggregate communication method provided according to Embodiment 1 of the present invention;

[0027] Figure 2 This is a physical connection diagram of a server cluster provided according to Embodiment 1 of the present invention;

[0028] Figure 3 is a flow chart of a tensor parallel-based aggregate communication method provided according to Embodiment 2 of the present invention;

[0029] Figure 4 This is a topological diagram of an in-machine full protocol logic provided according to Embodiment 2 of the present invention;

[0030] Figure 5This is a topological diagram of a full protocol logic between machines provided according to the second embodiment of the present invention;

[0031] Figure 6 is a schematic diagram of a bidirectional ring logical topology provided according to Embodiment 2 of the present invention;

[0032] Figure 7 is a schematic diagram of a single bidirectional ring logical topology provided according to Embodiment 2 of the present invention;

[0033] Figure 8 is a schematic diagram of a forward ring logical topology in a single bidirectional ring logical topology provided according to Embodiment 2 of the present invention;

[0034] Fig. 9 is a schematic diagram of a reverse ring logical topology in a single bidirectional ring logical topology provided according to Embodiment 2 of the present invention;

[0035] Fig.10 is a schematic diagram of a parallel reduction operation of tensors within a machine provided according to the second embodiment of the present invention;

[0036] Fig.11 is a schematic diagram of an in-machine tensor parallel broadcast operation provided according to Embodiment 2 of the present invention;

[0037] Fig.12 This is a schematic diagram of a target full protocol link corresponding to an inter-machine full protocol logic provided according to Embodiment 2 of the present invention;

[0038] Fig.13 is a schematic diagram of a rotation path provided according to Embodiment 2 of the present invention;

[0039] Fig.14 is a flowchart of an optional in-machine tensor parallel aggregation communication method provided according to Embodiment 2 of the present invention;

[0040] Fig.15 is a flowchart of an optional tensor parallel aggregation communication method between machines provided according to the second embodiment of the present invention;

[0041] Fig.16 is a structural schematic diagram of a tensor parallel-based aggregate communication device provided according to Embodiment 3 of the present invention;

[0042] Fig.17 It is a structural schematic diagram of an electronic device for implementing the tensor parallel-based aggregate communication method of an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0044] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0045] Embodiment 1

[0046] Figure 1 A flowchart of a tensor parallel based aggregation communication method provided in the first embodiment of the present invention. This embodiment can be applied to the situation where different tensor parallel splitting strategies are generated according to different requirements for parallel splitting of models of different sizes. The method can be executed by a tensor parallel based aggregation communication device, which can be implemented in the form of hardware and / or software. The tensor parallel based aggregation communication device can be configured in an electronic device. Exemplarily, it can be configured in a server. Figure 1 As shown, the method includes:

[0047] S110, obtaining computing card distribution information, parallel output tensor scale, and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model.

[0048] Among them, the target model may refer to a model that requires tensor parallel communication. Exemplarily, it may be a large language model. A computing card (General Compute Unit, GCU) may refer to a device used to complete artificial intelligence (Artificial Intelligence, AI) calculations. Exemplarily, a GCU may be a graphics processing unit (Graphics Processing Unit, GPU) or an acceleration unit for AI. The computing card distribution information may refer to the basic information of each computing card in a distributed system. Exemplarily, the computing card distribution information may include the total number of computing cards in the distributed system and the number of computing cards in each server.

[0049] Among them, the parallel output tensor may refer to each tensor output in parallel by the computing card corresponding to the target model. Usually, the weight of the target model will be split onto multiple computing cards, and then the same data to be trained will be input into each computing card respectively, and the weight split to the current computing card will be calculated, thereby obtaining the parallel output tensors corresponding to each computing card. The parallel output tensor scale may refer to the parameter scale of the parallel output tensor corresponding to the target model. Exemplarily, the parallel output tensor scale may be the number of computing cards that need to be carried. Usually, the size of the target model is different, and the corresponding parallel output tensor scale will also be different. The full parallel output tensor may refer to all parallel output tensors corresponding to the target model.

[0050] S120. Determine a target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold.

[0051] The preset scale threshold may refer to a preset value for evaluating the scale of parallel output tensors. Exemplarily, the preset scale threshold may be set according to the number of computing cards in each server in the computing card distribution information. Specifically, the preset scale threshold may be set to the number of computing cards in the server.

[0052] The topology logic may refer to the full-protocol topology connection logic. Exemplarily, the topology logic may be divided into the intra-machine full-protocol logic and the inter-machine full-protocol logic. The target topology logic may refer to the topology logic corresponding to the target model.

[0053] S130. Determine a target primary computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information.

[0054] The target main computing card may refer to a computing card selected to perform an allreduce operation, and the target secondary computing card may refer to a computing card selected to cooperate with the target main computing card to complete the allreduce operation.

[0055] In an optional embodiment, the number of port groups of the target main computing card and the number of port groups of the target secondary computing card are both preset associated with the basic number of computing cards in the same server in the computing card distribution information. The basic number of computing cards may refer to the number of computing cards contained in each server. Usually, in a distributed system, the basic number of computing cards in each server is consistent. The preset association relationship may refer to a pre-set relationship for evaluating the number of port groups. Exemplarily, the preset association relationship may be that the number of port groups of the target main computing card and the number of port groups of the target secondary computing card are both consistent with the basic number of computing cards in the same server in the computing card distribution information.

[0056] Figure 2 The figure shows a physical connection diagram of a server cluster provided by an embodiment of the present invention. Specifically, take the distributed cluster corresponding to the target model as an example, which contains N servers and each server contains M computing cards. According to the preset association relationship, each computing card has M groups of ports, wherein the M-1 group of ports is used to form a physical fully connected network within the server with other M-1 computing cards in the same server, and the remaining group of ports is used to complete the connection between servers through "inter-server interconnection". Each group of ports can be an Ethernet port, multiple Ethernet ports, or a high-speed interface such as a high-speed serial computer expansion bus standard (Peripheral Component Interconnect express, PCIe) port. The fully connected network can be a direct physical channel direct connection or an indirect physical interconnection through a switch. Inter-server interconnection is usually completed using a switch network. As a result, full connection is achieved between different GCUs in the server, and the GCUs between servers are also fully connected through the switch, constructing a physical interconnection channel, providing an effective basis for subsequent operations.

[0057] S140. Determine, based on the current serial number of the target main computing card, a first tensor slice on the target main computing card and a second tensor slice on the target secondary computing card in the full parallel output tensor.

[0058] The current number may refer to the computing card number corresponding to the target main computing card. Usually, in the same server, each computing card has a corresponding computing card number according to the arrangement order. It is worth noting that each computing card number in the same server is unique, and the computing card numbers between different servers may not be unique.

[0059] The first tensor shard may refer to the tensor shard that the target master computing card performs reduction calculations on in the current reduction process. The second tensor shard may refer to the tensor shard that the target slave computing card performs reduction calculations on in the current reduction process. The current reduction process may refer to the reduction calculation process with the target master computing card as the calculation subject.

[0060] S150. Generate a target full protocol link between the first tensor shard and each second tensor shard based on a preset protocol logic, so that the target main computing card can achieve aggregate communication with the target secondary computing card based on the target full protocol link.

[0061] The preset protocol logic may refer to a preset protocol computing logic. Generally, the preset protocol logic may be determined according to the target topology logic. The full protocol link may refer to a data transmission link between computing cards during the full protocol computing process. The target full protocol link may refer to a full protocol link between a target primary computing card and a target secondary computing card.

[0062] The technical solution of the embodiment of the present invention determines the target topology logic through the parallel output tensor scale corresponding to the target model and the preset scale threshold; then, the target main computing card and the target secondary computing card are determined based on the target topology logic and the computing card distribution information corresponding to the target model, and the first tensor shard on the target main computing card and the second tensor shard on the target secondary computing card in the full parallel output tensor corresponding to the target model are determined based on the current number of the target main computing card; finally, the target full protocol link between the first tensor shard and each second tensor shard is generated based on the preset protocol logic, so that the target main computing card can realize the aggregate communication with the target secondary computing card based on the target full protocol link. Due to the different requirements for parallel splitting according to models of different sizes, different topology logics can be used according to different communication ranges of aggregate communication. Therefore, the problem of low efficiency of tensor parallel aggregate communication is solved, tensor parallel aggregate communication can be realized, and the efficiency of aggregate communication is improved.

[0063] Embodiment 2

[0064] Figure 3 A flowchart of a tensor parallel-based aggregate communication method provided in the second embodiment of the present invention is provided. This embodiment is refined based on the above embodiment. In this embodiment, the operation of determining the target topology logic corresponding to the target model based on the parallel output tensor scale and the preset scale threshold is specifically refined. Specifically, it may include: if the parallel output tensor scale is lower than the preset scale threshold, the target topology logic is the in-machine full-reduction logic; if the parallel output tensor scale exceeds the preset scale threshold, the target topology logic is the inter-machine full-reduction logic. Figure 3 As shown, the method includes:

[0065] S210. Obtain computing card distribution information, parallel output tensor scale, and full parallel output tensor corresponding to the target model.

[0066] Among them, each parallel output tensor is stored on each computing card corresponding to the target model.

[0067] S220. If the parallel output tensor scale is lower than a preset scale threshold, the target topology logic is an in-machine full reduction logic.

[0068] Among them, the full reduction logic in the machine can refer to the full reduction operation logic implemented inside the server. Exemplarily, the full reduction logic in the machine can include the logical topology corresponding to each computing card in a server, that is, the parameter server logical topology. Usually, one computing card corresponds to one parameter server logical topology. Under each parameter server logical topology, the computing card is used as a server node to receive data from other computing cards in the same server, complete the reduction operation, and return the result to other computing cards.

[0069] Specifically, if the parallel output tensor scale is lower than the preset scale threshold, it means that the communication range of the parallel output tensor can be completed within a single server. Therefore, you can arbitrarily select a server in the distributed cluster as the target server, and implement the in-machine full-reduction logic in the target server. This avoids the problem of resource waste caused by all servers in the distributed cluster participating in the full-reduction calculation when the parallel output tensor scale is small.

[0070] S230. Use the current computing card in the computing card distribution information as the target primary computing card, and use the remaining computing cards in the computing card distribution information as the target secondary computing cards, and execute S280.

[0071] The current computing card may refer to the computing card currently selected for protocol computing in the target server. Exemplarily, it may be any computing card in the target server. The remaining computing cards may refer to all computing cards in the target server except the current computing card. It is worth noting that when the target topology logic is the full protocol logic in the machine, the target main computing card and the target secondary computing card are integrated in the same server.

[0072] Figure 4The topology diagram of a fully reduced logic in a machine provided by an embodiment of the present invention is shown. Specifically, taking the case where each server in a distributed cluster contains M computing cards and each computing card contains M groups of ports as an example, the fully reduced logic in the machine contains M parameter server logical topologies. Among them, parameter server logical topology 1 corresponds to computing card 1, parameter server logical topology 2 corresponds to computing card 2, and so on, parameter server logical topology M corresponds to computing card M. In parameter server logical topology 1, computing card 1 is used as the target main computing card, and the remaining M-1 computing cards are used as target secondary computing cards. In parameter server logical topology 2, computing card 2 is used as the target main computing card, and the remaining M-1 computing cards are used as target secondary computing cards. By analogy, in parameter server logical topology M, computing card M is used as the target main computing card, and the remaining M-1 computing cards are used as target secondary computing cards. In a fully connected network in the server, each computing card can be directly connected to other M-1 computing cards through M-1 groups of ports, and the two cards are interconnected through 1 group of ports.

[0073] S240: If the parallel output tensor scale exceeds a preset scale threshold, the target topology logic is an inter-machine full reduction logic.

[0074] Among them, the inter-machine full protocol logic may refer to the full protocol operation logic implemented between servers.

[0075] Specifically, if the scale of parallel output tensors exceeds the preset scale threshold, it means that the communication range of parallel output tensors cannot be completed within a single server. Therefore, multiple servers can be selected in the distributed cluster, and efficient multi-card communication can be constructed through the full-protocol logic between machines. The entire communication process is operated in parallel between logical topologies, so that all participating computing cards within the logical topology, whether within the machine or between machines, are unified into one plane, reducing the overhead caused by multi-machine operation.

[0076] S250. Determine a bidirectional ring logical topology corresponding to the target model based on the total number of computing cards in the computing card distribution information; wherein each bidirectional ring logical topology includes two unidirectional ring logical topologies, and each unidirectional ring logical topology includes all the computing cards in the computing card distribution information.

[0077] The bidirectional ring logical topology may refer to a ring logical topology including both forward and reverse directions.

[0078] Figure 5The figure shows a topological diagram of a fully-reduced logic between machines provided by an embodiment of the present invention. Specifically, a distributed cluster includes N servers, each server includes M computing cards (GCU), and each computing card includes M groups of ports. The physical topology is split into M ring logical topologies, and each server is split into M / 2 bidirectional ring logical topologies based on the fully connected topology. Each bidirectional ring logical topology includes all nodes in the server, and the M / 2 ring logical topologies correspond to M unidirectional ring logical topologies. The connection of the ring logical topologies between different servers forms a large ring logical topology according to the first connection method of the ring logical topologies with the same number, and the number of computing cards on each ring logical topology increases from M to N*M. In this way, a fully-reduced logical topology between machines is achieved.

[0079] Figure 6 The figure shows a schematic diagram of a bidirectional ring logical topology provided by an embodiment of the present invention. Specifically, take a distributed cluster containing 2 servers, each server containing 4 computing cards, and each computing card containing 4 groups of ports as an example. When M=4, M-1 groups of ports, that is, 3 groups of ports are used for full connection of the computing cards in the machine, and 1 group of ports is used to connect to the computing cards of other servers. M=4 can be split into M / 2, that is, 2 bidirectional ring logical topologies, corresponding to 4 unidirectional ring logical topologies. Figure 6 The dotted line ring logical topology and the solid line ring logical topology are two bidirectional ring logical topologies. The dotted line ring logical topology connects the computing cards 1, 2, 4, and 3 in the two servers in sequence to form a bidirectional ring logical topology with 8 computing cards, generating Figure 7 A schematic diagram of a single bidirectional ring logical topology provided by an embodiment of the present invention is shown. In the solid line ring logical topology, the computing cards 4, 1, 3 and 2 in the two servers are connected in sequence to form a bidirectional ring logical topology of 8 computing cards. It is worth noting that in actual applications, the bidirectional ring logical topology corresponding to the target model can be determined according to the above process and the actual total number of computing cards. In each bidirectional ring logical topology, the connection order of the computing cards inside each server is the same, and each computing card is guaranteed to appear once. The embodiment of the present invention does not limit the specific connection order of each computing card.

[0080] The unidirectional ring logical topology may refer to a ring logical topology in a certain direction in the bidirectional ring logical topology, for example, a clockwise positive ring logical topology or a counterclockwise negative ring logical topology.

[0081] Figure 8 FIG. 1 is a schematic diagram of a forward ring logic topology in a single bidirectional ring logic topology provided by an embodiment of the present invention. Specifically, Figure 7 Based on the single bidirectional ring logical topology shown in the figure, the clockwise unidirectional ring logical topology is extracted to generate the following Figure 8Forward ring logic topology shown. Fig. 9 FIG. 1 is a schematic diagram of a reverse ring logical topology in a single bidirectional ring logical topology provided by an embodiment of the present invention. Figure 7 Based on the single bidirectional ring logical topology shown in the figure, the counterclockwise unidirectional ring logical topology is extracted to generate the following Fig. 9 Reverse ring logical topology shown.

[0082] S260. Based on the basic number of computing cards in the same server in the computing card distribution information, each parallel output tensor is divided into blocks to obtain each tensor block, and the same-order tensor blocks on each computing card in the current unidirectional ring logical topology are used as current tensor blocks.

[0083] Among them, block processing can refer to the data segmentation operation performed on the parallel output tensor. Tensor block can refer to the tensor result obtained after block processing. The current unidirectional ring logical topology can refer to the unidirectional ring logical topology currently selected for reduction operation. Exemplarily, it can be any unidirectional ring logical topology. The same-order tensor block can refer to the tensor block with the same sequence number as the current unidirectional ring logical topology. The current tensor block can refer to the tensor block selected for reduction calculation.

[0084] Specifically, after determining the bidirectional ring logical topology corresponding to the target model based on the total number of computing cards in the computing card distribution information, each parallel output tensor can be divided into blocks according to the basic number of computing cards in each server to obtain each tensor block. Exemplarily, if the basic number of computing cards in each server is 4, each parallel output tensor can be evenly divided into 4 parts, that is, each parallel output tensor corresponds to 4 tensor blocks. Afterwards, the tensor blocks with the same number are used as the current tensor blocks according to the sequence numbering of the current unidirectional ring logical topology. For example, if the dashed line forward ring logical topology is specified as logical topology No. 1, the dashed line reverse ring logical topology is logical topology No. 2, the solid line forward ring logical topology is logical topology No. 3, and the solid line reverse ring logical topology is logical topology No. 4, then the No. 1 tensor block corresponding to each parallel output tensor can be used as the current tensor block corresponding to logical topology No. 1, the No. 2 tensor block corresponding to each parallel output tensor can be used as the current tensor block corresponding to logical topology No. 2, the No. 3 tensor block corresponding to each parallel output tensor can be used as the current tensor block corresponding to logical topology No. 3, and the No. 4 tensor block corresponding to each parallel output tensor can be used as the current tensor block corresponding to logical topology No. 4. In this way, the data on each computing card is divided into M tensor blocks, and the data of each tensor block is mapped to 1 unidirectional ring logical topology, providing an effective basis for subsequent operations.

[0085] It is worth noting that each tensor block can also be randomly mapped to a unidirectional ring logical topology, which is not limited in this embodiment of the present invention.

[0086] S270. Process the current tensor block based on the total amount of computing cards to obtain the tensor shards corresponding to the current tensor block, and use the computing cards of the same sequence corresponding to the current tensor shard as the target main computing cards, and use the remaining computing cards in the current unidirectional ring logical topology as the target secondary computing cards.

[0087] Among them, sharding processing may refer to the data slicing operation performed on the current tensor block. Tensor sharding may refer to the tensor result obtained after sharding processing. The same-sequence computing card may refer to the computing card with the same sequence number as the current tensor sharding. Exemplarily, if the current tensor sharding is tensor sharding No. 1, computing card No. 1 in the current unidirectional ring logical topology can be used as the same-sequence computing card.

[0088] It is worth noting that the same-order computing cards in the embodiment of the present invention are determined on the basis of the global server according to the current unidirectional ring logical topology. Specifically, take a distributed cluster containing 2 servers, each server containing 4 computing cards, and the computing cards in each server are numbered 1, 2, 3 and 4 respectively, and the order of the computing cards in the current unidirectional ring logical topology is server 1 computing card 1, server 1 computing card 2, server 1 computing card 4, server 1 computing card 3, server 2 computing card 1, server 2 computing card 2, server 2 computing card 4 and server 2 computing card 3 as an example. On the basis of the global server, each computing card needs to be reordered according to the order in the current unidirectional ring logical topology. For example, computing card 1 in server 1 is reordered to computing card 1, computing card 2 in server 1 is reordered to computing card 2, computing card 4 in server 1 is reordered to computing card 3, computing card 3 in server 1 is reordered to computing card 4, computing card 1 in server 2 is reordered to computing card 5, computing card 2 in server 2 is reordered to computing card 6, computing card 4 in server 2 is reordered to computing card 7, and computing card 3 in server 2 is reordered to computing card 8. Therefore, the current tensor shard can determine the corresponding computing card of the same sequence as the target main computing card according to the current tensor shard sequence number, providing an effective basis for subsequent operations.

[0089] S280. Determine, based on the current serial number of the target main computing card, a first tensor slice on the target main computing card and a second tensor slice on the target secondary computing card in the full parallel output tensor.

[0090] Specifically, if the target topology logic is the full reduction logic within the machine, it is necessary to slice each parallel output tensor according to the basic number of computing cards in the server to obtain each tensor slice, and then, according to the current number of the target main computing card, use the tensor slices with the same sequence on the target main computing card as the first tensor slice, and use the tensor slices with the same sequence on the target secondary computing card as the second tensor slice. If the target topology logic is the full reduction logic between machines, the tensor slices with the same sequence on the target main computing card with the same number as the target main computing card can be directly used as the first tensor slice, and the tensor slices with the same sequence on the target secondary computing card with the same number as the target main computing card can be used as the second tensor slice.

[0091] It is worth noting that if the target topology logic is the inter-machine full-reduction logic, the current number of the target main computing card is the re-ordered number. The specific ordering process is detailed in the above content, and the embodiment of the present invention will not be repeated here.

[0092] S290. Generate a target full protocol link between the first tensor shard and each second tensor shard based on a preset protocol logic, so that the target main computing card can achieve aggregate communication with the target secondary computing card based on the target full protocol link.

[0093] In an optional embodiment, if the target topology logic is an in-machine full-reduction logic, the target full-reduction link between the first tensor shard and each second tensor shard is generated based on the preset reduction logic, including: taking the first tensor shard as the reduction shard and the second tensor shard as the selected shard; constructing a data transmission path from the selected shard to the reduction shard and a data return path from the reduction shard to the selected shard according to the full-reduction calculation logic, and combining the data transmission path and the data return path to generate the target full-reduction link.

[0094] Among them, the reduction shard may refer to the tensor shard initially selected for the reduction operation. The candidate shard may refer to the tensor shard waiting to be selected for the reduction operation. Usually, the reduction shard and the candidate shard exist in groups. The full reduction computing logic may refer to the computing logic that distributes the new data in equal amounts to the computing cards specified during the operation after obtaining the result through a certain operation such as addition, maximum value or minimum value on the specified tensor shard, so that the data on each computing card is consistent.

[0095] The data transmission path may refer to a path for data transmission before the reduction calculation. For example, it may be a path for the target secondary computing card to transmit the second tensor slice to the target primary computing card. Fig.10The figure shows a schematic diagram of a tensor parallel reduction operation in a machine provided by an embodiment of the present invention. Specifically, taking the basic number of computing cards in the server as M and the parameter server logical topology as parameter server logical topology 1 as an example, according to the parameter server logical topology 1, it can be determined that the target main computing card is computing card 1, the first tensor shard (i.e., the reduction shard) is the tensor shard 1 on computing card 1, and the second tensor shard (i.e., the shard to be selected) is the tensor shard 1 on the target secondary computing card. In the reduction operation stage, computing card 1 is connected to other M-1 target secondary computing cards (i.e., computing card 2 to computing card M) through the M-1 group port, and the M-1 target secondary computing cards send their respective tensor shards 1 to computing card 1, and computing card 1 performs reduction operations on the M-1 tensor shards 1 from the target secondary computing card and the tensor shard 1 of the card in turn, so as to realize the tensor parallel reduction operation in the machine of computing card 1. At this time, the receiving direction of the interconnection port of computing card 1 is the data transmission path from the shard to be selected to the reduction shard.

[0096] The data return path may refer to a path for returning data after the reduced calculation. For example, it may be a path for the target primary computing card to transmit the reduced calculation result to the target secondary computing card. Fig.11 The figure shows a schematic diagram of a parallel broadcast operation of a tensor in a machine provided by an embodiment of the present invention. Specifically, taking the number of computing cards in the server as M and the parameter server logical topology as parameter server logical topology 1 as an example, according to Fig.10 After the reduction operation is completed, in the broadcast stage, the computing card 1 sends the reduction calculation result to the M-1 target secondary computing cards to replace the selected shards with the reduction calculation result, thus realizing the broadcast operation of the tensor parallelization in the machine. At this time, the sending direction of the interconnection port of the computing card 1 is the data return path from the reduced shard to the selected shard.

[0097] It is worth noting that in the embodiment of the present invention, the implementation method of the remaining parameter server logical topologies is consistent with the implementation method of the above-mentioned parameter server logical topology 1, and the M parameter server logical topologies can work in parallel. Each parameter server logical topology is responsible for the full reduction operation of the tensor shard with the corresponding sequence number, and the embodiment of the present invention will not be repeated here.

[0098] In an optional embodiment, if the target topology logic is an inter-machine full-protocol logic, the target full-protocol link between the first tensor shard and each second tensor shard is generated based on the preset reduction logic, including: taking the first tensor shard as the starting shard and the second tensor shard as the destination shard; constructing a rotation path from the starting shard to the destination shard based on the data flow direction in the current unidirectional ring logical topology, and combining the full rotation paths in the current unidirectional ring logical topology to generate a target full-protocol link.

[0099] The starting shard may refer to the first tensor shard in the data flow. The destination shard may refer to the remaining tensor shards in the same data flow except the starting shard. The round-robin path may refer to the transmission path of the data flow. Usually, the target full-protocol link corresponding to the current unidirectional ring logical topology starts with the starting shard and ends with the starting shard, forming a closed loop.

[0100] Fig.12 FIG. 1 is a schematic diagram of a target full protocol link corresponding to a full protocol logic between machines provided by an embodiment of the present invention. Figure 8In the forward ring logical topology in the single bidirectional ring logical topology shown, the dotted forward ring logical topology is taken as logical topology No. 1, and the No. 1 tensor block corresponding to each parallel output tensor is the current tensor block corresponding to logical topology No. 1 as an example. After processing each current tensor block by slicing based on the total number of computing cards N*M (i.e., 8) in the computing card distribution information to obtain each tensor slicing corresponding to each current tensor block, the same-sequence computing card corresponding to the current tensor slicing is used as the target main computing card, and the remaining computing cards in the current unidirectional ring logical topology are used as target secondary computing cards. For example, use server 1 computing card 1 as the target master computing card for tensor shard 1, and the remaining computing cards in the current unidirectional ring logical topology as the target slave computing cards; use server 1 computing card 2 as the target master computing card for tensor shard 2, and the remaining computing cards in the current unidirectional ring logical topology as the target slave computing cards; use server 1 computing card 4 as the target master computing card for tensor shard 3, and the remaining computing cards in the current unidirectional ring logical topology as the target slave computing cards; use server 1 computing card 3 as the target master computing card for tensor shard 4, and the remaining computing cards in the current unidirectional ring logical topology as the target slave computing cards ;Use server 2 computing card 1 as the target main computing card of tensor shard 5, and the remaining computing cards in the current one-way ring logical topology as the target secondary computing cards; use server 2 computing card 2 as the target main computing card of tensor shard 6, and the remaining computing cards in the current one-way ring logical topology as the target secondary computing cards; use server 2 computing card 4 as the target main computing card of tensor shard 7, and the remaining computing cards in the current one-way ring logical topology as the target secondary computing cards; and use server 2 computing card 3 as the target main computing card of tensor shard 8, and the remaining computing cards in the current one-way ring logical topology as the target secondary computing cards. Each computing card completes the reduction calculation of a tensor shard, and N*M computing cards go through the scattering reduction (reducescatter) and global gathering (all gather) processes on the current one-way ring logical topology, rotate 2 times in the form of slices, and complete the reduction calculation of tensor slices and the transmission of reduction results in turn. Finally, all computing cards get the complete results of the tensor shard data on their current one-way ring logical topology. At this time, the transmission path of all the protocol results in the current unidirectional ring logical topology is the target full protocol link corresponding to the inter-machine full protocol logic. It is worth noting that the generation method of the target full protocol link corresponding to other unidirectional ring logical topologies is consistent with the generation method of the target full protocol link corresponding to the dotted line forward ring logical topology, and the embodiment of the present invention does not make additional elaborations on this.

[0101] Fig.13 FIG. 1 is a schematic diagram of a rotation path provided by an embodiment of the present invention. Fig.12The target full-reduction link corresponding to the forward ring logical topology of the dashed line in the middle is that the computing card 1 of server 1 contains 8 tensor slices (a1, b1, c1, d1, e1, f1, g1, h1), the computing card 2 of server 1 contains 8 tensor slices (a2, b2, c2, d2, e2, f2, g2, h2), the computing card 4 of server 1 contains 8 tensor slices (a3, b3, c3, d3, e3, f3, g3, h3), the computing card 3 of server 1 contains (a4, b4, c4, d4, e4, f4, g4, h 4) For example, there are 8 tensor slices in total, including (a5, b5, c5, d5, e5, f5, g5, h5) on the computing card 1 of server 2, (a6, b6, c6, d6, e6, f6, g6, h6) on the computing card 2 of server 2, (a7, b7, c7, d7, e7, f7, g7, h7) on the computing card 4 of server 2, and (a8, b8, c8, d8, e8, f8, g8, h8) on the computing card 3 of server 2. Each computing card uses the tensor slice with the same number as the computing card itself as the starting slice, and the tensor slices with the same number as the computing card on other computing cards as the destination slice, and builds a rotation path from the starting slice to the destination slice according to the data flow direction in the current unidirectional ring logical topology. Exemplarily, taking the target computing card as server 1 computing card 1, tensor shard a1 is used as the starting shard, and tensor shard a2 to tensor shard a8 are used as the destination shards. According to the data flow of the dotted line forward ring logical topology, tensor shard a1 is passed to tensor shard a2 for reduction calculation, and the reduction result is passed to tensor shard a3 for further reduction calculation, and so on, until all computing cards have obtained the complete results of the tensor shard data on their current unidirectional ring logical topology after 2 rotations. At this point, the transmission path of the reduction result is the rotation path, and the rotation paths from the remaining starting shards to the destination shards are consistent with the above example. It is worth noting that in the same unidirectional ring logical topology, the rotation paths corresponding to the tensor shards on the same computing card can be completed in parallel.

[0102] The technical solution of the embodiment of the present invention obtains the computing card distribution information, parallel output tensor scale and full parallel output tensor corresponding to the target model, compares the parallel output tensor scale with a preset scale threshold, and if the parallel output tensor scale is lower than the preset scale threshold, the target topology logic is the full reduction logic within the machine, and the current computing card in the computing card distribution information is used as the target main computing card, and the remaining computing cards in the computing card distribution information are used as the target secondary computing cards; if the parallel output tensor scale exceeds the preset scale threshold, the target topology logic is the full reduction logic between machines, and based on the computing card distribution information, the current computing card in the computing card distribution information is used as the target main computing card, and the remaining computing cards in the computing card distribution information are used as the target secondary computing cards; if the parallel output tensor scale exceeds the preset scale threshold, the target topology logic is the full reduction logic between machines, and the current computing card in the computing card distribution information is used as the target main computing card, and the remaining computing cards in the computing card distribution information are used as the target secondary computing cards. The total number of computing cards in the information determines the bidirectional ring logical topology corresponding to the target model, and each parallel output tensor is processed in blocks based on the basic number of computing cards in the same server in the computing card distribution information to obtain each tensor block, and the same-order tensor blocks on each computing card in the current unidirectional ring logical topology are used as the current tensor block, and the current tensor block is processed based on the total number of computing cards to obtain each tensor shard corresponding to the current tensor block, and the same-order computing card corresponding to the current tensor shard is used as the target main computing card, and the remaining computing cards in the current unidirectional ring logical topology are used as the target secondary computing card. Further, the first tensor shard on the target main computing card and the second tensor shard on the target secondary computing card in the full parallel output tensor are determined based on the current number of the target main computing card. Finally, the target full-protocol link between the first tensor shard and each second tensor shard is generated based on the preset protocol logic, so that the target main computing card can realize the aggregate communication with the target secondary computing card based on the target full-protocol link. Due to the different requirements for parallel splitting according to models of different sizes, different topological logics can be used for different communication ranges of aggregate communication. Therefore, the problem of low efficiency of tensor parallel aggregate communication is solved, tensor parallel aggregate communication can be realized, and the efficiency of aggregate communication is improved.

[0103] Fig.14The flowchart of an optional in-machine tensor parallel aggregation communication method provided by an embodiment of the present invention is shown. Specifically, taking the case where each server in a distributed cluster contains M computing cards, and each computing card contains M groups of ports as an example, if the parallel output tensor scale is lower than the preset scale threshold, the target topology logic can be determined as the in-machine full-reduction logic. At this time, the full-reduction operation can be divided into three steps: parameter server logical topology splitting, data mapping, and parallel calculation of M parameter server logical topologies. Among them, in the parameter server logical topology splitting step, the physical topology is split into M parameter server logical topologies, one computing card corresponds to one parameter server logical topology, and under each parameter server logical topology, the computing card is used as a server node to receive data from other computing cards in the same server, complete the reduction operation, and return the result to other computing cards. In the data mapping step, it is necessary to slice each parallel output tensor according to the basic number of computing cards in the server to obtain each tensor slice. Then, according to the current number of the target main computing card, the tensor slices with the same sequence on the target main computing card are used as the first tensor slice, and the tensor slices with the same sequence on the target secondary computing card are used as the second tensor slice, that is, the first tensor slice and the second tensor slice corresponding to the target main computing card are mapped to the same parameter server logical topology. In the parallel computing step of M parameter server logical topologies, M parameter server logical topologies are calculated in parallel, and each parameter server logical topology is responsible for the corresponding reduction calculation between the first tensor slice and the second tensor slice, and the target full reduction link between the first tensor slice and each second tensor slice is generated based on the preset reduction logic, so that the target main computing card can realize the aggregate communication with the target secondary computing card according to the target full reduction link. Therefore, during the entire full reduction process, the receiving direction links and sending direction links of the M computing cards participating in tensor parallelism are working at the same time, and the interconnection ports with other computing cards are fully utilized, without any waste of ports. In addition, different parameter server logical topologies can work in parallel, improving communication efficiency.

[0104] Fig.15The flowchart of an optional inter-machine tensor parallel aggregation communication method provided by an embodiment of the present invention is shown. Specifically, a distributed cluster includes N servers, each server includes M computing cards, and each computing card includes M groups of ports. If the parallel output tensor scale exceeds the preset scale threshold, the target topology logic can be determined as the inter-machine full-reduction logic. At this time, the full-reduction operation can be divided into three steps: one-way ring logical topology splitting, data mapping, and parallel calculation of M one-way ring logical topologies. Among them, in the one-way ring logical topology splitting step, each server is split into M / 2 bidirectional ring logical topologies based on the fully connected topology, each bidirectional ring logical topology contains all the nodes in the server, and the M / 2 ring logical topologies correspond to M one-way ring logical topologies. In the data mapping step, the parallel output tensor on each computing card is first divided into M tensor blocks, each tensor block is mapped to a one-way ring logical topology, and the computing card on this one-way ring logical topology completes the full reduction operation of the tensor block. In the parallel computing steps of M unidirectional ring logical topologies, M unidirectional ring logical topologies work in parallel, and each unidirectional ring logical topology is responsible for the reduction calculation of the corresponding current tensor block. Each unidirectional ring logical topology connects N*M computing cards in series, and further divides the parallel output tensor into 1 / (N*M) tensor slices. Each tensor slice is transmitted in turn and performs the reduction operation. Finally, each computing card has the reduction result of each tensor slice, completing the scattering reduction stage. After that, the reduction result is transmitted in turn, and each computing card collects the reduction result of each tensor slice until the reduction result of the complete tensor slice is obtained on the unidirectional ring logical topology, completing the global collection stage. Therefore, in the entire full reduction process, the receiving direction link and the sending direction link of the N*M computing cards participating in the tensor parallel are working at the same time, corresponding to the data processing of the forward ring logical topology and the reverse ring logical topology respectively, and all interconnection ports are fully utilized at the same time, without port waste, and different unidirectional ring logical topologies can work in parallel, improving communication efficiency.

[0105] Embodiment 3

[0106] Fig.16 The schematic diagram of the structure of a tensor parallel-based aggregate communication device provided in the third embodiment of the present invention. Fig.16 As shown, the device includes: a data acquisition module 310, a first determination module 320, a second determination module 330, a third determination module 340 and an aggregate communication module 350;

[0107] The data acquisition module 310 is used to obtain the computing card distribution information, parallel output tensor scale and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model;

[0108] A first determination module 320, configured to determine a target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold;

[0109] A second determination module 330 is used to determine a target primary computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information;

[0110] A third determination module 340 is used to determine the first tensor slice on the target main computing card and the second tensor slice on the target secondary computing card in the full parallel output tensor based on the current serial number of the target main computing card;

[0111] The aggregate communication module 350 is used to generate a target full protocol link between the first tensor slice and each second tensor slice based on a preset protocol logic, so that the target main computing card can achieve aggregate communication with the target secondary computing card based on the target full protocol link.

[0112] The technical solution of the embodiment of the present invention determines the target topology logic through the parallel output tensor scale corresponding to the target model and the preset scale threshold; then, the target main computing card and the target secondary computing card are determined based on the target topology logic and the computing card distribution information corresponding to the target model, and the first tensor shard on the target main computing card and the second tensor shard on the target secondary computing card in the full parallel output tensor corresponding to the target model are determined based on the current number of the target main computing card; finally, the target full protocol link between the first tensor shard and each second tensor shard is generated based on the preset protocol logic, so that the target main computing card can realize the aggregate communication with the target secondary computing card based on the target full protocol link. Due to the different requirements for parallel splitting according to models of different sizes, different topology logics can be used according to different communication ranges of aggregate communication. Therefore, the problem of low efficiency of tensor parallel aggregate communication is solved, tensor parallel aggregate communication can be realized, and the efficiency of aggregate communication is improved.

[0113] Optionally, the first determination module 320 can be specifically used for: if the parallel output tensor scale is lower than a preset scale threshold, the target topology logic is the intra-machine full reduction logic; if the parallel output tensor scale exceeds the preset scale threshold, the target topology logic is the inter-machine full reduction logic.

[0114] Optionally, if the target topology logic is an in-machine full-protocol logic, the second determination module 330 can be specifically used to: use the current computing card in the computing card distribution information as the target main computing card, and use the remaining computing cards in the computing card distribution information as target secondary computing cards; wherein the target main computing card and the target secondary computing card are integrated in the same server.

[0115] Optionally, if the target topology logic is the in-machine full protocol logic, the aggregate communication module 350 can be specifically used to: use the first tensor shard as the protocol shard and the second tensor shard as the selected shard; construct a data transmission path from the selected shard to the protocol shard and a data return path from the protocol shard to the selected shard based on the full protocol computing logic, and combine the data transmission path and the data return path to generate a target full protocol link.

[0116] Optionally, if the target topology logic is an inter-machine full-protocol logic, the second determination module 330 may be specifically configured to:

[0117] Determine a bidirectional ring logical topology corresponding to the target model based on the total amount of computing cards in the computing card distribution information; wherein each bidirectional ring logical topology includes two unidirectional ring logical topologies, and each unidirectional ring logical topology includes all the computing cards in the computing card distribution information;

[0118] Based on the basic number of computing cards in the same server in the computing card distribution information, each parallel output tensor is divided into blocks to obtain each tensor block, and the same-order tensor blocks on each computing card in the current unidirectional ring logical topology are used as the current tensor blocks;

[0119] The current tensor block is processed based on the total amount of shards of the computing cards to obtain the tensor shards corresponding to the current tensor block, and the computing cards of the same sequence corresponding to the current tensor shards are used as the target main computing cards, and the remaining computing cards in the current unidirectional ring logical topology are used as the target secondary computing cards.

[0120] Optionally, if the target topology logic is an inter-machine full-protocol logic, the aggregate communication module 350 can be specifically used to: use the first tensor shard as the starting shard and the second tensor shard as the destination shard; build a rotation path from the starting shard to the destination shard based on the data flow direction in the current unidirectional ring logical topology, and combine the full rotation paths in the current unidirectional ring logical topology to generate a target full-protocol link.

[0121] Optionally, the number of port groups of the target primary computing card and the number of port groups of the target secondary computing card both have a preset correlation with the basic number of computing cards in the same server in the computing card distribution information.

[0122] The tensor parallel-based aggregate communication device provided in the embodiment of the present invention can execute the tensor parallel-based aggregate communication method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0123] Embodiment 4

[0124] Fig.17A schematic diagram of an electronic device 410 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0125] like Fig.17 As shown, the electronic device 410 includes at least one processor 420, and a memory connected to the at least one processor 420 in communication, such as a read-only memory (ROM) 430, a random access memory (RAM) 440, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 420 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 430 or the computer program loaded from the storage unit 490 to the random access memory (RAM) 440. In RAM440, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 420, ROM430 and RAM440 are connected to each other through a bus 450. An input / output (I / O) interface 460 is also connected to the bus 450.

[0126] Multiple components in the electronic device 410 are connected to the I / O interface 460, including: an input unit 470, such as a keyboard, a mouse, etc.; an output unit 480, such as various types of displays, speakers, etc.; a storage unit 490, such as a disk, an optical disk, etc.; and a communication unit 4100, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 4100 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0127] The processor 420 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 420 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 420 performs the various methods and processes described above, such as the tensor-parallel based aggregate communication method.

[0128] The method includes:

[0129] Obtain the computing card distribution information, parallel output tensor scale, and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model;

[0130] Determine the target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold;

[0131] Determine a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information;

[0132] Determine, based on the current serial number of the target main computing card, a first tensor slice on the target main computing card and a second tensor slice on the target secondary computing card in the full parallel output tensor;

[0133] A target full protocol link is generated between the first tensor slice and each second tensor slice based on the preset protocol logic, so that the target main computing card can realize aggregate communication with the target secondary computing card according to the target full protocol link.

[0134] In some embodiments, the tensor-parallel-based aggregate communication method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 490. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 410 via the ROM 430 and / or the communication unit 4100. When the computer program is loaded into the RAM 440 and executed by the processor 420, one or more steps of the tensor-parallel-based aggregate communication method described above may be performed. Alternatively, in other embodiments, the processor 420 may be configured to execute the tensor-parallel-based aggregate communication method by any other appropriate means (e.g., by means of firmware).

[0135] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0137] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0140] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0141] The embodiment of the present application also discloses a computer program product, which includes a computer program, and when the computer program is executed by a processor, the tensor-parallel-based aggregate communication method provided in any embodiment of the present application is implemented. The program product and the tensor-parallel-based aggregate communication method disclosed in each embodiment of the present application belong to the same inventive concept, so they are not described here.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A tensor-based parallel aggregation communication method, characterized in that: include: Obtain the computing card distribution information, parallel output tensor scale, and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model; Determine the target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold; Determine a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information; Determine, based on the current serial number of the target main computing card, a first tensor slice on the target main computing card and a second tensor slice on the target secondary computing card in the full parallel output tensor; A target full protocol link is generated between the first tensor slice and each second tensor slice based on the preset protocol logic, so that the target main computing card can realize aggregate communication with the target secondary computing card according to the target full protocol link.

2. The method according to claim 1, characterized in that The determining the target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold comprises: If the parallel output tensor scale is lower than a preset scale threshold, the target topology logic is an in-machine full reduction logic; If the parallel output tensor scale exceeds a preset scale threshold, the target topology logic is an inter-machine full reduction logic.

3. The method according to claim 2, characterized in that If the target topology logic is an in-machine full-reduction logic, determining the target main computing card and the target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information includes: The current computing card in the computing card distribution information is used as the target main computing card, and the remaining computing cards in the computing card distribution information are used as target secondary computing cards; wherein the target main computing card and the target secondary computing card are integrated in the same server.

4. The method according to claim 2, characterized in that: If the target topology logic is an in-machine full-reduction logic, generating a target full-reduction link between the first tensor slice and each second tensor slice based on the preset reduction logic includes: The first tensor slice is used as a reduced slice, and the second tensor slice is used as a to-be-selected slice; According to the full protocol computing logic, a data transmission path from the selected shard to the protocol shard and a data return path from the protocol shard to the selected shard are constructed, and the data transmission path and the data return path are combined to generate a target full protocol link.

5. The method according to claim 2, characterized in that: If the target topology logic is an inter-machine full-protocol logic, determining a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information includes: Determine a bidirectional ring logical topology corresponding to the target model based on the total amount of computing cards in the computing card distribution information; wherein each bidirectional ring logical topology includes two unidirectional ring logical topologies, and each unidirectional ring logical topology includes all the computing cards in the computing card distribution information; Based on the basic number of computing cards in the same server in the computing card distribution information, each parallel output tensor is divided into blocks to obtain each tensor block, and the same-order tensor blocks on each computing card in the current unidirectional ring logical topology are used as the current tensor blocks; The current tensor block is processed based on the total amount of shards of the computing cards to obtain the tensor shards corresponding to the current tensor block, and the computing cards of the same sequence corresponding to the current tensor shards are used as the target main computing cards, and the remaining computing cards in the current unidirectional ring logical topology are used as the target secondary computing cards.

6. The method according to claim 2, characterized in that If the target topology logic is an inter-machine full-reduction logic, generating a target full-reduction link between the first tensor shard and each second tensor shard based on the preset reduction logic includes: The first tensor slice is used as the starting slice and the second tensor slice is used as the destination slice; A rotation path from the starting shard to the destination shard is constructed according to the data flow direction in the current unidirectional ring logical topology, and all the rotation paths in the current unidirectional ring logical topology are combined to generate a target full-protocol link.

7. The method according to claim 1, characterized in that The number of port groups of the target primary computing card and the number of port groups of the target secondary computing card are both in a preset correlation relationship with the basic number of computing cards in the same server in the computing card distribution information.

8. A tensor-based parallel aggregation communication device, characterized in that: include: A data acquisition module is used to obtain the computing card distribution information, parallel output tensor scale and full parallel output tensor corresponding to the target model; wherein each parallel output tensor is stored on each computing card corresponding to the target model; A first determination module, configured to determine a target topology logic corresponding to the target model based on the parallel output tensor scale and a preset scale threshold; A second determination module is used to determine a target main computing card and a target secondary computing card corresponding to the target model based on the target topology logic and the computing card distribution information; A third determination module is used to determine the first tensor slice on the target main computing card and the second tensor slice on the target secondary computing card in the full parallel output tensor based on the current serial number of the target main computing card; The aggregate communication module is used to generate a target full protocol link between the first tensor slice and each second tensor slice based on a preset protocol logic, so that the target main computing card can realize aggregate communication with the target secondary computing card based on the target full protocol link.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the tensor parallel-based aggregate communication method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the tensor parallel-based aggregate communication method according to any one of claims 1 to 7 when executed.

Images