A processor unit set communication method, a processor system and an acceleration card

Abstract

The application discloses a processor unit set communication method, a processor system and an acceleration card. The method comprises the following steps: dividing a target processor unit array performing a calculation task into m first processor unit sub-arrays and n second processor unit sub-arrays, the size of the target processor unit array is 2m*2n, the size of the first processor unit sub-array is 2*2n, and the size of the second processor unit sub-array is 2m*2; controlling the first processor unit sub-array to perform set communication based on a bidirectional ring algorithm to perform the calculation task; and controlling the second processor unit sub-array to perform set communication based on the bidirectional ring algorithm to perform the calculation task.

Description

Technical Field

[0001] The present invention relates to the field of processor data communication technology, and more particularly, to a processor unit aggregate communication method, a processor system, and an accelerator card. Background Art

[0002] With the rapid development of deep learning and artificial intelligence (AI) technologies, the parameter scale of AI models is becoming increasingly large. The computational and memory bandwidth resources of traditional AI processors are no longer sufficient to meet the training and inference requirements of AI models with these large parameter scales. To address the bottlenecks of large-model training, such as limited computational power and memory bandwidth, a variety of new processor architectures have been proposed in academia and industry. Among them, wafer-level processor chips, due to their unique architectural design, have far superior computational power and memory bandwidth than traditional AI chips. This makes them stand out among many new processor architectures and has been widely researched and applied in the industry.

[0003] Wafer-scale processor chips consist of an on-chip interconnect network and a large number of processor cores (PEs). These PE cores are arranged in an array, with each PE core forming a bidirectional interconnect with adjacent PE cores using a 2D mesh structure, thus forming a wafer-level processor chip. The PE core array in a wafer-level processor chip is typically large, often containing tens of thousands or even more PE cores. Therefore, for processors with large arrays of PE cores, optimizing the communication efficiency between the internal processor cores is particularly important.

[0004] Currently, common communication algorithms in AI clusters are mainly divided into tree-based algorithms and ring-based algorithms. Because PE cores are directly connected only to adjacent PE cores, access to non-adjacent PE cores requires relay routing via other PE cores. Therefore, using tree-based algorithms for collective communication is prone to link contention, leading to severe network congestion on some paths and affecting communication efficiency. Therefore, for large-scale PE core arrays, ring algorithms are often used to achieve collective communication between PE cores.

[0005] Collective communication refers to the efficient data exchange and collaborative work between PE cores based on specific rules. Therefore, the communication efficiency of the collective communication operator will directly affect the computing efficiency of the AI ​​cluster.

[0006] The large-scale PE array causes the time consumption of the collective communication operation between the cores to increase sharply. For a 2D Mesh array, the existing collective communication algorithm is basically implemented by using a ring algorithm. The ring algorithm has obvious advantages in a small or medium scale node size, and has small transmission data and no bottleneck. However, once the node size becomes large, a large number of nodes form a very long ring, and the time consumption of the collective communication increases linearly, thereby reducing the communication efficiency. Therefore, the prior art has the following technical problems: 1. The existing ring-based collective communication algorithm performs the collective communication operation in a single direction for all the computing nodes in the ring. The bidirectional connection between the computing nodes in the Mesh network topology is not utilized, and the bandwidth utilization of the Mesh network is insufficient. 2. The processor cores in the wafer-level processor are numerous, and the array size is large. The time consumption of the collective communication of the traditional ring algorithm increases linearly with the PE array size, and cannot meet the demand for high collective communication efficiency. SUMMARY

[0007] In view of the deficiencies of the prior art, the present application provides a processor unit collective communication method, a processor system and an acceleration card.

[0008] According to one aspect of the present application, a processor unit collective communication method is provided, comprising:

[0009] dividing a target processor unit array performing a computing task into m first processor unit sub-arrays and n second processor unit sub-arrays, the size of the target processor unit array being 2m x 2n, the size of the first processor unit sub-array being 2 x 2n, and the size of the second processor unit sub-array being 2m x 2;

[0010] controlling the first processor unit sub-array to perform collective communication based on a bidirectional ring algorithm to perform the computing task; and

[0011] controlling the second processor unit sub-array to perform collective communication based on the bidirectional ring algorithm to perform the computing task.

[0012] Optionally, the method further comprises:

[0013] in a case where at least one of the number of rows and the number of columns of the processor unit array allocated for the computing task is odd, performing reduction processing on the processor unit array allocated for the computing task according to a preset reduction rule to form the target processor unit array;

[0014] in a case where the first processor unit sub-array and the second processor unit sub-array complete the collective communication, sending the computing result of the computing task to the processor units other than the target processor unit array in the processor unit array allocated for the computing task.

[0015] Optionally, in the case that the number of columns of the array of processor units assigned to the computing task is 2n+1, each processor unit of a certain column is controlled to send data of the computing task to a processor unit of a neighboring column in the same row, so that the processor unit of the neighboring column in the same row processes the data of the computing task with the processor unit.

[0016] In the case that the number of rows of the array of processor units assigned to the computing task is 2m+1, each processor unit of a certain row is controlled to send data of the computing task to a processor unit of a neighboring row in the same column, so that the processor unit of the neighboring row in the same column processes the data of the computing task with the processor unit.

[0017] Optionally, in the case that the number of columns of the array of processor units assigned to the computing task is 2n+1, the certain column is the first column or the (2n+1)th column.

[0018] In the case that the number of rows of the array of processor units assigned to the computing task is 2m+1, the certain column is the first row or the (2m+1)th row.

[0019] Optionally, the first array of processor units is controlled to perform collective communication based on a bidirectional ring algorithm, including:

[0020] The first array of processor units is controlled to perform collective communication based on a bidirectional ring algorithm, including:

[0021] The first array of processor units is controlled to perform collective communication based on a bidirectional ring algorithm, including:

[0022] The first array of processor units is controlled to perform collective communication based on a bidirectional ring algorithm, including:

[0023] Optionally, the first array of processor units is controlled to perform collective communication based on a bidirectional ring algorithm, including:

[0024] controlling a fifth group of processor units in the first subarray of processor units to perform collective communication based on a unidirectional ring algorithm, and controlling a sixth group of processor units in the first subarray of processor units to perform collective communication based on a unidirectional ring algorithm, the first subarray of processor units being divided into the fifth group of processor units and the sixth group of processor units in a manner of processor unit column intervals, the fifth group of processor units and the sixth group of processor units using unidirectional ring algorithms in opposite directions;

[0025] controlling the second subarray of processor units to perform collective communication based on a bidirectional ring algorithm, including:

[0026] controlling a seventh group of processor units in the second subarray of processor units to perform collective communication based on a unidirectional ring algorithm, and controlling an eighth group of processor units in the second subarray of processor units to perform collective communication based on a unidirectional ring algorithm, the second subarray of processor units being divided into the seventh group of processor units and the eighth group of processor units in a manner of processor unit intervals, the seventh group of processor units and the eighth group of processor units using unidirectional ring algorithms in opposite directions.

[0027] Optionally, the m first subarrays of processor units perform collective communication in parallel; the n second subarrays of processor units perform collective communication in parallel.

[0028] Optionally, controlling the second subarray of processor units to perform collective communication based on a bidirectional ring algorithm, including:

[0029] In a case where the m first subarrays of processor units complete collective communication, controlling the second subarray of processor units to perform collective communication based on a bidirectional ring algorithm.

[0030] According to yet another aspect of the present application, there is provided a processor system comprising a compiler and an array of processor units, the compiler being configured to perform the method according to any of the above aspects of the present application.

[0031] According to yet another aspect of the present application, there is provided an accelerator card comprising a compiler and an array of processor units, the compiler being configured to perform the processor system according to the above aspect of the present application.

[0032] According to yet another aspect of the present application, there is provided a server comprising a compiler and an array of processor units, the compiler being configured to perform the accelerator card according to the above aspect of the present application.

[0033] Therefore, the application provides a processor unit set communication method, based on the characteristics of the two-way connection of the computing nodes in the 2D Mesh network topology, a two-way ring set communication method is provided, which can improve the utilization rate of the Mesh network bandwidth. A hierarchical communication division strategy is provided, which can effectively solve the shortcoming that the conventional ring algorithm cannot be applied to large PE arrays. Through hierarchical division of the nodes in the PE array, the PE array can be divided into multiple small arrays, and the PE cores in the small arrays can be formed into rings according to the rules, the number of the PE cores in the rings is obviously reduced, and the rings can also perform set communication operations in parallel. Therefore, the set communication efficiency of the large PE array can be obviously improved. BRIEF DESCRIPTION OF DRAWINGS

[0034] The exemplary embodiments of the present application will be more fully understood from the following description of example embodiments, taken together with the drawings, in which:

[0035] Figure 1 is a flowchart of the hierarchical processor unit set communication optimization method based on the two-way ring provided by an exemplary embodiment of the application;

[0036] Figure 2 is a schematic diagram of the implementation of the two-way ring-A method provided by an exemplary embodiment of the application;

[0037] Figure 3 is a schematic diagram of the implementation of the two-way ring-B method provided by an exemplary embodiment of the application;

[0038] Figure 4 is a schematic diagram of the reduction processing of the array with an odd number of rows or columns provided by an exemplary embodiment of the application;

[0039] Figure 5 is a schematic diagram of the hierarchical communication division strategy provided by an exemplary embodiment of the application. DETAILED DESCRIPTION

[0040] Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and are not all the embodiments of the present application, and it should be understood that the present application is not limited to the described example embodiments.

[0041] It should be noted that: the relative arrangement, numerical expression and numerical value of the components and steps set forth in these embodiments do not limit the scope of the present application, unless otherwise specified.

[0042] Those skilled in the art can understand that the terms "first", "second", etc. in the embodiments of the present application are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they represent the inevitable logical sequence between them.

[0043] It should also be understood that, in the embodiments of the application, “multiple” can refer to two or more, and “at least one” can refer to one, two or more.

[0044] It should also be understood that, for any component, data or structure mentioned in the embodiments of the application, one or more can be generally understood without explicit limitation or in the context of the opposite implication.

[0045] In addition, the term “and / or” in the present application is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in the present application generally represents an “or” relationship between the front and rear associated objects.

[0046] It should also be understood that the description of each embodiment of the present application emphasizes the differences between each embodiment, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0047] At the same time, it should be understood that, for the convenience of description, the size of each part shown in the drawings is not drawn according to the actual proportional relationship.

[0048] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application or its application or use.

[0049] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0050] It should be noted that: similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in the subsequent drawings.

[0051] The scheme of the present application runs on AI chips and computing power chips, and the product form can be these chips and products equipped with the chips.

[0052] In an embodiment of the present application, Figure 1 The flowchart of the processor unit set communication method is shown. The embodiment can be applied to AI chips and computing power chips, such as Figure 1 As shown in the figure, the processor unit set communication method 100 includes the following steps:

[0053] Step 101, divide the target processor unit array performing a computing task into m first processor unit sub-arrays and n second processor unit sub-arrays, the size of the target processor unit array is 2m*2n, the size of the first processor unit sub-array is 2*2n, and the size of the second processor unit sub-array is 2m*2;

[0054] Step 102, control the first processor unit sub-array to perform collective communication based on a bidirectional ring algorithm to execute the computing task; and

[0055] Step 103, control the second processor unit sub-array to perform collective communication based on the bidirectional ring algorithm to execute the computing task.

[0056] Specifically, to solve the technical problems existing in the prior art, the application provides a processor unit collective communication method, which is specifically as follows:

[0057] Wherein, the bidirectional ring algorithm includes two forms, namely, a bidirectional ring-A method and a bidirectional ring-B method.

[0058] 1) Bidirectional ring-A method: for a 2*N or N*2 PE array (i.e. processor unit array, hereinafter referred to as PE array), the two types of arrays are equivalent; taking 2*N as an example, first, number the PEs in the array in a clockwise direction, numbered 0~2N-1; all PEs numbered with even numbers form a forward ring A, in the forward ring A, the data flow of collective communication flows in a clockwise direction; all PEs numbered with odd numbers form a reverse ring A', in the reverse ring A', the data flow of collective communication flows in a counterclockwise direction. Since the adjacent PEs in the array are bidirectionally connected, the data flow of the forward ring and the reverse ring will not conflict with each other, and the collective communication operation can be performed in parallel. Figure 2

[0059] All PEs in the forward ring A perform all-reduce operation in a clockwise direction using a conventional ring algorithm; all PEs in the reverse ring A' perform all-reduce operation in a counterclockwise direction using a conventional ring algorithm.

[0060] 2) Bidirectional ring-B method: for a 2*2M or 2M*2 PE array, the two types of arrays are equivalent; taking a 2M*2 PE array as an example, first, number the PEs in the array in a clockwise direction, numbered 0~2M-1; all PEs numbered with even numbers form a forward ring B, in the forward ring B, the data flow of collective communication flows in a clockwise direction; all PEs numbered with odd numbers form a reverse ring B', in the reverse ring B', the data flow of collective communication flows in a counterclockwise direction. Since the adjacent PEs in the array are bidirectionally connected, the data flow of the forward ring and the reverse ring will not conflict with each other, and the collective communication operation can be performed in parallel. Figure 3 ​As shown, all PE cores in even-numbered rows form a forward ring B, where collective communication data flows clockwise. All PE cores in odd-numbered rows form a reverse ring B', where collective communication data flows counterclockwise. Because adjacent PE cores in the array are bidirectionally connected, data flows in forward and reverse rings B' do not conflict, allowing collective communication operations to proceed in parallel.

[0061] All PE cores in the forward ring B use the conventional ring algorithm to perform all-reduce operations in a clockwise direction; all PE cores in the reverse ring B' use the conventional ring algorithm to perform all-reduce operations in a counterclockwise direction.

[0062] In one embodiment of the present application, a hierarchical communication partitioning strategy for collective communication based on a bidirectional ring algorithm includes:

[0063] 1) The hierarchical communication partitioning strategy is applicable to PE arrays with even numbers of rows and columns (i.e., 2m*2n). If the original array has odd numbers of rows or columns, the array needs to be trimmed to 2m*2n first. Figure 4 As shown, when the column is an odd number, the data of each PE core in the 2n+1th column is first sent to the PE core in the adjacent 2nth column, and then the data of the PE core in the 2nth column is reduced, and the processing result is saved in the corresponding PE core in the 2nth column; when the row is an odd number, the data of each PE core in the 2m+1th row is first sent to the PE core in the adjacent 2mth row, and then the data of the PE core in the 2mth row is reduced, and the processing result is saved in the corresponding PE core in the 2mth row; when both the row and the column are odd numbers, the row can be reduced first, and then the column can be reduced to obtain an even number of processor unit arrays.

[0064] 2) Divide the PE array with an array size of 2m*2n into two communication layers, the first communication layer and the second communication layer. Figure 5 As shown, where:

[0065] First communication level: Split the 2m*2n PE array into rows. The splitting unit is a 2*2n array, denoted as splitting unit A. A total of m splitting units A can be obtained.

[0066] Second communication level: Split the 2m*2n PE array into columns. The split unit is a 2m*2 array, denoted as split unit B. A total of n split units B can be obtained.

[0067] When the 2m*2n array performs all-reduce communication, the operation steps can be divided into two steps: first, the all-reduce operation is completed in the first communication level, wherein m split units A perform all-reduce operation in parallel, and each split unit A performs collective communication using a bidirectional ring-A method; then, the all-reduce operation is completed in the second communication level, wherein n split units B perform all-reduce operation in parallel, and each split unit B performs collective communication using a bidirectional ring-B method.

[0068] 3) When the rows or columns of the original array are both even numbers, the all-reduce operation of the original array is completed, and each processor unit obtains the reduction data; when the rows or columns of the original array are odd numbers, the all-reduce result (i.e., the intermediate result of calculation) needs to be sent to 2m+1 rows or 2n+1 columns; specifically, when the column is odd, the reduction data of each PE core in the 2n-th column is sent to the adjacent 2n+1-th column PE core; when the row is odd, the reduction data of each PE core in the 2m-th row is sent to the adjacent 2m+1-th row PE core; when the row and the column are both odd, the original processor unit array is recovered according to the rules of reduction processing, so that each processor unit of the original processor unit array obtains the reduction data.

[0069] Therefore, the application is suitable for large-scale PE arrays, and the application can effectively improve the communication efficiency of the all-reduce collective communication of the PE array; compared with the conventional ring all-reduce, the collective communication efficiency is obviously improved. The main application scenarios include wafer-level processor chips, wafer-level processor boards and wafer-level processor servers and clusters.

[0070] In an embodiment of the application, a task scheduler allocates a target computing task to a specific processor unit array, and the allocation result is given to a compiler (the compiler can perceive even-scale and odd-scale configurations (0 / 1)), the target computing task is split into each PE core in the array, according to the size of the actual used processor array, the selected processor array is divided into a first communication level and a second communication level, and in the running process of the computing task, the collective communication between the PE cores can be performed according to the hierarchical collective communication method provided by the application.

[0071] Therefore, based on the feature of the bidirectional connection of the computing nodes in the 2D Mesh network topology, the application proposes a bidirectional ring set communication method, which can improve the utilization rate of the Mesh network bandwidth. A hierarchical communication division strategy is proposed, which can effectively solve the shortcoming that the conventional ring algorithm cannot be applied to large PE arrays. Through hierarchical division of the nodes in the PE array, the PE array can be divided into multiple small arrays, and the PE cores in the small arrays are formed into rings according to rules. The number of the PE cores in the rings is obviously reduced, and the rings can also perform set communication operations in parallel. Therefore, the set communication efficiency of the large PE array can be obviously improved.

[0072] In another embodiment of the application, a processor system is provided, comprising a compiler and an array of processor units, the compiler being configured to perform the method of any of the above-mentioned aspects of the application.

[0073] In another embodiment of the application, an accelerator card is provided, comprising a compiler and an array of processor units, the compiler being configured to perform the processor system of the above-mentioned aspects of the application.

[0074] In another embodiment of the application, a server is provided, comprising a compiler and an array of processor units, the compiler being configured to perform the accelerator card of the above-mentioned aspects of the application.

[0075] The above description has been given for the purpose of illustration and description. Furthermore, this description does not intend to limit the embodiments of the application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of skill in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method of communicating among a set of processor units, the method comprising: The method comprises: dividing a target processor unit array performing a computing task into m first processor unit sub-arrays and n second processor unit sub-arrays, the size of the target processor unit array being 2m×2n, the size of the first processor unit sub-arrays being 2×2n, and the size of the second processor unit sub-arrays being 2m×2; controlling the first processor unit sub-arrays to perform collective communication based on a bidirectional ring algorithm to execute the computing task; controlling the second processor unit sub-arrays to perform collective communication based on a bidirectional ring algorithm to execute the computing task; in a case where at least one of the number of rows and the number of columns of the processor unit array allocated for the computing task is odd, performing reduction processing on the processor unit array allocated for the computing task according to a preset reduction rule to form the target processor unit array; in a case where the first processor unit sub-arrays and the second processor unit sub-arrays complete collective communication, sending the computing result of the computing task to a processor unit other than the target processor unit array in the processor unit array allocated for the computing task; controlling the first processor unit sub-arrays to perform collective communication based on a bidirectional ring algorithm, comprising: controlling a first processor unit group in the first processor unit sub-arrays to perform collective communication based on a unidirectional ring algorithm, and controlling a second processor unit group in the first processor unit sub-arrays to perform collective communication based on a unidirectional ring algorithm, the first processor unit sub-arrays being divided into the first processor unit group and the second processor unit group in a processor unit interval manner, and the first processor unit group and the second processor unit group using unidirectional ring algorithms in opposite directions; controlling the second processor unit sub-arrays to perform collective communication based on a bidirectional ring algorithm, comprising: controlling a third processor unit group in the second processor unit sub-arrays to perform collective communication based on a unidirectional ring algorithm, and controlling a fourth processor unit group in the second processor unit sub-arrays to perform collective communication based on a unidirectional ring algorithm, the second processor unit sub-arrays being divided into the third processor unit group and the fourth processor unit group in a processor unit row interval manner, and the third processor unit group and the fourth processor unit group using unidirectional ring algorithms in opposite directions.

2. The method according to claim 1, wherein: in a case where the number of columns of the processor unit array allocated for the computing task is 2n+1, controlling each processor unit in a specific column to send data of the computing task to a processor unit in a neighboring column in the same row, so that the processor unit in the neighboring column in the same row performs reduction processing on the data of the computing task with the processor unit; in a case where the number of rows of the processor unit array allocated for the computing task is 2m+1, controlling each processor unit in a specific row to send data of the computing task to a processor unit in a neighboring row in the same column, so that the processor unit in the neighboring row in the same column performs reduction processing on the data of the computing task with the processor unit. ​ 3. The method of claim 2, wherein: in a case where a column number of the processor element array allocated for the computing task is 2n+1, the specific column is a first column or a 2n+1th column; in a case where a row number of the processor element array allocated for the computing task is 2m+1, the specific column is a first row or a 2m+1th row.

4. The method of claim 1, wherein, controlling the first processor element sub-array to perform collective communication based on a bidirectional ring algorithm, including: controlling a fifth processor element group in the first processor element sub-array to perform collective communication based on a unidirectional ring algorithm and controlling a sixth processor element group in the first processor element sub-array to perform collective communication based on a unidirectional ring algorithm, the first processor element sub-array being divided into the fifth processor element group and the sixth processor element group in a manner of processor element column interval, the fifth processor element group and the sixth processor element group using unidirectional ring algorithms in opposite directions; controlling the second processor element sub-array to perform collective communication based on a bidirectional ring algorithm, including: controlling a seventh processor element group in the second processor element sub-array to perform collective communication based on a unidirectional ring algorithm and controlling an eighth processor element group in the second processor element sub-array to perform collective communication based on a unidirectional ring algorithm, the second processor element sub-array being divided into the seventh processor element group and the eighth processor element group in a manner of processor element interval, the seventh processor element group and the eighth processor element group using unidirectional ring algorithms in opposite directions.

5. The method according to claim 1 or 4, characterized in that, m of the first processor element sub-arrays perform collective communication in parallel; n of the second processor element sub-arrays perform collective communication in parallel.

6. The method according to claim 1 or 4, characterized in that, the controlling the second processor element sub-array to perform collective communication based on a bidirectional ring algorithm, including: in a case where the m of the first processor element sub-arrays complete collective communication, controlling the second processor element sub-array to perform collective communication based on a bidirectional ring algorithm.

7. A processor system, characterized by a compiler and a processor element array, the compiler being configured to perform the method of any one of claims 1 to 6.

8. An accelerator card, comprising: a processor system of claim 7.

9. A server, characterized by an acceleration card of claim 8.

Images