N-dimensional tensor segmentation method, device, equipment, medium and product of high-computing-power chip computing architecture

Abstract

The invention relates to an N-dimensional tensor segmentation method and device for a high-computing-power chip computing architecture, equipment, a medium and a product. The method comprises the following steps: in a training process of a Transform network, splitting an input matrix in the Transform network into N second column input vectors according to a column dimension; splitting a weight matrix in the Transform network into 2N column weight elements according to a column dimension and a row dimension; the splitting frequency of the row dimension is N-1; deploying a group of corresponding second column input vectors and two column weight elements on each computing device in the N different computing devices; on the basis of a set communication mode, the sum of products of each second column input vector and two column weight elements is obtained through two rounds of calculation; and integrating the product sum of each second column input vector and the two column weight elements to obtain an output matrix. By adopting the method, the tensor storage and communication overhead of Transform network training is effectively reduced.

Description

Technical Field

[0001] This application relates to the field of model computing technology, and in particular to an N-dimensional tensor segmentation method, apparatus, device, medium and product for a high-computing-power chip computing architecture. Background Technology

[0002] In recent years, neural network models have developed rapidly and achieved remarkable results in fields such as natural language processing and computer vision. However, this has come at the cost of a huge number of parameters and a massive amount of computation required to train the network. Therefore, using distributed computing to distribute the neural network parameters and corresponding calculations across different devices has become the best choice to improve computational efficiency.

[0003] For Transformer networks, whose architecture primarily involves linear matrix multiplication, effectively partitioning the tensor of matrix multiplication to simultaneously satisfy dimensionality consistency during backpropagation and gradient updates has become a focus of current academic research. Currently, an N-dimensional tensor partitioning scheme for high-performance chip computing architectures has been proposed, uniformly partitioning the computational tensor along three dimensions. This forces different devices to simultaneously execute diagonal and neighboring ring communication modes to receive the next round of data. In this scenario, diagonally transmitted data always arrives at the computing device later than horizontally transmitted data, and this imbalance in data transmission leads to reduced computational efficiency.

[0004] Therefore, improving computational efficiency has become an urgent technical problem to be solved. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, device, storage medium, and program product for N-dimensional tensor segmentation based on streaming processing that can improve computational efficiency in response to the above-mentioned technical problems.

[0006] Firstly, this application provides an N-dimensional tensor partitioning method for a high-performance computing chip architecture, including:

[0007] During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension;

[0008] The weight matrix in the Transformer network is split into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0009] On each of the N different computing devices, a set of corresponding second column input vectors and two column weight elements are deployed.

[0010] Based on the set communication method, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation;

[0011] The output matrix is ​​obtained by combining the product of each second column input vector with the weight elements of the two columns.

[0012] In one embodiment, the above-mentioned set communication-based method, after two rounds of calculation, obtains the product of each second column input vector and the weight elements of the two columns, including:

[0013] For any computing device, in the first round of computation, obtain the product of the second column input vector and the weight element of any column in the computing device;

[0014] In the second round of calculation, while obtaining the sum of the products of the second column input vector and the weight element of another column in the computing device, the product of the second column input vector and any column weight element in each computing device is also obtained.

[0015] Obtain the sum of the products of the second column input vector and the weight elements of the other column in each computing device.

[0016] In one embodiment, the method further includes:

[0017] The output matrix is ​​split into N row output vectors according to the row dimension;

[0018] The transpose of the weight matrix in the Transformer network is split into N column weight transpose vectors according to the column dimension;

[0019] On each of the N different computing devices, deploy a set of corresponding row output vectors and column weight transpose vectors.

[0020] Based on the set communication method, after N rounds of calculation, the product of each row output vector and each column weight transpose vector is obtained;

[0021] The output loss matrix is ​​obtained by combining the product of the output vector of each row and the transpose of the weights of each column.

[0022] In one embodiment, the above-described set communication-based approach, after N rounds of calculation, yields the product of each row output vector and each column weight transpose vector, including:

[0023] For any computing device, in the first round of computation, obtain the product of the row output vector and the column weight transpose vector in the computing device;

[0024] In rounds 2 through N of the calculation, the following steps are performed: obtain the column weight transpose vectors in the adjacent computing devices, and determine the column weight transpose vectors in the adjacent computing devices as the new column weight transpose vectors;

[0025] Determine the product of the row output vector and the new column weight transpose vector to obtain the product of each row output vector and each column weight transpose vector.

[0026] In one embodiment, the method further includes:

[0027] The transpose of the input matrix is ​​split into N parts according to the row and column dimensions. 2 Each row of input elements;

[0028] The output matrix is ​​split into N row output vectors according to the row dimension;

[0029] On each of the N different computing devices, a set of corresponding row output vectors and N row input elements are deployed;

[0030] Based on set communication, the sum of the products between each of the N row input elements and the output vector of each row is obtained after N rounds of calculation;

[0031] The weight gradient matrix is ​​obtained by combining the products on each computing device.

[0032] In one embodiment, the above-mentioned set communication-based method, after N rounds of calculation, obtains the sum of the products between each of the N row input elements and the output vector of each row, including:

[0033] For any computing device, in the first round of computation, obtain the product of the row output vector in the computing device and any one of the N row input elements;

[0034] In rounds 2 through N of computation, the following steps are performed: obtain the row output vectors from adjacent computing devices and use the row output vectors from adjacent computing devices as new row output vectors;

[0035] Determine the product of the new row output vector and the target input element to obtain the sum of the products between each of the N row input elements and each row output vector; the target input element refers to the row input element other than any one of the N row input elements.

[0036] Secondly, this application also provides an N-dimensional tensor partitioning device for a high-computing-power chip computing architecture, comprising:

[0037] The second row splitting module is used to split the input matrix in the Transformer network into N second column input vectors according to the column dimension during the training process of the Transformer network;

[0038] The second column splitting module is used to split the weight matrix in the Transformer network into 2N column weight elements according to the column dimension and the row dimension; the number of splits in the row dimension is N-1.

[0039] The second deployment module is used to deploy a set of corresponding second column input vectors and two column weight elements on each of N different computing devices.

[0040] The second calculation module is used to obtain the product of each second column input vector and the weight elements of the two columns through two rounds of calculation based on set communication.

[0041] The second acquisition module is used to integrate the product of each second column input vector and the weight elements of the two columns to obtain the output matrix.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0043] During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension;

[0044] The weight matrix in the Transformer network is split into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0045] On each of the N different computing devices, a set of corresponding second column input vectors and two column weight elements are deployed.

[0046] Based on the set communication method, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation;

[0047] The output matrix is ​​obtained by combining the product of each second column input vector with the weight elements of the two columns.

[0048] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0049] During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension;

[0050] The weight matrix in the Transformer network is split into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0051] On each of the N different computing devices, a set of corresponding second column input vectors and two column weight elements are deployed.

[0052] Based on the set communication method, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation;

[0053] The output matrix is ​​obtained by combining the product of each second column input vector with the weight elements of the two columns.

[0054] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0055] During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension;

[0056] The weight matrix in the Transformer network is split into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0057] On each of the N different computing devices, a set of corresponding second column input vectors and two column weight elements are deployed.

[0058] Based on the set communication method, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation;

[0059] The output matrix is ​​obtained by combining the product of each second column input vector with the weight elements of the two columns.

[0060] The aforementioned N-dimensional tensor partitioning method, apparatus, device, storage medium, and program product based on streaming processing, by partitioning the transpose of the input matrix according to the column dimension and the output matrix according to the column and row dimensions during the training process of the Transformer network, allows each computing device to pass vectors sequentially according to the order of the computing devices during multiple rounds of computation, avoiding the drawback of data delay caused by the need for cross-passing vectors in existing methods. This effectively reduces the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also partitions both the transpose of the input matrix and the output matrix, avoiding the copying of tensors in different devices and reducing storage overhead. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram of a tensor segmentation scheme based on time-space joint operation in one embodiment;

[0063] Figure 2 This is an application environment diagram of the N-dimensional tensor partitioning method for a high-computing-power chip computing architecture in one embodiment.

[0064] Figure 3 This is a flowchart illustrating an N-dimensional tensor partitioning method for a high-performance computing chip architecture in one embodiment.

[0065] Figure 4 This is a flowchart illustrating the forward computation process in one embodiment;

[0066] Figure 5 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0067] Figure 6 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0068] Figure 7 This is a schematic diagram of the backpropagation process in one embodiment;

[0069] Figure 8 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0070] Figure 9 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0071] Figure 10 This is a schematic diagram of the gradient update process in one embodiment;

[0072] Figure 11 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0073] Figure 12 This is a flowchart illustrating the N-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0074] Figure 13This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0075] Figure 14 This is a flowchart illustrating the forward computation process in one embodiment;

[0076] Figure 15 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0077] Figure 16 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0078] Figure 17 This is a schematic diagram of the backpropagation process in one embodiment;

[0079] Figure 18 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0080] Figure 19 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0081] Figure 20 This is a schematic diagram of the gradient update process in one embodiment;

[0082] Figure 21 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0083] Figure 22 This is a flowchart illustrating the M-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0084] Figure 23 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0085] Figure 24 This is a flowchart illustrating the forward computation process in one embodiment;

[0086] Figure 25 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0087] Figure 26 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0088] Figure 27 This is a schematic diagram of the backpropagation process in one embodiment;

[0089] Figure 28 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0090] Figure 29 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0091] Figure 30 This is a schematic diagram of the gradient update process in one embodiment;

[0092] Figure 31 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0093] Figure 32 This is a flowchart illustrating the K-dimensional tensor partitioning method for a high-performance computing chip architecture in another embodiment.

[0094] Figure 33 This is a structural block diagram of an N-dimensional tensor partitioning device for a high-computing-power chip computing architecture in one embodiment. Detailed Implementation

[0095] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0096] In recent years, with the development of artificial intelligence, neural network models have achieved remarkable success in fields such as natural language processing and computer vision. However, this has come at the cost of a massive number of parameters and the enormous computational demands required to train the network. Therefore, distributed computing, which distributes the neural network parameters and corresponding computations across different devices, has become the best choice to improve computational efficiency.

[0097] For Transformer networks, their architecture mainly involves linear matrix multiplication. For example, the following formula (1) shows the forward computation of linear matrix multiplication in Transformer networks, where I represents the input matrix, containing two dimensions M and N, W represents the weight matrix, containing two dimensions N and K, and O represents the output matrix, containing two dimensions M and K. Therefore, the tensor partitioning research for the forward computation of linear matrix multiplication in Transformer networks can be summarized as partitioning the tensors of the three dimensions M, N, and K, so that the corresponding sub-computations after partitioning can be distributed on different computing devices for distributed parallel computation, thereby improving the training and application efficiency of Transformer network models.

[0098] O [M，K]=I [M，N] ×W [N，K] (1);

[0099] Accordingly, equations (2) and (3) below demonstrate the calculation of the same matrix multiplication during the backpropagation and gradient update stages. Specifically, during backpropagation, the output loss matrix dI (i.e., the loss value of the i-th layer) is equal to the input loss matrix dO (i.e., the loss value of the (i+1)-th layer) multiplied by the transpose of the weight matrix W (W T During gradient updates, the weight gradient matrix dW is equal to the transpose of the input matrix I (i.e., the forward data tensor of the i-th layer) multiplied by the input loss matrix dO (the loss value of the (i+1)-th layer).

[0100] dI [M，K] =dO [M,N] ×W T [K，N] (2);

[0101] dW [N，K] =I T [N，M] ×dO [M，K] (3);

[0102] Therefore, how to effectively partition the tensor of matrix multiplication to simultaneously satisfy the dimensionality consistency of backpropagation and gradient update during training has become a focus of current academic research. Currently, a tensor partitioning scheme based on the time-space domain—PrimePar—has been proposed, such as... Figure 1 As shown, the calculation is divided into two rounds. In the first round, device D0 calculates the partial sum of I0×W1. At the same time, device D0 receives the input data tensor I1 from device D1 and the weight tensor W3 from device 3. In the second round, device D0 continues to calculate the result of I1×W3. Finally, the results of the two rounds are summed (I0×W1+I1×W3).

[0103] Although the above method successfully avoids tensor copying, it still has three significant drawbacks. First, due to the unbalanced data transmission path, it causes severe tailing delays, such as... Figure 1 As shown, due to inherent data dependencies, Primepar must simultaneously execute diagonal-ring and neighbor-ring communication modes to receive data for the next round of computation. In this case, data transmitted diagonally always arrives at the computation core later than data transmitted horizontally. This imbalance in data propagation will lead to severe tail latency and reduced computational efficiency of the computing device.

[0104] Secondly, due to the coarse-grained tensor partitioning, the utilization of computing nodes is insufficient. The above method needs to perform uniform partitioning in both M and K dimensions simultaneously, thus limiting it to 2k×2k devices. Unfortunately, this coarse-grained partitioning may lead to insufficient utilization of the remaining devices in the computing cluster. For example, when there are six devices waiting for computing tasks to be assigned, Primepar can only effectively utilize four of them, resulting in a waste of about 30%.

[0105] Third, the excessive communication traffic is caused by the lack of flexibility in the segmentation dimensions, such as... Figure 1 As shown, the above method always transmits 1 / 4 of the input and weights, regardless of their actual size. However, in scenarios with long sequences and today's large language models, such as GPT-4 32k, the size of the input tensor far exceeds the weights. In this case, transmitting only the weights might be a better choice, and the blind splitting design like in Technique 2 would lead to a large amount of data transmission traffic, which would actually increase the communication burden on the computing nodes.

[0106] In summary, avoiding the three major pain points of severe trailing latency caused by unbalanced transmission paths, insufficient utilization of computing nodes due to coarse-grained tensor partitioning, and excessive communication volume caused by inflexible partitioning dimensions are urgent technical problems to be solved. This application provides a novel N-dimensional tensor partitioning method for high-performance computing chip architectures, aiming to address these problems.

[0107] Having described the background technology of the N-dimensional tensor partitioning method for high-performance chip computing architecture provided in the embodiments of this application, the implementation environment involved in the N-dimensional tensor partitioning method for high-performance chip computing architecture provided in the embodiments of this application will be briefly described below. The N-dimensional tensor partitioning method for high-performance chip computing architecture provided in the embodiments of this application can be applied to, for example... Figure 2The computer device shown includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an N-dimensional tensor partitioning method for a high-performance computing chip architecture. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0108] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0109] Having described the application scenarios of the N-dimensional tensor partitioning method for high-performance chip computing architecture provided in the embodiments of this application, the following section focuses on the N-dimensional tensor partitioning method for high-performance chip computing architecture described in this application.

[0110] It should be noted that matrix partitioning in Transformer networks generally includes three types of partitioning: Type I, partitioning along the M-dimensional plane; Type II, partitioning along the N-dimensional plane; and Type III, partitioning along the K-dimensional plane.

[0111] In one scenario, we will first introduce the N-dimensional tensor partitioning method for high-performance chip computing architectures, which corresponds to Type II, i.e., N-dimensional partitioning.

[0112] In one embodiment, such as Figure 13As shown, an N-dimensional tensor partitioning method for a high-performance computing chip architecture is provided, and this method is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0113] S601. During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension.

[0114] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. So, for the input matrix I... [M，N] This is split into N second-column input vectors according to the column dimension. For example, as shown below. Figure 14 As shown, the input matrix I [M，N] The input vectors are split into four second-column vectors according to the column dimension: I0, I1, I2, and I3.

[0115] S602. Split the weight matrix in the Transformer network into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0116] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. So, for the weight matrix W... [N，K] This is done by splitting it into 2N column weight elements according to both row and column dimensions. Specifically, it's split N-1 times along the row dimension and once along the column dimension, resulting in 2N column weight elements. For example, ... Figure 14 As shown, the weight matrix W [N，K] It is split into 8 second output elements, namely W. 00 W 01 W 10 W 11 W 20 W 21 W 30 and W 31 .

[0117] S603. On each of the N different computing devices, deploy a set of corresponding second column input vectors and two column weight elements.

[0118] In this embodiment, after obtaining N second-column input vectors and 2N column weight elements, a corresponding set of second-column input vectors and two column weight elements are deployed on each of the N different computing devices. For example, on computing device D0, the second-column input vector I0 and two column weight elements W are deployed. 00 and W01 On computing device D1, a second column input vector I1 and two column weight elements W are deployed. 10 and W 11 On computing device D2, a second column input vector I2 and two column weight elements W are deployed. 20 and W 21 On computing device D3, deploy the second column input vector I3 and two column weight elements W. 30 and W 31 .

[0119] S604. Based on set communication, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation.

[0120] In this embodiment, in order to obtain such Figure 14 The output matrix O shown [M,K] After two rounds of calculation, the sum of the products of each second column input vector and the weight elements of the two columns is obtained.

[0121] S605. Combine the sum of the products of each second column input vector and the weight elements of the two columns to obtain the output matrix.

[0122] The output matrix is ​​the matrix used to train the Transformer network.

[0123] In this embodiment, after obtaining the sum of the products of each second column input vector and the weight elements of the two columns, the sum of the products of each second column input vector and the weight elements of the two columns is integrated to obtain the output matrix O. [M,K] .

[0124] In this embodiment, by dividing the transpose of the input matrix according to the column dimension and the output matrix according to the column and row dimensions, each computing device only needs to pass vectors sequentially according to the order of the computing devices during multiple rounds of computation. This avoids the drawback of data delay caused by the need for cross-passing vectors in existing methods, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the transpose of the input matrix and the output matrix to avoid copying tensors in different devices and reduce storage overhead.

[0125] In one embodiment, in Figure 13 Based on the illustrated embodiment, the process of obtaining the sum of the products between each first column input vector and every two second output elements can be described, such as... Figure 15 As shown, the above S213 "based on set communication, the sum of the products between each first column input vector and every two second output elements is obtained through two rounds of calculation" includes:

[0126] S701. For any computing device, in the first round of calculation, obtain the product of the second column input vector and the weight element of any column in the computing device.

[0127] In this embodiment, during the first round of computation in each computing device, the product of the second column input vector and any column weight element in each computing device is obtained. For example, for computing device D0, it deploys the second column input vector I0 and the column weight element W. 00 and W 01 Then we directly calculate the second column input vector I0 and the column weight element W. 00 The product of O is obtained. 00 For computing device D1, it deploys a second column input vector I1 and column weight elements W. 10 and W 11 Then we directly calculate the second column input vector I1 and the column weight element W. 10 The product of O is obtained. 10 For computing device D2, it deploys a second column input vector I2 and column weight elements W. 20 and W 21 Then we directly calculate the second column input vector I2 and the column weight element W. 20 The product of O is obtained. 20 For computing device D3, it deploys a second column input vector I3 and column weight elements W. 30 and W 31 Then we can directly calculate the second column input vector I3 and the column weight element W. 30 The product of O is obtained. 30 .

[0128] S702. In the second round of calculation, while obtaining the sum of the products of the second column input vector and the weight element of another column in the computing device, the product of the second column input vector and any column weight element in each computing device is also obtained.

[0129] In this embodiment, after completing the first round of calculations, the sum of the products of the second column input vector and the weight elements of the other column in each computing device can be obtained, i.e., O′=O 00 +O 10 +O 20 +O 30 In addition to obtaining the sum of the products of the second column input vector and the weight element of another column, in the second round of calculation for each computing device, the product of the second column input vector and any column weight element in each computing device is obtained. That is, for computing device D0, it deploys the second column input vector I0 and the column weight element W. 00 and W 01 Then we directly calculate the second column input vector I0 and the column weight element W. 01The product of O is obtained. 01 For computing device D1, it deploys a second column input vector I1 and column weight elements W. 10 and W 11 Then we directly calculate the second column input vector I1 and the column weight element W. 11 The product of O is obtained. 11 For computing device D2, it deploys a second column input vector I2 and column weight elements W. 20 and W 21 Then we directly calculate the second column input vector I2 and the column weight element W. 21 The product of O is obtained. 21 For computing device D3, it deploys a second column input vector I3 and column weight elements W. 30 and W 31 Then we can directly calculate the second column input vector I3 and the column weight element W. 31 The product of O is obtained. 31 .

[0130] S703. Obtain the sum of the products of the second column input vector and the weight elements of the other column in each computing device.

[0131] In this embodiment, after completing the second round of calculations, the sum of the products of the second column input vector and the weight elements of the other column in each computing device can be obtained, i.e., O″ = O. 01 +O 11 +O 21 +O 31 .

[0132] It should be noted that O′ is assigned to computing devices D0 and D1 respectively, and O″ is assigned to computing devices D2 and D3 respectively.

[0133] In this embodiment, while obtaining the sum of the products of the second column input vector and the weight element of another column in the computing device, the product of the second column input vector and any column weight element in each computing device can be obtained. This can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0134] In one embodiment, in the above Figure 13 Based on the illustrated embodiment, the matrix partitioning process used in the forward computation of the Transformer network can also be described, see [link to documentation]. Figure 16 The above methods also include:

[0135] S606. Split the output matrix into N row output vectors according to the row dimension.

[0136] Please refer to formula (2) above. For backpropagation of the Transformer model, the tensors that need to be split include the transpose of the output matrix and the weight matrix. Therefore, for the output matrix dO[M,N], it is split into N row output vectors according to the row dimension. For example, as shown in the formula (2), the tensors that need to be split include the output matrix and the transpose of the weight matrix. Figure 17 As shown, the output matrix dO[M,N] is split into four row output vectors according to the row dimension, namely O0, O1, O2 and O3.

[0137] S607. Split the transpose of the weight matrix in the Transformer network into N column weight transpose vectors according to the column dimension.

[0138] Please refer to formula (2) above. For backpropagation of the Transformer model, the tensors that need to be split include the output matrix and the transpose of the weight matrix. Then, for the transpose of the weight matrix W... T [K,N] This is then split into N column weight transpose vectors according to the column dimension. For example, as... Figure 17 As shown, the transpose of the weight matrix W T [K,N] It is split into four column weight transpose vectors according to the column dimension, namely W0, W1, W2 and W3.

[0139] S608. On each of the N different computing devices, deploy a set of corresponding row output vectors and column weight transpose vectors.

[0140] In this embodiment, after obtaining N row output vectors and N column weight transpose vectors as described above, a set of corresponding row output vectors and column weight transpose vectors are deployed on each of the N different computing devices. For example, row output vector O0 and column weight transpose vector W0 are deployed on computing device D0, row output vector O1 and column weight transpose vector W1 are deployed on computing device D1, row output vector O2 and column weight transpose vector W2 are deployed on computing device D2, and row output vector O3 and column weight transpose vector W3 are deployed on computing device D3.

[0141] S609. Based on set communication, after N rounds of calculation, the product of each row output vector and each column weight transpose vector is obtained.

[0142] In this embodiment, in order to obtain such Figure 17 The output loss matrix dI shown [M，K] The column weight transpose vectors W0 (from computing device D0), W1 (from computing device D1), W2 (from computing device D2), and W3 (from computing device D3) can be transformed according to... Figure 17The data transmission method shown in the figure transmits data sequentially. After each transmission, each computing device calculates the product of its row output vector and column weight transpose vector. After N rounds of calculation, the product of each row output vector and each column weight transpose vector is obtained.

[0143] S610. Integrate the product of each row output vector and each column weight transpose vector to obtain the output loss matrix.

[0144] The output loss matrix is ​​the matrix used to train the Transformer network.

[0145] In this embodiment, after obtaining the product of each row output vector and each column weight transpose vector, the product of the row output vector and each column weight transpose vector is integrated to obtain the output loss matrix dI. [M，K] .

[0146] In this embodiment, by dividing the output matrix according to the row dimension and the transpose of the weight matrix according to the column dimension, each computing device only needs to pass vectors sequentially according to the order of the computing devices when performing multiple rounds of calculations. This avoids the drawback of data delay caused by the need for cross-passing of vectors in the existing method, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the output matrix and the transpose of the weight matrix to avoid copying tensors in different devices and reduce storage overhead.

[0147] In one embodiment, in Figure 16 Based on the illustrated embodiment, the process of obtaining the product of each row output vector and each column weight transpose vector can be described, such as... Figure 18 As shown, the above S804, "based on set communication, after N rounds of calculation, the product of each row output vector and each column weight transpose vector is obtained," includes:

[0148] S801. For any computing device, in the first round of calculation, obtain the product of the row output vector and the column weight transpose vector in the computing device.

[0149] In this embodiment, for any one of the N computing devices, in the first round of calculation for that computing device, the product of the row output vector and the column weight transpose vector is directly determined. For example, for computing device D0, which deploys the row output vector O0 and the column weight transpose vector W0, the product of the deployed row output vector O0 and the column weight transpose vector W0 is directly calculated to obtain I. 00 For computing device D1, which has a row output vector O1 and a column weight transpose vector W1, we can directly calculate the product of the row output vector O1 and the column weight transpose vector W1 to obtain I.11 For computing device D2, which has deployed row output vector O2 and column weight transpose vector W2, we can directly calculate the product of row output vector O2 and column weight transpose vector W2 to obtain I. 22 For computing device D3, which has deployed row output vector O3 and column weight transpose vector W3, we can directly calculate the product of row output vector O3 and column weight transpose vector W3 to obtain I. 33 .

[0150] S802. In the second to Nth rounds of calculation, perform the following steps: obtain the column weight transpose vectors in the adjacent computing devices, and determine the column weight transpose vectors in the adjacent computing devices as the new column weight transpose vectors.

[0151] In this embodiment, after the first round of calculation is completed, during the second round of calculation, computing device D1 sends its column weight transpose vector W1 to computing device D0, computing device D2 sends its column weight transpose vector W2 to computing device D1, computing device D3 sends its column weight transpose vector W3 to computing device D2, and computing device D0 sends its column weight transpose vector W0 to computing device D3. Therefore, during the second round of calculation, computing device D0 will deploy column weight transpose vector W1 and row output vector O0, computing device D1 will deploy column weight transpose vector W2 and row output vector O1, computing device D2 will deploy column weight transpose vector W3 and row output vector O2, and computing device D3 will deploy column weight transpose vector W0 and row output vector O3.

[0152] In the third round of calculation, computing device D1 sends its column weight transpose vector W2 to computing device D0, computing device D2 sends its column weight transpose vector W3 to computing device D1, computing device D3 sends its column weight transpose vector W0 to computing device D2, and computing device D0 sends its column weight transpose vector W1 to computing device D3. Therefore, in the third round of calculation, computing device D0 will deploy column weight transpose vector W2 and row output vector O0, computing device D1 will deploy column weight transpose vector W3 and row output vector O1, computing device D2 will deploy column weight transpose vector W0 and row output vector O2, and computing device D3 will deploy column weight transpose vector W1 and row output vector O3.

[0153] In the fourth round of calculation, computing device D1 sends its column weight transpose vector W3 to computing device D0, computing device D2 sends its column weight transpose vector W0 to computing device D1, computing device D3 sends its column weight transpose vector W1 to computing device D2, and computing device D0 sends its column weight transpose vector W2 to computing device D3. Therefore, in the fourth round of calculation, computing device D0 will deploy column weight transpose vector W3 and row output vector O0, computing device D1 will deploy column weight transpose vector W0 and row output vector O1, computing device D2 will deploy column weight transpose vector W1 and row output vector O2, and computing device D3 will deploy column weight transpose vector W2 and row output vector O3.

[0154] S803. Determine the product of the row output vector and the new column weight transpose vector to obtain the product of each row output vector and each column weight transpose vector.

[0155] In this embodiment, during the second round of calculation, for computing device D0, which deploys row output vector O0 and column weight transpose vector W1, the product of row output vector O0 and column weight transpose vector W1 is directly calculated to obtain I. 01 For computing device D1, which has deployed row output vector O1 and column weight transpose vector W2, the product of row output vector O1 and column weight transpose vector W2 is directly calculated to obtain I. 12 For computing device D2, which has deployed row output vector O2 and column weight transpose vector W3, we can directly calculate the product of row output vector O2 and column weight transpose vector W3 to obtain I. 23 For computing device D3, which deploys row output vector O3 and column weight transpose vector W0, we can directly calculate the product of row output vector O3 and column weight transpose vector W0 to obtain I. 30 .

[0156] In the third round of calculation, for computing device D0, which has deployed column weight transpose vector W2 and row output vector O0, we can directly calculate column weight transpose vector W2 and row output vector O0 to obtain I. 02 For computing device D1, which has a column weight transpose vector W3 and a row output vector O1, the product of the column weight transpose vector W3 and the row output vector O1 is directly calculated to obtain I. 13 For computing device D2, which has a column weight transpose vector W0 and a row output vector O2, we can directly calculate the product of the column weight transpose vector W0 and the row output vector O2 to obtain I. 20For computing device D3, which has a column weight transpose vector W1 and a row output vector O3, we can directly calculate the product of the column weight transpose vector W1 and the row output vector O3 to obtain I. 31 .

[0157] In the fourth round of calculation, for computing device D0, which has deployed column weight transpose vector W3 and row output vector O0, the product of column weight transpose vector W3 and row output vector O0 is directly calculated to obtain I. 03 For computing device D1, which has a column weight transpose vector W0 and a row output vector O1, the product of the column weight transpose vector W0 and the row output vector O1 is directly calculated to obtain I. 10 For computing device D2, which has a column weight transpose vector W1 and a row output vector O2, we can directly calculate the product of the column weight transpose vector W1 and the row output vector O2 to obtain I. 21 For computing device D3, which has a column weight transpose vector W2 and a row output vector O3, we can directly calculate the product of the column weight transpose vector W2 and the row output vector O3 to obtain I. 32 .

[0158] In summary, for the case where N=4, after the above four rounds of calculations, we can obtain the following result: Figure 4 The output loss matrix dI shown [M，K] .

[0159] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0160] In one embodiment, in the above Figure 16 Based on the illustrated embodiment, after obtaining the output matrix as described above, the matrix partitioning process used in the backpropagation process of the Transformer network can also be described. See [link to documentation]. Figure 19 The above methods also include:

[0161] S611. Split the transpose of the input matrix into N parts according to the row and column dimensions. 2 Each row of input elements.

[0162] Please refer to formula (2) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. Then, for the transpose of the input matrix I... T [N,M]In other words, it is split into N according to the row and column dimensions. 2 Each line contains input elements. For example, such as... Figure 20 As shown, the transpose of the input matrix I is... T [N,M] Split into N based on row and column dimensions. 2 Each row of input elements, i.e., I 00 I 01 I 02 I 03 I 10 I 11 I 12 I 13 I 20 I 21 I 22 I 23 I 30 I 31 I 32 I 33 .

[0163] S612. Split the output matrix into N row output vectors according to the row dimension.

[0164] Please refer to formula (2) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. Therefore, for the output matrix dO[M,K], it is split into N row output vectors according to the row dimension. For example, as shown in the formula (2), the tensors that need to be split include the transpose of the input matrix and the output matrix. Figure 20 As shown, the output matrix dO[M,K] will be... ] The output vectors are split into four rows according to the row dimension: O0, O1, O2, and O3.

[0165] S613. On each of the N different computing devices, deploy a set of corresponding row output vectors and N row input elements.

[0166] In this embodiment, after obtaining N row output vectors as described above, and after obtaining N in S612 above... 2 After the input elements are defined, a corresponding set of row output vectors and N row input elements can be deployed on each of the N different computing devices. For example, on computing device D0, a row output vector O0 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ... 00 I 01 I 02 I 03 On computing device D1, deploy row output vector O1 and 4 row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99 10 I 11 I 12 I 13On computing device D2, deploy row output vector O2 and 4 row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I220 I 21 I 22 I 23 On computing device D3, deploy row output vector O3 and 4 row input elements (I0, I ... 30 I 31 I 32 I 33 ).

[0167] S614. Based on set communication, the sum of the products between the N row input elements and the output vectors of each row is obtained after N rounds of calculation.

[0168] In this embodiment, in order to obtain such Figure 20 The weight gradient matrix dW shown [N,K] The row output vectors O0, O1, O2, and O3 in computing device D0, computing device D1, computing device D2, and computing device D3 can be arranged according to... Figure 20 The data transmission method shown in the figure transmits data sequentially. After each transmission, each computing device calculates the product of its row output vector and any row input element. After N rounds of calculation, for N computing devices, the products of the row output vector and row input elements calculated on their devices are summed to obtain the sum of the products of the N row input elements and each row output vector.

[0169] S615. Integrate the sum of the products on each computing device to obtain the weight gradient matrix.

[0170] The weight gradient matrix is ​​the matrix used to train the Transformer network.

[0171] In this embodiment, after obtaining the sum of the products between each of the N row input elements and the output vector of each row, the sums of the products between each row input element and the output vector of each row are integrated to obtain the weight gradient matrix dW. [N,K] .

[0172] In this embodiment, the output matrix is ​​divided according to the row dimension, and the transpose of the input matrix is ​​divided into N... 2 The single-row input element allows each computing device to pass vectors sequentially according to their arrangement during multiple rounds of computation, avoiding the data delay caused by cross-passing vectors in existing systems. This effectively reduces the tensor storage and communication overhead in distributed training of neural networks. In addition, this application also splits the transpose matrix of the input matrix and the output matrix, avoiding the copying of tensors in different devices and reducing storage overhead.

[0173] In one embodiment, in Figure 21 Based on the illustrated embodiment, the process of obtaining the sum of the products between each of the N row input elements and the output vector of each row can be described, such as... Figure 22 As shown, the above S614, "based on set communication, after N rounds of calculation, the sum of the products between the N row input elements and each row output vector is obtained", includes:

[0174] S801. For any computing device, in the first round of calculation, obtain the product of the row output vector in the computing device and any one of the N row input elements.

[0175] In this embodiment, on the aforementioned computing device D0, a row output vector O0 and four row input elements (I) are deployed. 00 I 01 I 02 I 03 On computing device D1, a row output vector O1 and four row input elements (I) are deployed. 10 I 11 I 12 I 13 On computing device D2, row output vector O2 and 4 row input elements (I) are deployed. 20 I 21 I 22 I 23 On computing device D3, row output vector O3 and four row input elements (I) are deployed. 30 I 31 I 32 I 33 After that, during the first round of calculation, the row output vector O0 and the row input element I are obtained on the computing device D0. 00 The product of these two products yields W. 00 Obtain the row output vector O1 and row input element I on computing device D1. 10 The product of these two products yields W. 10 Obtain the row output vector O2 and row input element I on computing device D2. 20 The product of these two products yields W. 20 Obtain the row output vector O3 and row input element I on computing device D3. 30 The product of these two products yields W. 30 .

[0176] S802. In the second to Nth rounds of calculation, perform the following steps: obtain the row output vector in the adjacent computing device, and use the row output vector in the adjacent computing device as the new row output vector.

[0177] In this embodiment, after the first round of calculation is completed, during the second round, computing device D1 sends its row output vector O1 to computing device D0, computing device D2 sends its row output vector O2 to computing device D1, computing device D3 sends its row output vector O3 to computing device D2, and computing device D0 sends its row output vector O0 to computing device D3. Therefore, during the second round of calculation, computing device D0 will deploy row output vector O1 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ... 00 I 01 I 02 I 03 The computing device D1 will then deploy a row output vector O2 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I11 10 I 11 I 12 I 13 On computing device D2, a row output vector O3 and four row input elements (I, I ... 20 I 21 I 22 I 23 On computing device D3, a row output vector O0 and four row input elements (I, I ... 30 I 31 I 32 I 33 ).

[0178] In the third round of computation, computing device D1 sends its row output vector O2 to computing device D0, computing device D2 sends its row output vector O3 to computing device D1, computing device D3 sends its row output vector O0 to computing device D2, and computing device D0 sends its row output vector O1 to computing device D3. Therefore, in the third round of computation, computing device D0 will have row output vector O2 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ... 00 I 01 I 02 I 03 The computing device D1 will then deploy a row output vector O3 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I11 10 I 11 I 12 I 13 On computing device D2, a row output vector O0 and four row input elements (I, I ... 20 I 21 I 22 I 23 ), and the computing device D3 will deploy a row output vector O1 and four row input elements (I, I ... 30 I 31I 32 I 33 ).

[0179] In the fourth round of computation, computing device D1 sends its row output vector O3 to computing device D0, computing device D2 sends its row output vector O0 to computing device D1, computing device D3 sends its row output vector O1 to computing device D2, and computing device D0 sends its row output vector O2 to computing device D3. Therefore, in the fourth round of computation, computing device D0 will have row output vector O3 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ... 00 I 01 I 02 I 03 The computing device D1 will then deploy a row output vector O0 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I9 10 I 11 I 12 I 13 On computing device D2, a row output vector O1 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99, 20 I 21 I 22 I 23 On computing device D3, a row output vector O2 and four row input elements (I, I ... 30 I 31 I 32 I 33 ).

[0180] S803. Determine the product of the new row output vector and the target input element to obtain the sum of the products between each of the N row input elements and each row output vector; the target input element refers to the row input element other than any one of the N row input elements.

[0181] In this embodiment, during the second round of calculation, for computing device D0, it deploys a row output vector O1 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I9 00 I 01 I 02 I 03 Then, directly calculate the row output vector O1 and the row input element I. 01 The product of these two products yields W. 01 For computing device D1, it deploys a row output vector O2 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I2 10 I 11 I 12 I 13 Then, directly calculate the row output vector O2 and the row input element I. 11 The product of these two products yields W. 11For computing device D2, it deploys a row output vector O3 and four row input elements (I0, I ... 20 I 21 I 22 I 23 Then, directly calculate the row output vector O3 and the row input element I. 21 The product of these two products yields W. 21 For computing device D3, it deploys a row output vector O0 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I 30 I 31 I 32 I 33 Then, directly calculate the row output vector O0 and the row input element I. 31 The product of these two products yields W. 31 .

[0182] In the third round of computation, computing device D0 is configured with a row output vector O2 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ... 00 I 01 I 02 I 03 Then, directly calculate the row output vector O2 and the row input element I. 02 The product of these two products yields W. 02 For computing device D1, it deploys a row output vector O3 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I1 10 I 11 I 12 I 13 Then, directly calculate the row output vector O3 and the row input element I. 12 The product of these two products yields W. 12 For computing device D2, it deploys a row output vector O0 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99, I99, I99, I99, 20 I 21 I 22 I 23 Then, directly calculate the row output vector O0 and the row input element I. 22 The product of these two products yields W. 22 For computing device D3, it deploys a row output vector O1 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99, I99, I99, I 30 I 31 I 32 I 33 Then, directly calculate the row output vector O1 and the row input element I. 32 The product of these two products yields W. 32 .

[0183] In the fourth round of computation, computing device D0 deployed a row output vector O3 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...00 I 01 I 02 I 03 Then, directly calculate the row output vector O3 and the row input element I. 03 The product of these two products yields W. 03 For computing device D1, it deploys a row output vector O0 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99, I99, I99, I 10 I 11 I 12 I 13 Then, directly calculate the row output vector O0 and the row input element I. 13 The product of these two products yields W. 13 For computing device D2, it deploys a row output vector O1 and four row input elements (I1, I2, I3, I4, I5, I6, I7, I8, I9 ...9, I99, I99, I99, I99, I9 20 I 21 I 22 I 23 Then, directly calculate the row output vector O1 and the row input element I. 23 The product of these two products yields W. 23 For computing device D3, it deploys a row output vector O2 and four row input elements (I2, I3, I4, I5, I6, I7, I8, I9, I10, I11, I2, I3 30 I 31 I 32 I 33 Then, directly calculate the row output vector O2 and the row input element I. 33 The product of these two products yields W. 33 .

[0184] In summary, for the case where N=4, after the above four rounds of calculations, the computing device D0 obtained four products, including: W 00 W 01 W 02 W 03 The sum of these four products is W0. The computing device D1 then generates four products, including: W... 10 W 11 W 12 W 13 The four products are summed to obtain W1. The computing device D2 then obtains four products, including: W... 20 W 21 W 22 W 23 Adding these four products together yields W2. The computing device D2 then displays four products, including: W... 30 W 31 W 32 W 33 Add these four products together to get W3, which gives the result as follows: Figure 20 The weight gradient matrix dW shown [N，K] .

[0185] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0186] In one embodiment, such as Figure 23 As shown, an N-dimensional tensor partitioning method for a high-performance computing chip architecture is also provided, including:

[0187] T1. During the training process of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension;

[0188] T2. Split the weight matrix in the Transformer network into 2N column weight elements according to the column dimension and the row dimension; the number of times the row dimension is split is N-1.

[0189] T3. On each of the N different computing devices, deploy a set of corresponding second column input vectors and two column weight elements.

[0190] T4. For any computing device, in the first round of calculation, obtain the product of the second column input vector and the weight element of any column in the computing device;

[0191] T5. In the second round of calculation, while obtaining the sum of the products of the second column input vector and the weight element of another column in the computing device, the product of the second column input vector and any column weight element in each computing device is also obtained.

[0192] T6. Obtain the sum of the products of the second column input vector and the weight elements of the other column in each computing device;

[0193] T7. Combine the products of each second column input vector with the weight elements of the two columns to obtain the output matrix;

[0194] T8. Split the output matrix into N row output vectors according to the row dimension;

[0195] T9. Split the transpose of the weight matrix in the Transformer network into N column weight transpose vectors according to the column dimension;

[0196] T10. On each of N different computing devices, deploy a set of corresponding row output vectors and column weight transpose vectors.

[0197] T11. For any computing device, in the first round of calculation, obtain the product of the row output vector and the column weight transpose vector in the computing device;

[0198] T12. In the calculation from the 2nd to the Nth round, perform the following steps: obtain the column weight transpose vectors in the adjacent computing devices, and determine the column weight transpose vectors in the adjacent computing devices as the new column weight transpose vectors;

[0199] T13. Determine the product of the row output vector and the new column weight transpose vector to obtain the product of each row output vector and each column weight transpose vector;

[0200] T14. Combine the product of each row's output vector and each column's weight transpose vector to obtain the output loss matrix;

[0201] T15. Split the transpose of the input matrix into N parts according to the row and column dimensions. 2 Each row of input elements;

[0202] T16. Split the output matrix into N row output vectors according to the row dimension;

[0203] T17. On each of N different computing devices, deploy a set of corresponding row output vectors and N row input elements;

[0204] T18. For any computing device, in the first round of calculation, obtain the product of the row output vector in the computing device and any one of the N row input elements;

[0205] T19. In the calculation from round 2 to round N, perform the following steps: obtain the row output vectors in the adjacent computing devices, and use the row output vectors in the adjacent computing devices as the new row output vectors;

[0206] T20. Determine the product of the new row output vector and the target input element to obtain the sum of the products between each of the N row input elements and the respective row output vector; the target input element refers to the row input element other than any one of the N row input elements.

[0207] T21. Integrate the sum of the products on each computing device to obtain the weight gradient matrix.

[0208] It should be noted that the descriptions of T1-T21 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0209] In the second scenario, we will introduce the M-dimensional tensor partitioning method for high-performance chip computing architectures corresponding to type I, which involves partitioning the M-dimensional dimensions.

[0210] In one embodiment, such as Figure 3 As shown, an M-dimensional tensor partitioning method for a high-performance computing chip architecture is provided, and this method is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0211] S201. During the training of the Transformer network, the input matrix in the Transformer network is split into N first-row input vectors according to the row dimension.

[0212] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. So, for the input matrix I... [M，N] This is done by splitting the input matrix into N first-row input vectors according to the row dimension (i.e., the M dimensions of the input matrix). For example, as... Figure 4 As shown, the input matrix I [M，N] The input vectors are split into four first-row input vectors according to the row dimension: I0, I1, I2, and I3.

[0213] S202. Split the weight matrix in the Transformer network into N first-column weight vectors according to the column dimension.

[0214] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. So, for the weight matrix W... [N，K] This is done by splitting it into N weight vectors in the first column according to the column dimension (i.e., the K dimensions of the weight matrix). For example, as... Figure 4 As shown, the weight matrix W [N，K] It is split into four weight vectors based on the column dimension: W0, W1, W2, and W3.

[0215] S203. On each of the N different computing devices, deploy a set of corresponding first row input vectors and first column weight vectors.

[0216] In this embodiment, after obtaining N first-row input vectors and N first-column weight vectors as described above, a corresponding set of first-row input vectors and first-column weight vectors is deployed on each of the N different computing devices. For example, on computing device D0, the first-row input vector I0 and the first-column weight vector W0 are deployed; on computing device D1, the first-row input vector I1 and the first-column weight vector W1 are deployed; on computing device D2, the first-row input vector I2 and the first-column weight vector W2 are deployed; and on computing device D3, the first-row input vector I3 and the first-column weight vector W3 are deployed.

[0217] S204. Based on set communication, N rounds of calculations are performed to obtain the product of each first row input vector and each first column weight vector.

[0218] In this embodiment, in order to obtain such Figure 4 The output matrix O shown [M，K] The first column weight vector W0 in computing device D0, the first column weight vector W1 in computing device D1, the first column weight vector W2 in computing device D2, and the first column weight vector W3 in computing device D3 can be arranged according to... Figure 4 The data transmission method shown in the figure transmits data sequentially. After each transmission, each computing device calculates the product of the first row input vector and each first column weight vector. After N rounds of calculation, the product of each first row input vector and each first column weight vector is obtained.

[0219] S205. Integrate the product of each first row input vector and each first column weight vector to obtain the output matrix.

[0220] The output matrix is ​​the matrix used to train the Transformer network.

[0221] In this embodiment, after obtaining the product of each first row input vector and each first column weight vector, the product of each first row input vector and each first column weight vector is integrated to obtain the output matrix O. [M，K] .

[0222] In this embodiment, by dividing the input matrix only according to the row dimension and the weight matrix only according to the column dimension, each computing device only needs to pass vectors sequentially according to the order of the computing devices when performing multiple rounds of calculations. This avoids the drawback of data delay caused by the need for cross-passing of vectors in the existing method, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the input matrix and the weight matrix to avoid copying tensors in different devices and reduce storage overhead.

[0223] In one embodiment, in Figure 3 Based on the illustrated embodiment, the process of obtaining the product of each first row input vector and each first column weight vector can be described, such as... Figure 5 As shown, the above S204, "based on set communication, through N rounds of calculation to obtain the product of each first row input vector and each first column weight vector," includes:

[0224] S301. For any computing device, in the first round of calculation, obtain the product of the first row input vector and the first column weight vector in the computing device.

[0225] In this embodiment, for any one of the N computing devices, in the first round of computation of that computing device, the product of the first row input vector and the first column weight vector is directly determined. For example, for computing device D0, which deploys the first row input vector I0 and the first column weight vector W0, the product of the first row input vector I0 and the first column weight vector W0 is directly calculated to obtain O. 00 For computing device D1, which has a first row input vector I1 and a first column weight vector W1, the product of the first row input vector I1 and the first column weight vector W1 is directly calculated to obtain O. 11 For computing device D2, which has a first row input vector I2 and a first column weight vector W2, the product of the first row input vector I2 and the first column weight vector W2 is directly calculated to obtain O. 22 For computing device D3, which has a first row input vector I3 and a first column weight vector W3, the product of the first row input vector I3 and the first column weight vector W3 is directly calculated to obtain O. 33 .

[0226] S302. In the second to Nth rounds of calculation, perform the following steps: obtain the first column weight vector in the adjacent computing device, and determine the first column weight vector in the adjacent computing device as the new first column weight vector.

[0227] In this embodiment, after the first round of calculation is completed, during the second round of calculation, computing device D1 sends its first column weight vector W1 to computing device D0, computing device D2 sends its first column weight vector W2 to computing device D1, computing device D3 sends its first column weight vector W3 to computing device D2, and computing device D0 sends its first column weight vector W0 to computing device D3. Therefore, during the second round of calculation, computing device D0 will deploy the first column weight vector W1 and the first row input vector I0, computing device D1 will deploy the first column weight vector W2 and the first row input vector I1, computing device D2 will deploy the first column weight vector W3 and the first row input vector I2, and computing device D3 will deploy the first column weight vector W0 and the first row input vector I3.

[0228] In the third round of computation, computing device D1 sends its first column weight vector W2 to computing device D0, computing device D2 sends its first column weight vector W3 to computing device D1, computing device D3 sends its first column weight vector W0 to computing device D2, and computing device D0 sends its first column weight vector W1 to computing device D3. Therefore, in the third round of computation, computing device D0 will have its first column weight vector W2 and first row input vector I0 deployed, computing device D1 will have its first column weight vector W3 and first row input vector I1 deployed, computing device D2 will have its first column weight vector W0 and first row input vector I2 deployed, and computing device D3 will have its first column weight vector W1 and first row input vector I3 deployed.

[0229] In the fourth round of calculation, computing device D1 sends its first column weight vector W3 to computing device D0, computing device D2 sends its first column weight vector W0 to computing device D1, computing device D3 sends its first column weight vector W1 to computing device D2, and computing device D0 sends its first column weight vector W2 to computing device D3. Therefore, in the fourth round of calculation, computing device D0 will deploy the first column weight vector W3 and the first row input vector I0, computing device D1 will deploy the first column weight vector W0 and the first row input vector I1, computing device D2 will deploy the first column weight vector W1 and the first row input vector I2, and computing device D3 will deploy the first column weight vector W2 and the first row input vector I3.

[0230] S303. Determine the product of the first row input vector and the new first column weight vector to obtain the product of each first row input vector and each first column weight vector.

[0231] In this embodiment, during the second round of calculation, for computing device D0, which has a first row input vector I0 and a first column weight vector W1 deployed, the product of the first row input vector I0 and the first column weight vector W1 is directly calculated to obtain O. 01 For computing device D1, which has a first row input vector I1 and a first column weight vector W2, the product of the first row input vector I1 and the first column weight vector W2 is directly calculated to obtain O. 12 For computing device D2, which has a first row input vector I2 and a first column weight vector W3, the product of the first row input vector I2 and the first column weight vector W3 is directly calculated to obtain O. 23 For computing device D3, which has a first row input vector I3 and a first column weight vector W0, the product of the first row input vector I3 and the first column weight vector W0 is directly calculated to obtain O. 30 .

[0232] In the third round of calculation, for computing device D0, which has a first row input vector I0 and a first column weight vector W2, the product of the first row input vector I0 and the first column weight vector W2 is directly calculated to obtain O. 02 For computing device D1, which has a first row input vector I1 and a first column weight vector W3, the product of the first row input vector I1 and the first column weight vector W3 is directly calculated to obtain O. 13 For computing device D2, which has a first row input vector I2 and a first column weight vector W0, the product of the first row input vector I2 and the first column weight vector W0 is directly calculated to obtain O. 20 For computing device D3, which has a first row input vector I3 and a first column weight vector W1, the product of the first row input vector I3 and the first column weight vector W1 is directly calculated to obtain O. 31 .

[0233] In the fourth round of calculation, for computing device D0, which has deployed the first row input vector I0 and the first column weight vector W3, the product of the first row input vector I0 and the first column weight vector W3 is directly calculated to obtain O. 03 For computing device D1, which has a first row input vector I1 and a first column weight vector W0, the product of the first row input vector I1 and the first column weight vector W0 is directly calculated to obtain O. 10 For computing device D2, which has a first row input vector I2 and a first column weight vector W1, the product of the first row input vector I2 and the first column weight vector W1 is directly calculated to obtain O. 21 For computing device D3, which has a first row input vector I3 and a first column weight vector W2, the product of the first row input vector I3 and the first column weight vector W2 is directly calculated to obtain O. 32 .

[0234] In summary, for the case where N=4, after the above four rounds of calculations, we can obtain the following result: Figure 4 The output matrix O shown [M，K] .

[0235] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0236] In one embodiment, in the above Figure 3Based on the illustrated embodiment, after obtaining the output matrix as described above, the matrix partitioning process used in the backpropagation process of the Transformer network can also be described. See [link to documentation]. Figure 6 The above methods also include:

[0237] S206. Split the transpose of the weight matrix into N row weight vectors according to the row dimension.

[0238] Please refer to formula (2) above. For backpropagation of the Transformer model, the tensors that need to be split include the output matrix and the transpose of the weight matrix. Then, for the transpose of the weight matrix W... T [K,N] This is done by splitting it into N row weight vectors according to the row dimension (i.e., the K dimensions of the transpose of the weight matrix). For example, as... Figure 7 As shown, the transpose of the weight matrix W T [K,N] It is split into four row weight vectors according to the row dimension, namely W0, W1, W2 and W3.

[0239] S207. On each of N different computing devices, deploy a set of corresponding row weight vectors and N first output elements in the output matrix; the output matrix includes N... 2 The first output element.

[0240] In this embodiment, after obtaining N row weight vectors as described above, and after obtaining N in S303 above... 2 After the first output element, a set of corresponding row weight vectors and N can be deployed on each of the N different computing devices. 2 Any N first output elements from a set of first output elements. For example, on computing device D0, deploy row weight vector W0 and 4 first output elements (O, W0 ... 00 O 01 O 02 and O 03 On computing device D1, row weight vector W1 and four first output elements (O1, O2, and O3, O4, O5, O6, O7, O8, O9 ... 10 O 11 O 12 and O 13 On computing device D2, row weight vector W2 and four first output elements (O2, O2, and O2) are deployed. 20 O 11 O 12 and O 13 On computing device D3, row weight vector W3 and four first output elements (O3, O2, and O2) are deployed. 30 O 31 O 32and O 33 ).

[0241] S208. Based on set communication, after N rounds of calculation, the sum of the products between the N first output elements and the weight vectors of each row is obtained.

[0242] In this embodiment, in order to obtain such Figure 7 The output loss matrix dI shown [M，K] The row weight vector W0 in computing device D0, the row weight vector W1 in computing device D1, the row weight vector W2 in computing device D2, and the row weight vector W3 in computing device D3 can be arranged according to... Figure 7 The data transmission method shown in the figure transmits data sequentially. After each transmission, each computing device calculates the product of its row weight vector and any first output element. After N rounds of calculation, for N computing devices, the products of the row weight vectors and first output elements calculated on their respective devices are summed to obtain the sum of the products of the N first output elements and each row weight vector.

[0243] S209. Integrate the sum of products on each computing device to obtain the output loss matrix.

[0244] The output loss matrix is ​​the matrix used to train the Transformer network.

[0245] In this embodiment, after obtaining the sum of the products of the N first output elements and each row weight vector, and integrating the sums of the products of the N first output elements and each row weight vector, the output loss matrix dI is obtained. [M，K] .

[0246] In this embodiment, the transpose of the weight matrix is ​​divided according to the row dimension, and the obtained output matrix is ​​divided into N... 2 The first output element allows each computing device to pass vectors sequentially according to the order of the computing devices during multiple rounds of computation, avoiding the data delay caused by the need for cross-passing vectors in existing systems. This effectively reduces the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also partitions the transpose matrix of the weight matrix and the output matrix, avoiding the copying of tensors in different devices and reducing storage overhead.

[0247] In one embodiment, in Figure 6 Based on the illustrated embodiment, the process of obtaining the sum of the products between each of the N first output elements and the weight vector of each row can be described, such as... Figure 8 As shown, the above S208 "based on set communication, after N rounds of calculation, the sum of the products between the N first output elements and the weight vectors of each row is obtained" includes:

[0248] S401. For any computing device, in the first round of calculation, obtain the product of the row weight vector in the computing device and any one of the N first output elements.

[0249] In this embodiment, on the aforementioned computing device D0, a row weight vector W0 and four first output elements (O, W0 ... 00 O 01 O 02 and O 03 On computing device D1, row weight vector W1 and four first output elements (O1, O2, and O3, O4, O5, O6, O7, O8, O9 ... 10 O 11 O 12 and O 13 On computing device D2, row weight vector W2 and four first output elements (O2, O2, and O2) are deployed. 20 O 21 O 22 and O 23 On computing device D3, row weight vector W3 and four first output elements (O3, O2, and O2) are deployed. 30 O 31 O 32 and O 33 After that, during the first round of calculation, the row weight vector W0 and the first output element O are obtained on the computing device D0. 00 The product of and gives I 00 Obtain the row weight vector W1 and the first output element O on computing device D1. 10 The product of and gives I 10 Obtain the row weight vector W2 and the first output element O on computing device D2. 20 The product of and gives I 20 Obtain the row weight vector W3 and the first output element O on computing device D3. 30 The product of and gives I 30 .

[0250] S402. In the second to Nth rounds of calculation, perform the following steps: obtain the row weight vectors in the adjacent computing devices, and use the row weight vectors in the adjacent computing devices as the new row weight vectors.

[0251] In this embodiment, after the first round of calculation is completed, during the second round, computing device D1 sends its row weight vector W1 to computing device D0, computing device D2 sends its row weight vector W2 to computing device D1, computing device D3 sends its row weight vector W3 to computing device D2, and computing device D0 sends its row weight vector W0 to computing device D3. Therefore, during the second round of calculation, computing device D0 will deploy the row weight vector W1 and four first output elements (0, ... 00 O 01 O 02 and O 03 On computing device D1, row weight vector W2 and four first output elements (O2, O2, and O2) will be deployed. 10 O 11 O 12 and O 13 On computing device D2, row weight vector W3 and four first output elements (O3, O4, and O5) will be deployed. 20 O 21 O 22 and O 23 ), and on computing device D3, row weight vector W0 and four first output elements (O, O, and W0 respectively) will be deployed. 30 O 31 O 32 and O 33 ).

[0252] In the third round of computation, computing device D1 sends its row weight vector W2 to computing device D0, computing device D2 sends its row weight vector W3 to computing device D1, computing device D3 sends its row weight vector W0 to computing device D2, and computing device D0 sends its row weight vector W1 to computing device D3. Therefore, in the third round of computation, computing device D0 will have row weight vector W2 and four first output elements (0, ... 00 O 01 O 02 and O 03 On computing device D1, row weight vector W3 and four first output elements (O3, O4, and O5) will be deployed. 10 O 11 O 12 and O 13 On computing device D2, row weight vector W0 and four first output elements (O, W0 ... 20 O 21 O 22 and O 23 ), and on computing device D3, row weight vector W1 and four first output elements (O, O, and W1 respectively) will be deployed. 30 O31 O 32 and O 33 ).

[0253] In the fourth round of computation, computing device D1 sends its row weight vector W3 to computing device D0, computing device D2 sends its row weight vector W0 to computing device D1, computing device D3 sends its row weight vector W1 to computing device D2, and computing device D0 sends its row weight vector W2 to computing device D3. Therefore, in the fourth round of computation, computing device D0 will have row weight vector W3 and four first output elements (0, ... 00 O 01 O 02 and O 03 On computing device D1, row weight vector W0 and four first output elements (O, W0 ... 10 O 11 O 12 and O 13 On computing device D2, row weight vector W1 and four first output elements (O1, O2, and O2) will be deployed. 20 O 11 O 12 and O 13 ), and on computing device D3, row weight vector W2 and four first output elements (O, O, and O) will be deployed. 30 O 31 O 32 and O 33 ).

[0254] S403. Determine the product of the new row weight vector and the target first output element to obtain the sum of the products between the N first output elements and each row weight vector; the target first output element refers to the first output element other than any one of the N first output elements.

[0255] In this embodiment, during the second round of calculation, for computing device D0, it deploys a row weight vector W1 and four first output elements (O1, O2, and O3 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the row weight vector W1 and the first output element O. 01 The product of and gives I 01 For computing device D1, it deploys a row weight vector W2 and four first output elements (O2, O2, and O2, ... 10 O 11 O 12 and O 13 Then, directly calculate the row weight vector W2 and the first output element O.11 The product of and gives I 11 For computing device D2, it deploys row weight vector W3 and four first output elements (O2, O2, and O2 respectively). 20 O 21 O 22 and O 23 Then, directly calculate the row weight vector W3 and the first output element O. 21 The product of and gives I 21 For computing device D3, it deploys a row weight vector W0 and four first output elements (O, W0 ... 30 O 31 O 32 and O 33 Then, directly calculate the row weight vector W0 and the first output element O. 31 The product of and gives I 31 .

[0256] In the third round of computation, for computing device D0, it deploys row weight vector W2 and four first output elements (O2, O2, and O2 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the row weight vector W2 and the first output element O. 02 The product of and gives I 02 For computing device D1, it deploys row weight vector W3 and four first output elements (O2, O3, O4, O5, O6, O7, O8, O9 ... 10 O 11 O 12 and O 13 Then, directly calculate the row weight vector W3 and the first output element O. 12 The product of and gives I 12 For computing device D2, it deploys a row weight vector W0 and four first output elements (O, W0 ... 20 O 21 O 22 and O 23 Then, directly calculate the row weight vector W0 and the first output element O. 22 The product of and gives I 22 For computing device D3, it deploys row weight vector W1 and four first output elements (O1, O2, and O3 respectively). 30 O 31 O 32 and O 33 Then, directly calculate the row weight vector W1 and the first output element O. 32 The product of and gives I 32 .

[0257] In the fourth round of computation, for computing device D0, it deploys row weight vector W3 and four first output elements (O3, O4, and O5 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the row weight vector W3 and the first output element O. 03 The product of and gives I 03 For computing device D1, it deploys a row weight vector W0 and four first output elements (O, W0 ... 10 O 11 O 12 and O 13 Then, directly calculate the row weight vector W0 and the first output element O. 13 The product of and gives I 13 For computing device D2, it deploys a row weight vector W1 and four first output elements (O1, O2, and O2 respectively). 20 O 21 O 22 and O 23 Then, directly calculate the row weight vector W1 and the first output element O. 23 The product of and gives I 23 For computing device D3, it deploys row weight vector W2 and four first output elements (O2, O2, and O2 respectively). 30 O 31 O 32 and O 33 Then, directly calculate the row weight vector W2 and the first output element O. 33 The product of and gives I 33 .

[0258] In summary, for the case where N=4, after the above four rounds of calculations, the computing device D0 obtained four products, including: I 00 I 01 I 02 I 03 The sum of these four products is obtained as I0. The computing device D1 then generates four products, including: I... 10 I 11 I 12 I 13 Adding these four products together yields I1. The computing device D2 then generates four products, including: I... 20 I 21 I 22 I 23 Adding these four products together yields I2. The computing device D2 then displays four products, including: I... 30 I 31 I32 I 33 Adding these four products together gives I3, which gives the result as follows: Figure 7 The output matrix O shown [M，K] .

[0259] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0260] In one embodiment, in the above Figure 3 Based on the illustrated embodiment, after obtaining the output matrix as described above, the matrix partitioning process used in the gradient update process of the Transformer network can also be described. See [link to documentation]. Figure 9 The above methods also include:

[0261] S210. Split the transpose of the input matrix into N first-column input vectors according to the column dimension.

[0262] Please refer to formula (3) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. Then, for the transpose of the input matrix I... T [N,M] This is done by splitting the input matrix into N first-column input vectors according to the column dimension (i.e., the M dimensions of the input matrix). For example, as... Figure 10 As shown, the transpose of the input matrix I is... T [N,M] The input vectors are split into four first-column vectors according to the column dimension: I0, I1, I2, and I3.

[0263] S211. Split the output matrix into 2N second output elements according to the column dimension and the row dimension; the number of splits in the row dimension is N-1.

[0264] Please refer to formula (3) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. Therefore, for the output matrix dO[M,K], it is split into 2N second output elements according to the row dimension and column dimension, that is, split it N-1 times according to the row dimension and 1 time according to the column dimension to obtain 2N second output elements. For example, as... Figure 10 As shown, the output matrix dO[M,K] is split into 8 second output elements, i.e., O 00 O 01 O10 O 11 O 20 O 21 O 30 and O 31 .

[0265] S212. On each of the N different computing devices, deploy a set of corresponding first column input vectors and two second output elements.

[0266] In this embodiment, after obtaining N first column input vectors and 2N second output elements, a corresponding set of first column input vectors and two second output elements are deployed on each of the N different computing devices. For example, on computing device D0, the first column input vector I0 and two second output elements O are deployed. 00 and O 01 On computing device D1, the first column input vector I1 and two second output elements O are deployed. 10 and O 11 On computing device D2, the first column input vector I2 and two second output elements O are deployed. 20 and O 21 On computing device D3, the first column input vector I3 and two second output elements O are deployed. 30 and O 31 .

[0267] S213. Based on set communication, the sum of the products between each first column input vector and each pair of second output elements is obtained through two rounds of calculation.

[0268] In this embodiment, in order to obtain such Figure 10 The weight gradient matrix dW shown [N，K] After two rounds of calculation, the sum of the products between each first column input vector and each pair of second output elements is obtained.

[0269] S214. Integrate the sum of the products between each first column input vector and every two second output elements to obtain the weight gradient matrix.

[0270] The weight gradient matrix is ​​the matrix used to train the Transformer network.

[0271] In this embodiment, after obtaining the sum of the products between each first column input vector and every two second output elements, the sum of the products between each first column input vector and every two second output elements is integrated to obtain the weight gradient matrix dW. [N，K] .

[0272] In this embodiment, by dividing the transpose of the input matrix according to the column dimension and the output matrix according to the column and row dimensions, each computing device only needs to pass vectors sequentially according to the order of the computing devices during multiple rounds of computation. This avoids the drawback of data delay caused by the need for cross-passing of vectors in existing systems, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the transpose of the input matrix and the output matrix to avoid copying tensors in different devices and reduce storage overhead.

[0273] In one embodiment, in Figure 9 Based on the illustrated embodiment, the process of obtaining the sum of the products between each first column input vector and every two second output elements can be described, such as... Figure 11 As shown, the above S213 "based on set communication, the sum of the products between each first column input vector and every two second output elements is obtained through two rounds of calculation" includes:

[0274] S501. In the first round of calculation, obtain the product of the first column input vector and any second output element in each computing device.

[0275] In this embodiment, during the first round of computation in each computing device, the product of the first column input vector and any second output element in each computing device is obtained. For example, for computing device D0, it deploys the first column input vector I0 and the second output element O. 00 and O 01 Then we directly calculate the first column input vector I0 and the second output element O. 00 The product of these two products yields W. 00 For computing device D1, it has a first input vector I1 and a second output element O. 10 and O 11 Then we directly calculate the first column input vector I1 and the second output element O. 10 The product of these two products yields W. 10 For computing device D2, it has a first input vector I2 and a second output element O. 20 and O 21 Then we can directly calculate the first column input vector I2 and the second output element O. 20 The product of these two products yields W. 20 For computing device D3, it deploys a first column of input vector I3 and a second output element θ. 30 and O 31 Then we can directly calculate the first column input vector I3 and the second output element O. 30 The product of these two products yields W. 30 .

[0276] S502. In the second round of calculation, while obtaining the sum of the products of the first column input vector and any second output element in each computing device, the product of the first column input vector and another second output element in each computing device is also obtained.

[0277] In this embodiment, after completing the first round of calculations, the sum of the products of the first column input vector and any second output element in each computing device can be obtained, i.e., W0 + W1 = W 00 +W 10 +W 20 +W 30 And while obtaining the sum of the products of the first column input vector and any second output element in each computing device, in the second round of calculation of each computing device, the product of the first column input vector and any second output element in each computing device is obtained. That is, for computing device D0, it deploys the first column input vector I0 and the second output element O. 01 Then we directly calculate the first column input vector I0 and the second output element O. 01 The product of these two products yields W. 01 For computing device D1, it has a first input vector I1 and a second output element O. 11 Then we directly calculate the first column input vector I1 and the second output element O. 11 The product of these two products yields W. 11 For computing device D2, it has a first column input vector I2 and a second output element O. 21 Then we can directly calculate the first column input vector I2 and the second output element O. 21 The product of these two products yields W. 21 For computing device D3, it has a first input vector I3 and a second output element O. 31 Then we can directly calculate the first column input vector I3 and the second output element O. 31 The product of these two products yields W. 31 .

[0278] S503. Obtain the sum of the products of the first column input vector and the other second output element in each computing device.

[0279] In this embodiment, after completing the second round of calculations, the sum of the products of the first column input vector and the other second output element in each computing device can be obtained, i.e., W2 + W3 = W 01 +W 11 +W 21 +W 31 .

[0280] It should be noted that W0+W1 are assigned to computing devices D0 and D1 respectively, and W2+W3 are assigned to computing devices D2 and D3 respectively.

[0281] In this embodiment, while obtaining the sum of the products of the first column input vector and any second output element in each computing device, the product of the first column input vector and another second output element in each computing device is also obtained. This can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0282] In one embodiment, such as Figure 12 As shown, a method for M-dimensional tensor partitioning of a high-performance computing chip architecture is also provided, including:

[0283] S1. During the training process of the Transformer network, the input matrix in the Transformer network is split into N first-row input vectors according to the row dimension;

[0284] S2. Split the weight matrix in the Transformer network into N first-column weight vectors according to the column dimension;

[0285] S3. On each of the N different computing devices, deploy a set of corresponding first row input vectors and first column weight vectors.

[0286] S4. For any computing device, in the first round of calculation, obtain the product of the first row of input vector and the first column of weight vector in the computing device;

[0287] S5. In the calculation from the 2nd to the Nth round, perform the following steps: obtain the first column weight vector in the adjacent computing device, and determine the first column weight vector in the adjacent computing device as the new first column weight vector;

[0288] S6. Determine the product of the first row input vector and the new first column weight vector to obtain the product of each first row input vector and each first column weight vector;

[0289] S7. Integrate the product of each first row input vector and each first column weight vector to obtain the output matrix;

[0290] S8. Split the transpose of the weight matrix into N row weight vectors according to the row dimension;

[0291] S9. On each of N different computing devices, deploy a set of corresponding row weight vectors and N first output elements in the output matrix; the output matrix includes N... 2 The first output element;

[0292] S10. For any computing device, in the first round of calculation, obtain the product of the row weight vector in the computing device and any one of the N first output elements;

[0293] S11. In the calculation from the 2nd to the Nth round, perform the following steps: obtain the row weight vector in the adjacent computing device, and use the row weight vector in the adjacent computing device as the new row weight vector;

[0294] S12. Determine the product of the new row weight vector and the target first output element to obtain the sum of the products between the N first output elements and each row weight vector; the target first output element refers to the first output element other than any one of the N first output elements.

[0295] S13. Integrate the sum of products on each computing device to obtain the output loss matrix;

[0296] S14. Split the transpose of the input matrix into N input vectors in the first column according to the column dimension;

[0297] S15. Split the output matrix into 2N second output elements according to the column dimension and the row dimension; the number of splits in the row dimension is N-1.

[0298] S16. On each of the N different computing devices, deploy a set of corresponding first column input vectors and two second output elements;

[0299] S17. In the first round of calculation, obtain the product of the first column input vector and any second output element in each computing device;

[0300] S18. In the second round of calculation, while obtaining the sum of the products of the first column input vector and any second output element in each computing device, the product of the first column input vector and another second output element in each computing device is also obtained.

[0301] S19. Obtain the sum of the products of the first column input vector and the other second output element in each computing device;

[0302] S20. Integrate the sum of the products between each first column input vector and each N second output elements to obtain the weight gradient matrix.

[0303] It should be noted that the descriptions in S1-S20 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0304] In the third scenario, we will finally introduce the K-dimensional tensor partitioning method for high-performance chip computing architectures corresponding to type III K-dimensional partitioning.

[0305] In one embodiment, such as Figure 23 As shown, a K-dimensional tensor partitioning method for a high-performance computing chip architecture is provided, and this method is applied to... Figure 1 Taking a computer device as an example, the explanation includes the following steps:

[0306] S901. During the training of the Transformer network, the transpose of the input matrix in the Transformer network is split into N column transpose vectors according to the column dimension.

[0307] Please refer to formula (3) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. Then, for the transpose of the input matrix I... T [N，M] This is split into N column transpose vectors according to the column dimension. For example, as shown below. Figure 24 As shown, the transpose of the input matrix I is... T [N，M] The input is split into four transpose vectors along the column dimension: I0, I1, I2, and I3.

[0308] S902. Split the output matrix of the Transformer network into N columns and rows according to the column and row dimensions. 2 The third output element.

[0309] Please refer to formula (3) above. For gradient updates of the Transformer model, the tensors that need to be split include the transpose of the input matrix and the output matrix. So, for the output matrix dO[M,K], it is split into N according to the row dimension and the row dimension. 2 The third output element is obtained by splitting it N-1 times along the row dimension and N-1 times along the column dimension, resulting in N. 2 The third output element. For example, such as Figure 24 As shown, the output matrix dO[ M [K] is split into 16 second output elements, i.e., O 00 O 01 O 02 O 03 O 10 O 11 O 12 O 13 O 20 O 21 O 22 O 23 O 30 O 31 O 32 and O 33 .

[0310] S903. On each of N different computing devices, deploy a set of corresponding column input transpose vectors and N third output elements in the output matrix; the output matrix includes N... 2 The third output element.

[0311] In this embodiment, after obtaining the N column input transpose vectors and N... 2 Following the third output element, on each of the N different computing devices, a corresponding set of column input transpose vectors and N third output elements are deployed. For example, on computing device D0, the column input transpose vector I0 and the third output element O are deployed. 00 O 01 O 02 O 03 On computing device D1, the column input transpose vector I1 and the third output element O are deployed. 10 O 11 O 12 O 13 On computing device D2, deploy the column input transpose vector I2 and the third output element O. 20 O 21 O 22 O 23 On computing device D3, deploy the column input transpose vector I3 and the third output element O. 30 O 31 O 32 O 33 .

[0312] S904. Based on set communication, after N rounds of calculation, the sum of the products between the N third output elements and the transpose vectors of each column input is obtained.

[0313] In this embodiment, in order to obtain such Figure 24 The weight gradient matrix dW shown [N，K] After N rounds of calculation, the sum of the products of the N third output elements and the transpose of each column of input vectors is obtained.

[0314] S905. Integrate the sum of products on each computing device to obtain the weight gradient matrix.

[0315] Wherein, the weight gradient matrix is ​​the matrix used to train the Transformer network. In this embodiment, after obtaining the sum of the products between the N third output elements and the transpose of each column of input vectors, the sum of the products between the N third output elements and the transpose of each column of input vectors is integrated to obtain the weight gradient matrix dW. [N，K] .

[0316] In this embodiment, by dividing the transpose of the input matrix according to the column dimension and the output matrix according to the column and row dimensions, each computing device only needs to pass vectors sequentially according to the order of the computing devices during multiple rounds of computation. This avoids the drawback of data delay caused by the need for cross-passing vectors in existing methods, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the transpose of the input matrix and the output matrix to avoid copying tensors in different devices and reduce storage overhead.

[0317] In one embodiment, in Figure 23 Based on the illustrated embodiment, the process of obtaining the sum of the products of N third output elements and the transpose of each column of input vectors can be described, such as... Figure 25 As shown, the above S904, "based on set communication, after N rounds of calculation, the sum of the products of N third output elements and the transpose vectors of each column of input are obtained", includes:

[0318] S1001. For any computing device, in the first round of calculation, obtain the product of the column input transpose vector and any one of the N third output elements in the computing device.

[0319] In this embodiment, on the aforementioned computing device D0, the column input transpose vector I0 and the third output element O are deployed. 00 O 01 O 02 O 03 On computing device D1, the column input transpose vector I1 and the third output element O are deployed. 10 O 11 O 12 O 13 On computing device D2, deploy the column input transpose vector I2 and the third output element O. 20 O 21 O 22 O 23 On computing device D3, deploy the column input transpose vector I3 and the third output element O. 30 O 31 O 32 O 33 Subsequently, during the first round of calculations, the column input transpose vector I0 and the third output element O are obtained on computing device D0. 00 The product of these two products yields W. 00 Obtain the column input transpose vector I1 and the third output element O on computing device D1. 10 The product of these two products yields W. 10 Obtain the column input transpose vector I2 and the third output element O on computing device D2. 20The product of these two products yields W. 20 Obtain the column input transpose vector I3 and the third output element O on computing device D3. 30 The product of these two products yields W. 30 .

[0320] S1002. In the second to Nth rounds of calculation, perform the following steps: obtain the column input transpose vector in the adjacent computing device, and use the column input transpose vector in the adjacent computing device as the new column input transpose vector.

[0321] In this embodiment, after the first round of calculation is completed, during the second round, computing device D1 sends its column input transpose vector I1 to computing device D0, computing device D2 sends its column input transpose vector I2 to computing device D1, computing device D3 sends its column input transpose vector I3 to computing device D2, and computing device D0 sends its column input transpose vector I0 to computing device D3. Therefore, during the second round of calculation, computing device D0 will deploy the column input transpose vector I1 and four first output elements (0, ... 00 O 01 O 02 and O 03 The computing device D1 will then deploy the column input transpose vector I2 and four first output elements (O2, O ... 10 O 11 O 12 and O 13 The computing device D2 will then deploy the column input transpose vector I3 and four first output elements (O3, O4, and O5, O6, O7, O8, O9 ... 20 O 21 O 22 and O 23 ), and the computing device D3 will deploy the column input transpose vector I0 and four first output elements (O, O, and O respectively). 30 O 31 O 32 and O 33 ).

[0322] In the third round of computation, computing device D1 sends its column input transpose vector I2 to computing device D0, computing device D2 sends its column input transpose vector I3 to computing device D1, computing device D3 sends its column input transpose vector I0 to computing device D2, and computing device D0 sends its column input transpose vector I1 to computing device D3. Therefore, in the third round of computation, computing device D0 will have the column input transpose vector I2 and four first output elements (O1, O2, and O2 respectively). 00 O 01 O 02 and O 03The computing device D1 will then deploy the column input transpose vector I3 and four first output elements (O3, O4, O5, O6, O7, O8, O9 ... 10 O 11 O 12 and O 13 The computing device D2 will then deploy the column input transpose vector I0 and four first output elements (O1, O2, O3, O4, O5, O6, O7, O8, O9 ... 20 O 21 O 22 and O 23 ), and the computing device D3 will deploy the column input transpose vector I1 and four first output elements (O1, O2, and O3 respectively). 30 O 31 O 32 and O 33 ).

[0323] In the fourth round of computation, computing device D1 sends its column input transpose vector I3 to computing device D0, computing device D2 sends its column input transpose vector I0 to computing device D1, computing device D3 sends its column input transpose vector I1 to computing device D2, and computing device D0 sends its column input transpose vector I2 to computing device D3. Therefore, in the fourth round of computation, computing device D0 will have the column input transpose vector I3 and four first output elements (O1, O2, and O2 respectively). 00 O 01 O 02 and O 03 The computing device D1 will then deploy the column input transpose vector I0 and four first output elements (O1, O2, and O3 respectively). 10 O 11 O 12 and O 13 The computing device D2 will then deploy the column input transpose vector I1 and four first output elements (O1, O2, O3, O4, O5, O6, O7, O8, O9 ... 20 O 21 O 22 and O 23 ), and the computing device D3 will deploy the column input transpose vector I2 and four first output elements (O, O ... 30 O 31 O 32 and O 33 ).

[0324] S1003. Determine the product of the new column input transpose vector and the target third output element to obtain the sum of the products of N third output elements and each column input transpose vector; the target third output element refers to the third output element other than any one of the N third output elements.

[0325] In this embodiment, during the second round of calculation, for computing device D0, it deploys a column input transpose vector I1 and four first output elements (O1, O2, and O3 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the transpose vector I1 and the first output element O. 01 The product of these two products yields W. 01 For computing device D1, it deploys a column input transpose vector I2 and four first output elements (O2, O ... 10 O 11 O 12 and O 13 Then, directly calculate the column input transpose vector I2 and the first output element O. 11 The product of these two products yields W. 11 For computing device D2, it deploys a column input transpose vector I3 and four first output elements (O2, O3, O2, O3, O4, O5, O6, O7, O8, O9 ... 20 O 21 O 22 and O 23 Then, directly calculate the column input transpose vector I3 and the first output element O. 21 The product of these two products yields W. 21 For computing device D3, it deploys a column input transpose vector I0 and four first output elements (O1, O2, and O3 respectively). 30 O 31 O 32 and O 33 Then, directly calculate the column input transpose vector I0 and the first output element O. 31 The product of these two products yields W. 31 .

[0326] In the third round of computation, for computing device D0, it deploys the column input transpose vector I2 and four first output elements (O2, O2, and O2 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the column input transpose vector I2 and the first output element O. 02 The product of these two products yields W. 02 For computing device D1, it deploys a column input transpose vector I3 and four first output elements (O3, O4, and O5, respectively). 10 O 11 O 12 and O 13 Then, directly calculate the column input transpose vector I3 and the first output element O. 12 The product of these two products yields W. 12For computing device D2, it deploys a column input transpose vector I0 and four first output elements (O1, O2, and O2, O3, O4, O5, O6, O7, O8, O9 ... 20 O 21 O 22 and O 23 Then, directly calculate the column input transpose vector I0 and the first output element O. 22 The product of these two products yields W. 22 For computing device D3, it deploys a column input transpose vector I1 and four first output elements (O1, O2, and O3 respectively). 30 O 31 O 32 and O 33 Then, directly calculate the column input transpose vector I1 and the first output element O. 32 The product of these two products yields W. 32 .

[0327] In the fourth round of computation, for computing device D0, it deploys the column input transpose vector I3 and four first output elements (O3, O4, and O5 respectively). 00 O 01 O 02 and O 03 Then, directly calculate the column input transpose vector I3 and the first output element O. 03 The product of these two products yields W. 03 For computing device D1, it deploys a column input transpose vector I0 and four first output elements (O1, O2, and O3 respectively). 10 O 11 O 12 and O 13 Then, directly calculate the column input transpose vector I0 and the first output element O. 13 The product of these two products yields W. 13 For computing device D2, it deploys a column input transpose vector I1 and four first output elements (O1, O2, and O2 respectively). 20 O 21 O 22 and O 23 Then, directly calculate the column input transpose vector I1 and the first output element O. 23 The product of these two products yields W. 23 For computing device D3, it deploys a column input transpose vector I2 and four first output elements (O2, O ... 30 O 31 O 32 and O 33 Then, directly calculate the column input transpose vector I2 and the first output element O. 33 The product of these two products yields W. 33 .

[0328] In summary, for the case where N=4, after the above four rounds of calculations, the computing device D0 obtained four products, including: W 00 W 01 W 02 W 03 The sum of these four products is W0. The computing device D1 then generates four products, including: W... 10 W 11 W 12 W 13 The four products are summed to obtain W1. The computing device D2 then obtains four products, including: W... 20 W 21 W 22 W 23 Adding these four products together yields W2. The computing device D2 then displays four products, including: W... 30 W 31 W 32 W 33 Add these four products together to get W3, which gives the result as follows: Figure 24 The weight gradient matrix dW shown [N，K] .

[0329] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0330] In one embodiment, in the above Figure 23 Based on the illustrated embodiment, the matrix partitioning process used in the forward computation of the Transformer network can also be described, see [link to documentation]. Figure 26 The above methods also include:

[0331] S906. Split the input matrix in the Transformer network into N second-row input vectors according to the row dimension.

[0332] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. Then, for the input matrix I[ M N ] This is split into N second-row input vectors according to the row dimension. For example, as shown below. Figure 27 As shown, the input matrix I [M，N]The input vectors are split into four second-row input vectors according to the row dimension: I0, I1, I2, and I3.

[0333] S907. Split the weight matrix in the Transformer network into N second-column weight vectors according to the column dimension.

[0334] Please refer to formula (1) above. For the forward computation of the Transformer model, the tensors that need to be split include the input matrix and the weight matrix. So, for the weight matrix W... [N，K] This is done by splitting it into N second-column weight vectors according to the column dimension (i.e., the K dimensions of the weight matrix). For example, as... Figure 27 As shown, the weight matrix W [N，K] It is split into four second-column weight vectors according to the column dimension, namely W0, W1, W2 and W3.

[0335] S908. On each of the N different computing devices, deploy a set of corresponding second row input vectors and second column weight vectors.

[0336] In this embodiment, after obtaining N second-row input vectors and N second-column weight vectors as described above, a corresponding set of second-row input vectors and second-column weight vectors is deployed on each of the N different computing devices. For example, on computing device D0, the second-row input vector I0 and the second-column weight vector W0 are deployed; on computing device D1, the second-row input vector I1 and the second-column weight vector W1 are deployed; on computing device D2, the second-row input vector I2 and the second-column weight vector W2 are deployed; and on computing device D3, the second-row input vector I3 and the second-column weight vector W3 are deployed.

[0337] S909, based on set communication, calculates the product of each second row input vector and each second column weight vector through N rounds of computation.

[0338] In this embodiment, in order to obtain such Figure 27 The output matrix O shown [M，K] The second row input vector I0 in computing device D0, the second row input vector I1 in computing device D1, the second row input vector I2 in computing device D2, and the second row input vector I3 in computing device D3 can be processed according to... Figure 27 The data transmission method shown in the figure transmits data sequentially. After each transmission, each computing device calculates the product of the second row input vector and each second column weight vector. After N rounds of calculation, the product of each second row input vector and each second column weight vector is obtained.

[0339] S910. Integrate the product of each second row input vector and each second column weight vector to obtain the output matrix.

[0340] The output matrix is ​​the matrix used to train the Transformer network.

[0341] In this embodiment, after obtaining the product of each second-row input vector and each second-column weight vector, the product of each second-row input vector and each second-column weight vector is integrated to obtain the output matrix O. [M，K] .

[0342] In this embodiment, by dividing the input matrix only according to the row dimension and the weight matrix only according to the column dimension, each computing device only needs to pass vectors sequentially according to the order of the computing devices when performing multiple rounds of calculations. This avoids the drawback of data delay caused by the need for cross-passing of vectors in the existing method, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the input matrix and the weight matrix to avoid copying tensors in different devices and reduce storage overhead.

[0343] In one embodiment, in Figure 26 Based on the illustrated embodiment, the process of obtaining the product of each second-row input vector and each second-column weight vector can be described, such as... Figure 28 As shown, the above-mentioned S909 "based on set communication, through N rounds of calculation to obtain the product of each second row input vector and each second column weight vector" includes:

[0344] S1101. For any computing device, in the first round of calculation, obtain the product of the second row input vector and the second column weight vector in the computing device.

[0345] In this embodiment, for any one of the N computing devices, in the first round of calculation for that computing device, the product of the second row input vector and the second column weight vector is directly determined. For example, for computing device D0, which deploys the second row input vector I0 and the second column weight vector W0, the product of the second row input vector I0 and the second column weight vector W0 is directly calculated to obtain O. 00 For computing device D1, which has a second row of input vector I1 and a second column of weight vector W1, we can directly calculate the product of the second row of input vector I1 and the second column of weight vector W1 to obtain O. 11 For computing device D2, which has a second row of input vector I2 and a second column of weight vector W2, we can directly calculate the product of the second row of input vector I2 and the second column of weight vector W2 to obtain O. 22For computing device D3, which has a second row of input vector I3 and a second column of weight vector W3, we can directly calculate the product of the second row of input vector I3 and the second column of weight vector W3 to obtain O. 33 .

[0346] S1102. In the second to Nth rounds of computation, perform the following steps: obtain the second row input vector from the adjacent computing device, and determine the second row input vector from the adjacent computing device as the new second row input vector.

[0347] In this embodiment, after the first round of calculation is completed, in the second round of calculation, computing device D1 sends its second row input vector I1 to computing device D0, computing device D2 sends its second row input vector I2 to computing device D1, computing device D3 sends its second row input vector I3 to computing device D2, and computing device D0 sends its second row input vector I0 to computing device D3. Therefore, in the second round of calculation, computing device D0 will deploy the second row input vector I1 and the second column weight vector W0, computing device D1 will deploy the second row input vector I2 and the second column weight vector W1, computing device D2 will deploy the second row input vector I3 and the second column weight vector W2, and computing device D3 will deploy the second row input vector I0 and the second column weight vector W3.

[0348] In the third round of computation, computing device D1 sends its second row input vector I2 to computing device D0, computing device D2 sends its second row input vector I3 to computing device D1, computing device D3 sends its second row input vector I0 to computing device D2, and computing device D0 sends its second row input vector I1 to computing device D3. Therefore, in the third round of computation, computing device D0 will have its second row input vector I2 and second column weight vector W0 deployed, computing device D1 will have its second row input vector I3 and second column weight vector W1 deployed, computing device D2 will have its second row input vector I0 and second column weight vector W2 deployed, and computing device D3 will have its second row input vector I1 and second column weight vector W3 deployed.

[0349] In the fourth round of computation, computing device D1 sends its second row input vector I3 to computing device D0, computing device D2 sends its second row input vector I0 to computing device D1, computing device D3 sends its second row input vector I1 to computing device D2, and computing device D0 sends its second row input vector I2 to computing device D3. Therefore, in the fourth round of computation, computing device D0 will deploy the second row input vector I3 and the second column weight vector W0, computing device D1 will deploy the second row input vector I0 and the second column weight vector W1, computing device D2 will deploy the second row input vector I1 and the second column weight vector W2, and computing device D3 will deploy the second row input vector I2 and the second column weight vector W3.

[0350] S1103. Determine the product of the second column weight vector and the new second row input vector to obtain the product of each second row input vector and each second column weight vector.

[0351] In this embodiment, during the second round of calculation, for computing device D0, which has deployed a second row input vector I1 and a second column weight vector W0, the product of the second row input vector I1 and the second column weight vector W0 is directly calculated to obtain O. 01 For computing device D1, which has a second row of input vector I2 and a second column of weight vector W1, the product of the second row of input vector I2 and the second column of weight vector W1 is directly calculated to obtain O. 12 For computing device D2, which has a second row input vector I3 and a second column weight vector W2, the product of the second row input vector I3 and the second column weight vector W2 is directly calculated to obtain O. 23 For computing device D3, which has a second row of input vector I0 and a second column of weight vector W3, the product of the second row of input vector I0 and the second column of weight vector W3 is directly calculated to obtain O. 30 .

[0352] In the third round of calculation, for computing device D0, which has a second row of input vector I2 and a second column of weight vector W0, the product of the second row of input vector I2 and the second column of weight vector W0 is directly calculated to obtain O. 02 For computing device D1, which has a second row of input vector I3 and a second column of weight vector W1, the product of the second row of input vector I3 and the second column of weight vector W1 is directly calculated to obtain O. 13 For computing device D2, which has a second row of input vector I0 and a second column of weight vector W2, we can directly calculate the product of the second row of input vector I0 and the second column of weight vector W2 to obtain O. 20For computing device D3, which has a second row of input vector I1 and a second column of weight vector W3, we can directly calculate the product of the second row of input vector I1 and the second column of weight vector W3 to obtain O. 31 .

[0353] In the fourth round of calculation, for computing device D0, which has a second row input vector I3 and a second column weight vector W0, the product of the second row input vector I3 and the second column weight vector W0 is directly calculated to obtain O. 03 For computing device D1, which has a second row of input vector I0 and a second column of weight vector W1, the product of the second row of input vector I0 and the second column of weight vector W1 is directly calculated to obtain O. 10 For computing device D2, which has a second row of input vector I1 and a second column of weight vector W2, we can directly calculate the product of the second row of input vector I1 and the second column of weight vector W2 to obtain O. 21 For computing device D3, which has a second row of input vector I2 and a second column of weight vector W3, we can directly calculate the product of the second row of input vector I2 and the second column of weight vector W3 to obtain O. 32 .

[0354] In summary, for the case where N=4, after the above four rounds of calculations, we can obtain the following result: Figure 27 The output matrix O shown [M，K] .

[0355] In this embodiment, when each computing device performs multiple rounds of computation, it only needs to pass vectors sequentially according to the order of the computing devices. This avoids the drawback of data delay caused by the need for cross-passing vectors in the existing method. It can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0356] In one embodiment, in the above Figure 26 Based on the illustrated embodiment, after obtaining the output matrix as described above, the matrix partitioning process used in the backpropagation process of the Transformer network can also be described. See [link to documentation]. Figure 29 The above methods also include:

[0357] S911. Split the output matrix into N column output vectors according to the column dimension.

[0358] Please refer to formula (2) above. For backpropagation of the Transformer model, the tensors that need to be split include the transpose of the output matrix and the weight matrix. Then, for the output matrix dO [M,N] This is then split into N column output vectors according to the column dimension. For example, as shown below. Figure 30 As shown, the output matrix dO [M,N] The output vector is split into four columns according to the column dimension: O0, O1, O2, and O3.

[0359] S912. The transpose of the weight matrix is ​​split into 2N weight transpose elements according to the row and column dimensions.

[0360] Please refer to formula (2) above. For backpropagation of the Transformer model, the tensors that need to be split include the output matrix and the transpose of the weight matrix. Then, for the transpose of the weight matrix I... T [N，M] This is split into 2N weighted transpose elements based on row and column dimensions. For example, as... Figure 30 As shown, the transpose of the weight matrix is ​​I. T [N，M] The data is split three times along the row dimension and once along the column dimension, resulting in eight weighted transpose elements, namely W. 00 W 01 W 10 W 11 W 20 W 21 W 30 W 31 .

[0361] S913. On each of the N different computing devices, deploy a set of corresponding column output vectors and two weight transpose elements.

[0362] In this embodiment, after obtaining N column output vectors and 2N weight transpose elements, a corresponding set of column output vectors and two weight transpose elements are deployed on each of the N different computing devices. For example, on computing device D0, column output vector O0 and weight transpose element W are deployed. 00 W 01 On computing device D1, deploy column output vector O1 and weight transpose element W. 10 W 11 On computing device D2, deploy column output vector O2 and weight transpose element W. 20 W 21 On computing device D3, deploy column output vector O3 and weight transpose element W. 30 W 31 .

[0363] S914. Based on set communication, the sum of the products of each column output vector and the two weight transpose elements is obtained after two rounds of calculation.

[0364] In this embodiment, in order to obtain such Figure 30 The output loss matrix dW shown [N，K] After two rounds of calculation, the sum of the product of the column output vector and the two weight transpose elements is obtained.

[0365] S915. Combine the sum of the products of each column output vector and the transpose of the two weights to obtain the output loss matrix.

[0366] The output loss matrix is ​​the matrix used to train the Transformer network.

[0367] In this embodiment, after obtaining the sum of the products of each column output vector and the two transpose elements of the weights, the sum of the products of each column output vector and the two transpose elements of the weights is integrated to obtain the output loss matrix dW. [N，K] .

[0368] In this embodiment, by dividing the output matrix according to the column dimension and the transpose of the weight matrix according to the column and row dimensions, each computing device only needs to pass vectors sequentially according to the order of the computing devices when performing multiple rounds of calculations. This avoids the drawback of data delay caused by the need for cross-passing of vectors in the existing method, and can effectively reduce the tensor storage and communication overhead under distributed training of neural networks. In addition, this application also divides the output matrix and the transpose of the weight matrix to avoid copying tensors in different devices and reduce storage overhead.

[0369] In one embodiment, in Figure 29 Based on the illustrated embodiment, the process of obtaining the sum of the products of each column output vector and the two weight transpose elements can be described, such as... Figure 31 As shown, the above S914, "based on set communication, after two rounds of calculation, the sum of the products of each column output vector and the two weight transpose elements is obtained," includes:

[0370] S1201. In the first round of calculation, obtain the product of the column output vector and any weight transpose element in each computing device.

[0371] In this embodiment, during the first round of computation in each computing device, the product of the column output vector and any weight transpose element in each computing device is obtained. For example, for computing device D0, it deploys column output vector O0 and weight transpose element W. 00 W 01 Then we can directly calculate the column output vector O0 and the weight transpose element W. 00 The product of and gives I 00 For computing device D1, it deploys column output vector O1 and weight transpose element W. 10 W 11Then we can directly calculate the column output vector O1 and the weight transpose element W. 10 The product of and gives I 10 For computing device D2, it deploys column output vector O2 and weight transpose element W. 20 W 21 Then we can directly calculate the column output vector O2 and the weight transpose element W. 20 The product of and gives I 20 For computing device D3, it deploys column output vector O3 and weight transpose element W. 30 W 31 Then we can directly calculate the column output vector O3 and the weight transpose element W. 30 The product of and gives I 30 .

[0372] S1202. In the second round of calculation, while obtaining the sum of the products of the column output vector and any weight transpose element in each computing device, the product of the column output vector and another weight transpose element in each computing device is also obtained.

[0373] In this embodiment, after completing the first round of calculations, the sum of the products of the column output vectors and any weight transpose element in each computing device can be obtained, i.e., I0 + I1 = I 00 +I 10 +I 20 +I 30 In addition to obtaining the sum of the products of the column output vectors and any weight transpose element in each computing device, in the second round of calculation for each computing device, the product of the column output vectors and another weight transpose element in each computing device is obtained. That is, for computing device D0, it deploys column output vector O0 and weight transpose element W. 00 W 01 Then we can directly calculate the column output vector O0 and the weight transpose element W. 01 The product of and gives I 01 For computing device D1, it deploys column output vector O1 and weight transpose element W. 10 W 11 Then we can directly calculate the column output vector O1 and the weight transpose element W. 11 The product of and gives I 11 For computing device D2, it deploys column output vector O2 and weight transpose element W. 20 W 21 Then we can directly calculate the column output vector O2 and the weight transpose element W. 21 The product of and gives I 21 For computing device D3, it deploys column output vector O3 and weight transpose element W. 30 W 31Then we can directly calculate the column output vector O3 and the weight transpose element W. 31 The product of and gives I 31 .

[0374] S1203. Obtain the sum of the products of the column output vectors and the transpose of another weight in each computing device.

[0375] In this embodiment, after completing the second round of calculations, the sum of the products of the column output vectors and another weight transpose element in each computing device can be obtained, i.e., I2 + I3 = I 01 +I 11 +I 21 +I 31 .

[0376] It should be noted that I0+I1 are assigned to computing devices D0 and D1 respectively, and I2+I3 are assigned to computing devices D2 and D3 respectively.

[0377] In this embodiment, while obtaining the sum of the products of the column output vectors and any weight transpose element in each computing device, the product of the column output vectors and another weight transpose element in each computing device is also obtained. This can effectively reduce the tensor storage and communication overhead under distributed training of neural networks and improve the training and application efficiency of Transformer networks.

[0378] In one embodiment, such as Figure 32 As shown, a K-dimensional tensor partitioning method for a high-performance computing chip architecture is also provided, including:

[0379] Q1. During the training of the Transformer network, the transpose of the input matrix in the Transformer network is split into N column transpose vectors according to the column dimension.

[0380] Q2. Decompose the output matrix of the Transformer network into N columns and rows. 2 The third output element;

[0381] Q3. On each of N different computing devices, deploy a set of corresponding column input transpose vectors and N third output elements in the output matrix; the output matrix includes N... 2 The third output element;

[0382] Q4. For any computing device, in the first round of computation, obtain the product of the transpose of the column input vector and any one of the N third output elements in the computing device.

[0383] Q5. In the calculation from round 2 to round N, perform the following steps: obtain the column input transpose vectors in the adjacent computing devices, and use the column input transpose vectors in the adjacent computing devices as the new column input transpose vectors;

[0384] Q6. Determine the product of the new column input transpose vector and the target third output element to obtain the sum of the products of N third output elements and each column input transpose vector; the target third output element refers to the third output element other than any one of the N third output elements.

[0385] Q7. Combine the sum of the products on each computing device to obtain the weight gradient matrix;

[0386] Q8. Split the input matrix in the Transformer network into N second-row input vectors according to the row dimension;

[0387] Q9. Split the weight matrix in the Transformer network into N second-column weight vectors according to the column dimension;

[0388] Q10. On each of N different computing devices, deploy a set of corresponding second-row input vectors and second-column weight vectors.

[0389] Q11. For any computing device, in the first round of computation, obtain the product of the second row of input vector and the second column of weight vector in the computing device;

[0390] Q12. In rounds 2 to N of computation, perform the following steps: obtain the second row of input vectors from adjacent computing devices, and determine the second row of input vectors from adjacent computing devices as the new second row of input vectors;

[0391] Q13. Determine the product of the second column weight vector and the new second row input vector to obtain the product of each second row input vector and each second column weight vector;

[0392] Q14. Combine the products of each second-row input vector and each second-column weight vector to obtain the output matrix;

[0393] Q15. Split the output matrix into N column output vectors according to the column dimension;

[0394] Q16. Divide the transpose of the weight matrix into 2N weight transpose elements according to the row and column dimensions;

[0395] Q17. On each of N different computing devices, deploy a set of corresponding column output vectors and two weight transpose elements.

[0396] Q18. In the first round of calculation, obtain the product of the column output vector and any weight transpose element in each computing device;

[0397] Q19. In the second round of calculation, while obtaining the sum of the products of the column output vectors in each computing device and any weight transpose element, obtain the product of the column output vectors in each computing device and another weight transpose element.

[0398] Q20. Obtain the sum of the products of the column output vectors and the transpose of another weight in each computing device;

[0399] Q21. Combine the sum of the products of each column output vector and the transpose of the two weights to obtain the output loss matrix.

[0400] It should be noted that the descriptions in Q1-Q21 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0401] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0402] Based on the same inventive concept, this application also provides an N-dimensional tensor partitioning device for a high-performance chip computing architecture, used to implement the aforementioned N-dimensional tensor partitioning method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more embodiments of the N-dimensional tensor partitioning device for a high-performance chip computing architecture provided below can be found in the limitations of the N-dimensional tensor partitioning method for high-performance chip computing architecture described above, and will not be repeated here.

[0403] In one exemplary embodiment, such as Figure 33 As shown, an N-dimensional tensor partitioning device for a high-performance computing chip architecture is provided, comprising: a second row partitioning module 20, a second column partitioning module 21, a second deployment module 22, a second computing module 23, and a second acquisition module 24, wherein:

[0404] The second row splitting module 20 is used to split the input matrix in the Transformer network into N second column input vectors according to the column dimension during the training process of the Transformer network.

[0405] The second column splitting module 21 is used to split the weight matrix in the Transformer network into 2N column weight elements according to the column dimension and the row dimension; the number of splits in the row dimension is N-1.

[0406] The second deployment module 22 is used to deploy a set of corresponding second column input vectors and two column weight elements on each of the N different computing devices.

[0407] The second calculation module 23 is used to obtain the product of each second column input vector and the weight elements of the two columns through two rounds of calculation based on set communication.

[0408] The second acquisition module 24 is used to integrate the product of each second column input vector and the weight elements of the two columns to obtain the output matrix.

[0409] In an exemplary embodiment, the second calculation module 23 includes: a first acquisition unit, a second acquisition unit, and a third acquisition unit, wherein:

[0410] The first acquisition unit is specifically used to acquire, in the first round of calculation, the product of the second column input vector and the weight element of any column in any computing device.

[0411] The second acquisition unit is specifically used in the second round of calculation to acquire the sum of the products of the second column input vector and the weight element of another column in the computing device, and at the same time acquire the product of the second column input vector and any column weight element in each computing device.

[0412] The third acquisition unit is specifically used to acquire the sum of the products of the second column input vector and the weight elements of another column in each computing device.

[0413] In one exemplary embodiment, the above-described apparatus further includes:

[0414] The row splitting module is used to split the output matrix into N row output vectors according to the row dimension;

[0415] The row and column splitting module is used to split the transpose of the weight matrix in the Transformer network into N column weight transpose vectors according to the column dimension;

[0416] The deployment module is used to split the transpose of the weight matrix in the Transformer network into N column weight transpose vectors according to the column dimension;

[0417] The first acquisition module is used to obtain the product of each row output vector and each column weight transpose vector through N rounds of calculation based on set communication.

[0418] The second acquisition module is used to integrate the product of each row output vector and each column weight transpose vector to obtain the output loss matrix.

[0419] In an exemplary embodiment, the first acquisition module described above includes:

[0420] The first acquisition unit is specifically used to acquire, in the first round of calculation, the product of the row output vector and the column weight transpose vector in any computing device.

[0421] The second acquisition unit is specifically used to perform the following steps in the second to Nth rounds of calculation: acquire the column weight transpose vector in the adjacent computing device, and determine the column weight transpose vector in the adjacent computing device as the new column weight transpose vector;

[0422] The unit is specifically used to determine the product of the row output vector and the new column weight transpose vector, thus obtaining the product of each row output vector and each column weight transpose vector.

[0423] In one exemplary embodiment, the above-described apparatus further includes:

[0424] The row and column splitting module is used to split the transpose of the input matrix into N parts according to the row and column dimensions. 2 Each row of input elements;

[0425] The row splitting module is used to split the output matrix into N row output vectors according to the row dimension;

[0426] The deployment module is used to deploy a set of corresponding row output vectors and N row input elements on each of N different computing devices;

[0427] The calculation module is used to obtain the sum of the products between the N row input elements and the output vectors of each row through N rounds of calculation based on set communication.

[0428] The integration module is used to integrate the sum of products on each computing device to obtain the weight gradient matrix.

[0429] In one exemplary embodiment, the above-described computing module includes:

[0430] The acquisition unit is specifically used to acquire, in the first round of calculation, the product of the row output vector and any one of the N row input elements in the computing device;

[0431] The second acquisition unit is specifically used to perform the following steps in the second to Nth rounds of calculation: acquire the row output vectors in the adjacent computing devices, and use the row output vectors in the adjacent computing devices as new row output vectors;

[0432] The determining unit is specifically used to determine the product of the new row output vector and the target input element, so as to obtain the sum of the products between the N row input elements and each row output vector; the target input element refers to the row input element other than any one of the N row input elements.

[0433] Each module in the N-dimensional tensor partitioning device of the aforementioned high-computing-power chip architecture can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independent of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can call and execute the operations corresponding to each module.

[0434] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0435] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0436] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0437] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0438] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0439] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for N-dimensional tensor partitioning in a high-performance computing chip architecture, characterized in that, The method includes: During the training of the Transformer network, the input matrix in the Transformer network is split into N second-column input vectors according to the column dimension; The weight matrix in the Transformer network is split into 2N column weight elements according to the column dimension and the row dimension; the row dimension is split N-1 times. On each of the N different computing devices, a set of corresponding second column input vectors and two column weight elements are deployed. Based on the set communication method, the sum of the products of each second column input vector and the weight elements of the two columns is obtained after two rounds of calculation; The output matrix is ​​obtained by combining the product of each second column input vector with the weight elements of the two columns.

2. The method according to claim 1, characterized in that, The method based on set communication calculates the product of each second column input vector and the weight elements of the two columns through two rounds of computation, including: For any computing device, in the first round of calculation, the product of the second column input vector and any column weight element in the computing device is obtained; In the second round of calculation, while obtaining the sum of the products of the second column input vector and the weight element of another column in the computing device, the product of the second column input vector and any one of the column weight elements in each computing device is also obtained. Obtain the sum of the products of the second column input vector and the weight element of the other column in each of the computing devices.

3. The method according to claim 1, characterized in that, The method further includes: The output matrix is ​​split into N row output vectors according to the row dimension; The transpose of the weight matrix in the Transformer network is split into N column weight transpose vectors according to the column dimension; On each of the N different computing devices, deploy a set of corresponding row output vectors and column weight transpose vectors. Based on the set communication method, after N rounds of calculation, the product of each row output vector and each column weight transpose vector is obtained; The output loss matrix is ​​obtained by combining the product of the output vector of each row and the transpose of the weights of each column.

4. The method according to claim 3, characterized in that, Based on set communication, after N rounds of computation, the product of each row output vector and each column weight transpose vector is obtained, including: For any computing device, in the first round of calculation, the product of the row output vector and the column weight transpose vector in the computing device is obtained; In rounds 2 to N of the calculation, the following steps are performed: obtain the column weight transpose vectors in adjacent computing devices, and determine the column weight transpose vectors in the adjacent computing devices as new column weight transpose vectors; Determine the product of the row output vector and the new column weight transpose vector to obtain the product of each row output vector and each column weight transpose vector.

5. The method according to claim 1, characterized in that, The method further includes: The transpose of the input matrix is ​​split into N parts according to the row and column dimensions. 2 Each row of input elements; The output matrix is ​​split into N row output vectors according to the row dimension; On each of the N different computing devices, a set of corresponding row output vectors and N row input elements are deployed; Based on set communication, the sum of the products between the N row input elements and each row output vector is obtained after N rounds of calculation; The sum of the products on each computing device is combined to obtain the weight gradient matrix.

6. The method according to claim 5, characterized in that, The set-based communication method, after N rounds of calculation, obtains the sum of the products between each of the N row input elements and each row output vector, including: For any computing device, in the first round of calculation, the product of the row output vector in the computing device and any one of the N row input elements is obtained; In rounds 2 to N of computation, the following steps are performed: obtain the row output vectors from adjacent computing devices, and use the row output vectors from the adjacent computing devices as new row output vectors; Determine the product of the new row output vector and the target input element to obtain the sum of the products between the N row input elements and each row output vector; the target input element refers to the row input element other than any of the N row input elements.

7. An N-dimensional tensor partitioning device for a high-computing-power chip architecture, characterized in that, The device includes: The second row splitting module is used to split the input matrix in the Transformer network into N second column input vectors according to the column dimension during the training process of the Transformer network. The second column splitting module is used to split the weight matrix in the Transformer network into 2N column weight elements according to the column dimension and the row dimension; the number of splits in the row dimension is N-1. The second deployment module is used to deploy a set of corresponding second column input vectors and two column weight elements on each of N different computing devices. The second calculation module is used to obtain the product of each second column input vector and the weight elements of the two columns through two rounds of calculation based on set communication. The second acquisition module is used to integrate the product of each second column input vector and the weight elements of the two columns to obtain the output matrix.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images