Data processing architecture, chip, matrix multiplication computation method, and neural network computation method

Abstract

A data processing architecture, a chip, a matrix multiplication computation method, and a neural network computation method. The data processing architecture (20) comprises a node array having P rows and Q columns. P×Q nodes (10) in the node array respectively store first sub-matrices obtained by partitioning a first matrix X on which matrix multiplication is to be performed, and second sub-matrices obtained by partitioning a second matrix W on which matrix multiplication is to be performed. Each node (10) is configured to separately send the first sub-matrix stored in the node to the other Q-1 nodes in the same row as the node, separately send the second sub-matrix stored in the node to the other P-1 nodes in the same column as the node, and on the basis of sub-matrices stored in and received by the node, compute a result sub-matrix corresponding to the node, wherein the sub-matrices include the first sub-matrices and the second sub-matrices, and the result sub-matrices of the P×Q nodes constitute a result matrix Y obtained by performing matrix multiplication on the first matrix X and the second matrix W.

Description

A data processing architecture, chip, matrix multiplication, and neural network computation method Technical Field

[0001] This application relates to, but is not limited to, the field of data processing technology, and in particular to a data processing architecture, chip, matrix multiplication, and neural network computation method. Background Technology

[0002] Deep learning is a special form of neural network that learns complex representations of data by using multi-layered neural network structures. Deep learning has a wide range of applications in large models, such as image recognition and generation, natural language processing, speech recognition and synthesis, video generation, robot dialogue, and autonomous driving. It is evident that deep learning has broad application prospects in the future information transmission field. However, this involves a large number of matrix multiplication calculations, and the matrices involved are of a large dimension. Facing the problem of large-scale matrix multiplication calculations, improving the processing speed of matrix multiplication calculations has become an urgent technical challenge. Summary of the Invention

[0003] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0004] This application provides a data processing architecture, including: a node array of P rows and Q columns; the P×Q nodes in the node array respectively store a first sub-matrix obtained by partitioning a first matrix X to be multiplied, and a second sub-matrix obtained by partitioning a second matrix W to be multiplied;

[0005] The node is configured to send the first submatrix stored by the node to the other Q-1 nodes in the same row as the node, send the second submatrix stored by the node to the other P-1 nodes in the same column as the node, and calculate the result submatrix corresponding to the node based on the submatrix stored and received by the node; wherein, the submatrix includes the first submatrix and the second submatrix; the result submatrixes of the P×Q nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W.

[0006] On the other hand, embodiments of this application provide a matrix multiplication calculation method, which is implemented based on the above-described data processing architecture and includes:

[0007] Each node in the data processing architecture sends the first sub-matrix stored in this node to the other Q-1 nodes in the same row as this node, and sends the second sub-matrix stored in this node to the other P-1 nodes in the same column as this node;

[0008] Each node calculates its corresponding result submatrix based on the submatrix stored in the node and the received submatrix;

[0009] The submatrix includes a first submatrix and a second submatrix; the result submatrixes of P×Q nodes form the first matrix X and the result matrix Y of matrix multiplication of the second matrix W; the first submatrix stored by P×Q nodes forms the first matrix X; and the second submatrix stored by P×Q nodes forms the second matrix W.

[0010] Furthermore, embodiments of this application provide a neural network computation method, wherein each layer in the neural network model performs the following steps:

[0011] Using the input matrix of the current layer as the first matrix X and the weight matrix of the current layer as the second matrix W, perform matrix multiplication Y = X × W according to the matrix multiplication calculation method described above.

[0012] Calculate the result matrix Y* for the current layer based on Y;

[0013] Y* is input into the next layer of the neural network model as the first matrix X of the next layer, or Y* is used as the calculation result of the neural network.

[0014] In another aspect, embodiments of this application provide a chip, including the data processing architecture described above.

[0015] After reading and understanding the accompanying diagrams and detailed descriptions, the other aspects can be understood.

[0016] Overview of the attached figures

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0018] Figure 1 is a schematic diagram of the data processing architecture of an embodiment of this application;

[0019] Figure 2 is a schematic diagram of the layout of the data processing architecture in an exemplary embodiment;

[0020] Figure 3 is a schematic diagram of the node structure in an exemplary embodiment;

[0021] Figure 4 is a schematic diagram of a 4×4 node matrix in an example;

[0022] Figure 5 is a schematic diagram of a 2×4 node array storing a submatrix in an example;

[0023] Figure 6 is a flowchart of the matrix multiplication calculation method according to an embodiment of this application;

[0024] Figure 7 is a schematic diagram of a 4×4 node matrix in an example;

[0025] Figure 8 is a flowchart of the neural network calculation method according to an embodiment of this application;

[0026] Figure 9 is a three-dimensional schematic diagram of the chip in an exemplary embodiment;

[0027] Figure 10 is a cross-sectional schematic diagram of a chip in an exemplary embodiment.

[0028] Detailed Explanation

[0029] This disclosure describes several embodiments, but these descriptions are exemplary and not limiting, and it will be apparent to those skilled in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in detail, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature or element of any embodiment may be used in combination with, or may replace, any feature or element of any other embodiment.

[0030] The terms "first" and "second," etc., used in this disclosure and its accompanying drawings are intended to distinguish different objects or different treatments of the same object, and not to describe a particular order of objects. It should be understood that such use of data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.

[0031] Generally, in deep learning computations of large models, matrix multiplication Y[M][N] = X[M][L] × W[L][N] is frequently required. After matrix multiplication, matrix addition is also required, and the total computation is 2MLN multiplication and addition. Since the computational tasks in large models are usually quite complex, the dimensions L and N of the weight matrix W are generally in the range of several thousand to tens of thousands. The value of M varies depending on the application scenario and the stage of computation, and is generally between tens and hundreds of thousands. The total computation is very large, and low-cost and fast matrix multiplication has become an inevitable requirement for the development of large models.

[0032] This application proposes a data processing architecture 20, as shown in Figure 1, including: a node array of P rows and Q columns; the P×Q nodes 10 in the node array respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied;

[0033] The node 10 is configured to send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and to send the second submatrix stored in this node to the other P-1 nodes in the same column as this node; and to calculate the result submatrix corresponding to this node based on the submatrix stored in this node and the received submatrix; wherein, the submatrix includes the first submatrix and the second submatrix; the result submatrixes of the P×Q nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W.

[0034] In this article, node 10 is represented by E. p,q The subscript indicates the position of a node in the node array; p represents the p-th row in the node array, q represents the q-th column in the node array, p = 1, 2, ..., P, q = 1, 2, ..., Q.

[0035] In this embodiment, the data processing architecture 20 is a distributed two-dimensional node array architecture, with node E p,q In this context, 'p' represents the x-coordinate of the node in the node array, meaning 'p' indicates that the node is located in the p-th row of the node array; node E p,q In this context, q represents the vertical coordinate of the node in the node array, that is, q means that the node is located in the qth column of the node array.

[0036] In this embodiment, there are P×Q nodes E p,q The corresponding submatrix can be directly concatenated according to the position of the corresponding node in the node array (i.e., the row and column number) to obtain the final result of matrix multiplication of the first and second matrices; for example, assuming each node E p,q Each corresponds to a result submatrix Y pq Then the resulting submatrix Y with P×Q nodes pq We can concatenate them to obtain Y[M][N], that is: Y[M][N] can be regarded as a result submatrix array, where the first row is Y 11 Y 12 , ..., Y 1Q The second line is Y. 21 Y 22 , ..., Y 2Q The last line is Y. P1 Y P2 , ..., Y PQ And so on. When there are multiple result submatrices in each node, the multiple result submatrices in the node form a result subarray. The result subarrays corresponding to multiple nodes are concatenated according to the position of the corresponding node in the node array to obtain Y[M][N].

[0037] In this embodiment, the data processing architecture 20 has P×Q nodes Ep,q It can directly or indirectly connect to other nodes E in the same row and column of the node array. p,q This allows for the transmission and reception of the first and second submatrices; P×Q nodes E p,q It can, but is not limited to, forming a two-dimensional mesh network or a dedicated data link, with each pair of adjacent nodes E p,q They can communicate point-to-point or broadcast by row and column.

[0038] In this embodiment, each node E in the data processing architecture 20 p,q Based on the first submatrix stored in this node, the first submatrix received from the other Q-1 nodes in the same row, the second submatrix stored in this node, and the second submatrix received from the other P-1 nodes in the same row, calculate one or more result submatrixes corresponding to this node. That is, calculate the corresponding result submatrix based on the first submatrix in the Q nodes in the same row and the second submatrix in the P nodes in the same column.

[0039] In this embodiment, P and Q may be equal or unequal.

[0040] In this embodiment, the first submatrix derived from the first matrix can be viewed as an array of first submatrixes, and the second submatrix derived from the second matrix can be viewed as an array of second submatrixes. If each node E p,q If only one first submatrix and one second submatrix are stored, then the first matrix is ​​divided into P rows and Q columns of first submatrixes, and the second matrix is ​​divided into P rows and Q columns of second submatrixes; each node E p,q The positions of the saved first submatrix and the saved second submatrix within the first matrix are identical to the position of the node in the node array. If each node E... p,q It stores multiple first submatrices and multiple second submatrices, and any node E p,q The position of the matrix formed by the multiple first sub-matrices stored in the first matrix, and the position of the matrix formed by the multiple second sub-matrices stored in the second matrix, are the same as the position of the node in the node array. Here, position refers to the row and column number of the node or sub-matrix array.

[0041] In this embodiment, a node E p,q Each node can store one or more first submatrices and one or more second submatrices. The size of each first submatrix and each second submatrix can be the same. If each node contains only one first submatrix and one second submatrix, then the number of rows in the first submatrix is ​​M / P, the number of columns is L / Q, and the number of rows in the second submatrix is ​​L / P, and the number of columns is N / Q.

[0042] In this embodiment, the first submatrix, divided by the first matrix, can be evenly distributed across P×Q nodes E. p,q In the second matrix, the second submatrix can be evenly distributed across P×Q nodes E. p,q In this case, each node has the same number of first submatrices and the same number of second submatrices; M and L are integer multiples of P, and L and N are integer multiples of Q. Alternatively, M and L do not necessarily have to be integer multiples of P, and L and N do not necessarily have to be integer multiples of Q. The number of first submatrices and the number of second submatrices in each node are not necessarily the same, but in the array of first submatrices, the total number of columns in a row of all elements in the first submatrices is equal to the total number of rows in a column of all elements in the second submatrices, that is, the number of columns in the first matrix is ​​equal to the number of rows in the second matrix. Thus, for a distributed two-dimensional node array with a fixed number of rows and columns, this is applicable to matrix multiplication calculations of any structure, has high applicability, and is easy to promote and apply.

[0043] The data processing architecture 20 in this embodiment can divide two large matrices undergoing matrix multiplication into multiple relatively smaller sub-matrices, which are then distributed across multiple nodes for storage and computation. This allows each node to perform only the sub-matrix multiplication and store the resulting sub-matrices, significantly reducing costs. This data processing architecture provides a parallel architecture for the storage and arrangement of large-scale matrices, with each node E in the node array... p,q The system can compute submatrices in partial or full parallelism. Submatric transmission between nodes only requires communication within the same row and column. Each node is responsible for its own small matrix computation task, based on the first submatrix stored by Q nodes in the same row and the second submatrix stored by P nodes in the same column. This provides the hardware foundation for converting large-scale matrix multiplication operations into parallel computations of multiple small matrices, significantly reducing the time required for large-scale matrix multiplication operations. Furthermore, all nodes E in the node array... p,q The structure can be identical, and the number of nodes or columns can be increased according to actual needs. It is highly scalable, easy to implement standardized design and integration, and the performance between each node is consistent, which improves the reliability of the data processing architecture. When a single node fails, the faulty node can be replaced. It is easy to maintain, user-friendly and environmentally friendly, and reduces the maintenance cost of the data processing architecture.

[0044] In one exemplary embodiment, each node includes: a computing sub-node and a storage module;

[0045] The storage module is configured to save the first submatrix, the second submatrix, and the result submatrix;

[0046] The computed child nodes include:

[0047] The processing module is configured to read the first and second sub-matrices from the storage module; calculate the corresponding result sub-matrices for this node based on the sub-matrices stored and received by this node, and save them to the storage module.

[0048] The row data communication module is configured to send the first submatrix read by the processing module to the other Q-1 nodes in the same row as this node, and to receive the first submatrix from the other Q-1 nodes in the same row.

[0049] The column data communication module is configured to send the second submatrix read by the processing module to the other P-1 nodes in the same column as this node, and to receive the second submatrix from the other P-1 nodes in the same column.

[0050] In one exemplary embodiment of this example, the data processing architecture is shown in Figure 2. The row data communication module and the column communication module can be integrated into a network module, and the nodes communicate with each other through the network module. The storage module can include a temporary storage module and a memory module. Each node sends and receives sub-matrices through the network module and temporarily stores them in the temporary storage for the processing module to perform matrix multiplication calculations. The calculated multiplication results are temporarily stored in the temporary storage module and finally saved to the memory module. The sub-matrices saved by a node can all be stored in the memory module, or some can be stored in the cache and the other part can be stored in the memory module.

[0051] In one exemplary embodiment of this invention, the storage module includes: a temporary storage module set in the computational sub-node, configured to store the first sub-matrix, the second sub-matrix, and the result sub-matrix.

[0052] In this embodiment, each node may only include computational sub-nodes, and the first sub-matrix, the second sub-matrix, and the result sub-matrix may all be stored in the temporary storage module, without setting up additional memory sub-nodes.

[0053] In another exemplary embodiment of this invention, the storage module includes: a temporary storage module disposed in the computing sub-node, and a memory sub-node connected to the computing sub-node;

[0054] The memory child nodes are configured to store the first submatrix, the second submatrix, and the result submatrix;

[0055] The temporary storage module is configured to temporarily store the first and second sub-matrices read from the memory sub-nodes by the processing module, as well as the result sub-matrices calculated by the processing module;

[0056] The processing module is also configured to save the result submatrix temporarily stored in the temporary storage module to a memory subnode.

[0057] As shown in Figure 3, each node 10 may include:

[0058] Compute child node 30, and memory child node 40 connected to compute child node 30;

[0059] Memory sub-node 40 is configured to store the first submatrix, the second submatrix, and the result submatrix;

[0060] Calculating child node 30 includes:

[0061] Temporary storage module 31;

[0062] Processing module 32 is configured to read the first and second sub-matrices to be sent from the memory sub-nodes and temporarily store them in the temporary storage module; calculate the result sub-matrix corresponding to this node based on the first and second sub-matrices temporarily stored in the temporary storage module and save it to the memory sub-node;

[0063] The row data communication module 33 is configured to send the first sub-matrix temporarily stored in the temporary storage module 31 by the processing module 32 to the other Q-1 nodes in the same row as this node, and to receive the first sub-matrix from the other Q-1 nodes in the same row and temporarily store it in the temporary storage module 31.

[0064] The column data communication module 34 is configured to send the second sub-matrix temporarily stored in the temporary storage module 31 by the processing module 32 to the other P-1 nodes in the same column as the current node, and to receive the second sub-matrix from the other P-1 nodes in the same column and temporarily store it in the temporary storage module 31.

[0065] In this embodiment, the submatrix to be calculated by the processing module 32 is temporarily stored in the temporary storage module 31. The submatrix to be calculated includes two types: one is a submatrix stored in the memory child node of the local node and to be sent out; the other is a submatrix received from other nodes. The processing module 32 can use some or all of the submatrixes in the temporary storage module 31 during calculation. Which submatrixes to use can be selected by instructing the processing module 32 through programming or other means. A submatrix can be uniquely identified by its name (e.g., x, or w) and coordinates (represented in this document as the subscript of x or w).

[0066] In this embodiment, P×Q computational sub-nodes 30 and P×Q memory sub-nodes 40 are connected one-to-one to form a parallel memory access architecture. That is, each computational sub-node 30 has dedicated memory access bandwidth when reading the first and second sub-matrices from the memory sub-node 40, and when writing the result sub-matrices to the memory sub-node 40. Therefore, the computational sub-nodes in the entire node array can read and write to the memory sub-nodes in parallel. After reading the sub-matrices in parallel, the sub-matrices can be sent in parallel, which greatly improves the efficiency of memory access and the efficiency of transmitting sub-matrices, and reduces the need for large-capacity memory, thus reducing hardware costs.

[0067] In this embodiment, the memory sub-node 40 can be DRAM, or it can be one or more storage media such as SRAM, PCM, ReRAM, Flash, etc., and it is connected to the computing sub-node 30 through the memory port 41; the temporary storage module 31 can be, but is not limited to, SRAM, and the processing module 32 can be a matrix calculation circuit.

[0068] In this embodiment, the row data communication module 33 and the column data communication module 34 can reuse some hardware resources. For example, the row data communication module 33 may include a routing module and one or two first interfaces in the row direction. The first interfaces connect adjacent nodes in the same row. The routing module performs operations such as data packet encapsulation, routing, sending, receiving, and decapsulation. The column data communication module 34 may include the routing module and one or two second interfaces in the column direction. The second interfaces connect adjacent nodes in the same column. Alternatively, they can be independent data links.

[0069] In this embodiment, the hardware implementation of the computing sub-nodes and memory sub-nodes is not limited. For example, P×Q computing sub-nodes 30 can form a logic array, which can be packaged in a logic die; P×Q memory sub-nodes 40 can form a memory array, which can be packaged in a memory die. The logic dies and memory dies can be stacked and bonded together. Alternatively, a processing chip with memory can be used as a node to form a chip array. Another example is that different cores in a multi-core processor can be used as different computing sub-nodes. Or, for example, each computing sub-node and its corresponding memory sub-node can be directly connected in the same circuit and packaged in a die. Alternatively, the computing sub-nodes and memory sub-nodes can be bonded together or directly packaged in a single chip.

[0070] In other implementations, the memory sub-node may store only one of the first and second sub-matrices, while the other sub-matrice may be stored in the temporary storage module. For example, in the computation of a neural network, the first sub-matrix, which serves as the input to each layer, is stored in the temporary storage module, while the second sub-matrix, which serves as the weights, may be stored in the memory sub-node.

[0071] In other implementations, multiple computational child nodes can share one or more memory child nodes. Each computational child node reads the first submatrix and the second submatrix through different storage addresses and stores the calculated results in different storage addresses. For example, two memory child nodes can be set up: one stores the first submatrix of all computational child nodes, and the other stores the second submatrix of all computational child nodes.

[0072] In other implementations, a single memory can be used to store the first submatrix, the second submatrix, and the result submatrix in P×Q storage areas arranged in P rows and Q columns. The computational child nodes can access the corresponding storage area through two-dimensional coordinates (such as the storage area in which row and column).

[0073] In one exemplary embodiment, the node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:

[0074] The node E p,q Broadcast the first submatrix stored in this node to the other Q-1 nodes E in the same row as this node. p,q Broadcast the second submatrix stored in this node to the other P-1 nodes E in the same column as this node. p,q .

[0075] In this embodiment, when a node sends data in the same row and in the same column, it uses the same row broadcast and same column broadcast methods to broadcast the first submatrix to other nodes in the same row and the second submatrix to other nodes in the same column.

[0076] In other embodiments, multiple data packets can be sent, i.e., each node sends Q-1 data packets to send the first submatrix stored by the node to the other Q-1 nodes in the same row, and sends P-1 data packets to send the second submatrix stored by the node to the other P-1 nodes in the same column; or the first submatrix can be forwarded row by row and column by column for each node.

[0077] This application does not limit the form in which the first submatrix and the second submatrix are sent, as long as the first submatrix can be sent to other nodes in the same row and the second submatrix can be sent to other nodes in the same column.

[0078] In one exemplary embodiment, the P×Q nodes E p,q The first and second sub-matrices are transmitted in parallel, meaning that the P×Q nodes transmit the first sub-matrice in parallel, and the P×Q nodes transmit the second sub-matrice in parallel, with each node transmitting both the first and second sub-matrices in parallel. Specifically, all nodes can transmit the stored first sub-matrice and the stored second sub-matrice in parallel within the same time period; or, in other words, all P×Q nodes transmit the first sub-matrice and the second sub-matrice within the same time period.

[0079] In another exemplary embodiment, P×Q nodes can be sent in multiple predetermined batches according to a predetermined order, with only a portion of the nodes being sent in each batch.

[0080] In one embodiment of this example, the nodes E in different columns p,q The first sub-matrix can be sent at different time periods, and the nodes E in the same column can be sent separately. p,q The first submatrix is ​​sent in parallel; the nodes E in different rows p,q The second sub-matrix can be sent at different time periods, and the nodes E in the same row can be sent separately. p,q Send the second sub-matrix in parallel.

[0081] In this embodiment, there are P×Q nodes E p,q The transmission is performed column-by-column and row-by-row. That is, nodes in the same column send the first submatrix to other nodes in the same row in parallel, while nodes in different columns send the first submatrix at different times. Nodes in the same row send the second submatrix to other nodes in the same column in parallel, while nodes in different rows send the second submatrix at different times.

[0082] In one exemplary embodiment, each node can send and receive sub-matrices simultaneously within the same time period; that is, it can send and receive sub-matrices concurrently. Each node can send one type of sub-matrix first, and then send another type of sub-matrix. For example, it can send the first sub-matrix stored by its own node first, and then send the second sub-matrix stored by its own node after completing the first sub-matrix. Any node can send all of its first or second sub-matrixes stored at once, or, if a node has multiple first or second sub-matrixes stored, it can send them in multiple installments.

[0083] In one exemplary embodiment, the number of transmission submatrixes or the period can be set according to the size of P and Q in the node array. For example, if P is twice Q, the nodes in two rows and one column of the node array can be set to the transmission state each time or each period of transmission, or a same-column transmission can be performed every two row-to-row transmissions. When P and Q are equal, the nodes in one row and one column of the node array can be set to the transmission state each time or each period of transmission, or a same-column transmission can be performed every time a row-to-row transmission is performed. In one exemplary embodiment, a node E p,q Save a first submatrix X pq and a second submatrix W pq ;

[0084] Node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:

[0085] The node E p,q When the first preset condition is met, the first submatrix X stored in this node will be... pq Send the data to the other Q-1 nodes in the same row as this node; when the second preset condition is met, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.

[0086] In this embodiment, the first preset condition may be: reaching a first preset period, for example, sending when reaching period q, that is: when node E reaches the first period. p,1 Send the first submatrix X p1 When the second cycle is reached, node E p,2 Send the first submatrix X p2 The same logic applies to other cycles, and will not be elaborated here; alternatively, the first preset condition could be: after receiving the first sub-matrix sent by the predetermined node, this node sends the first sub-matrix stored by this node, for example, node E. 1,2 Received from node E 1,1 The first submatrix X sent 11 Then send the first submatrix X stored in this node. 12 Node E 2,3 Received from node E 2,2 The first submatrix X sent 22 Then send the first submatrix X saved in this node. 23 The same logic applies to other nodes, and will not be elaborated here.

[0087] In this embodiment, the second preset condition can be: reaching a second preset period, for example, sending when reaching period p, that is: when node E reaches the first period. 1,q Send the second submatrix W 1q When the second cycle is reached, node E 2,q Send the second submatrix W 2q The same logic applies to other periods, and will not be elaborated here; alternatively, the second preset condition could be: after receiving the second sub-matrix from the predetermined node, this node sends the second sub-matrix stored in this node, for example, node E. 1,2 Received from node E 2,2 The second submatrix W sent 22 Then send the second submatrix W stored in this node. 12 Node E 2,3 Received from node E 3,3 The second submatrix W sent 33 Then send the second submatrix W stored in this node. 23 The same logic applies to other nodes, and will not be elaborated here.

[0088] In this embodiment, the first preset condition and the second preset condition are not limited to the situations listed above.

[0089] In one exemplary embodiment of this example, node E p,q The first submatrix X stored in this node can be saved after power-on or after receiving the command to start the operation. pq It automatically sends the second submatrix W stored in this node to the other Q-1 nodes in the same row as this node. pq Automatically sends to P-1 other nodes in the same column as this node; "automatic sending" means that no other conditions need to be judged, and the power-on or operation command is used as the first and second preset conditions, such as no need to judge the sending period or whether other nodes have completed sending.

[0090] In another exemplary embodiment of this invention, each node can first store the X data saved by its own node. pq Send to the same location, in all X pq After sending is complete, each node will then store the W data that it has saved. pq Send in the same column; or conversely, complete W first. pq Send X, then send X pq .

[0091] In other embodiments, the sending conditions and order of the first and second submatrices can be set according to requirements, as long as it is ensured that the first submatrices saved by each node can be sent to other nodes in the same row, and the second submatrices saved by each node can be sent to other nodes in the same column.

[0092] In one exemplary embodiment, node E p,q The result submatrix corresponding to this node is obtained based on the submatrix stored and received by this node, and may include:

[0093] Node E p,q After receiving all the first sub-matrices stored in the Q-1 other nodes in the same row and all the second sub-matrices stored in the P-1 other nodes in the same column, the first sub-matrices stored in this node and the received first sub-matrices are concatenated according to the order of the first sub-matrices in the first matrix to obtain a first concatenated matrix. The second sub-matrices stored in this node and the received second sub-matrices are concatenated according to the order of the second sub-matrices in the second matrix to obtain a second concatenated matrix. Matrix multiplication is performed on the first concatenated matrix and the second concatenated matrix to obtain the result sub-matrice corresponding to this node.

[0094] The order in which the first submatrix is ​​arranged in the first matrix can refer to the order in which the coordinates representing the column numbers of the first submatrix are in ascending order, and the order in which the second submatrix is ​​arranged in the second matrix can refer to the order in which the coordinates representing the row numbers of the second submatrix are in ascending order.

[0095] In another exemplary embodiment, splicing is not necessary; the first matrix is ​​divided into R rows and S columns of first submatrices X. ij The second matrix is ​​divided into S rows and T columns of second submatrices W. hk ; i represents the first submatrix X ij The index of the row in the first matrix, j, indicates the first submatrix X. ij The column number in the first matrix; h represents the second submatrix W. hk The row number in the second matrix, k, indicates the second submatrix W. hk The column index in the second matrix; i = 1, 2, ..., R, j = 1, 2, ..., S, h = 1, 2, ..., S, k = 1, 2, ..., T;

[0096] In this embodiment, node E p,q The result submatrix corresponding to this node is obtained based on the submatrix stored and received by this node, and may include:

[0097] The node E p,q Among the saved and received first and second sub-matrices, find a first sub-matrix X in which j and h are equal. ij and a second submatrix W hk As a pair of submatrices, perform matrix multiplication on each pair of submatrices and sum the matrix multiplication results corresponding to the same result submatrix.

[0098] In this embodiment, in a pair of submatrices, the number of columns in the first submatrix is ​​equal to the number of rows in the second submatrix.

[0099] In this embodiment, the first matrix can be viewed as a first sub-matrix array divided into R rows and S columns, where the first sub-matrix X... ij In the i-th row and j-th column of the first submatrix array; the second matrix can be viewed as an S-row, T-column submatrix array, where the second submatrix W... hk In the h-th row and k-th column of the second submatrix array.

[0100] In one exemplary embodiment of this example, R = S = T = P = Q, the node array is a square matrix, and the total number of the first submatrix and the total number of the second submatrix are both equal to the total number of nodes. Each node stores one first submatrix and one second submatrix. The position or two-dimensional coordinates of the submatrix in the matrix are the same as the position or two-dimensional coordinates of the node in the node array. For example, if the node array is 2 rows and 3 columns, and the first matrix is ​​divided into 2 rows and 3 columns of first submatrix, then the first submatrix in the first row and first column is located at E. 1,1 The first submatrix in the 2nd row and 3rd column is located in E. 2,3 The same applies to the second submatrix.

[0101] In other implementations, P and Q, R and S, and S and T need not be equal; it is sufficient that each first submatrix has a pairable second submatrix during calculation, and the number of columns of the first submatrix is ​​equal to the number of rows of the paired second submatrix.

[0102] In one exemplary embodiment, each node E p,q Store a first submatrix X respectively pq and a second submatrix W pq When P = Q, P × Q nodes can send the first submatrix X in P cycles. pq Second submatrix W pq ;

[0103] In this embodiment, node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:

[0104] Node E p,q In the q-th period, store the first submatrix X at this node. pq Send the data to the other Q-1 nodes in the same row as this node; in the p-th cycle, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.

[0105] In this embodiment, the node array is a square matrix. Taking the square matrix with P=Q=4 shown in Figure 4 as an example, the 4×4 square matrix is ​​divided into 4 periods for the first submatrix X. pq Second submatrix W pq The transmission; the following uses several nodes in this 4×4 node matrix as examples to introduce the transmission of the submatrix, for example, node E. 1,1 In the first cycle, store the first submatrix X at this node. 11 Send to node E respectively 1,2 E1,3 E 1,4 The second submatrix W stored in this node 11 Send to node E respectively 2,1 E 3,1 E 4,1 Node E 2,3 In the second cycle, the second submatrix W stored at this node will be... 23 Send to node E respectively 1,3 E 3,3 E 4,3 In the third cycle, the first submatrix X stored at this node will be... 23 Send to node E respectively 2,1 E 2,2 E 2,4 Node E 4,2 In the second cycle, save the first submatrix X of this node. 42 Send to node E respectively 4,1 E 4,3 E 4,4 In the fourth cycle, the second submatrix W stored at this node will be... 42 Send to node E respectively 1,2 E 2,2 E 3,2 The same logic applies to other nodes, and will not be elaborated here.

[0106] In this embodiment, the multiplication of the first submatrix and the second submatrix can be performed once after each cycle of transmission is completed, or the submatrixes can be paired or concatenated before matrix multiplication is performed after all nodes have transmitted the first and second submatrixes.

[0107] In other embodiments, P and Q may not be equal, and the first and second sub-matrices may not be transmitted periodically; for example, all nodes may first send the first sub-matrices stored in their own nodes in sequence, and then send the second sub-matrices stored in their own nodes in sequence; or, for another example, all nodes may send the first and second sub-matrices in parallel.

[0108] In one exemplary embodiment of this invention, one node corresponds to one result submatrix Y. pq It is possible to obtain Q-1 first submatrices X pq and P-1 second submatrices W pq Then, horizontally concatenate the first submatrix X stored in this node and the first submatrix received, according to the ascending order of q. pq The first concatenated matrix is ​​obtained, and then the second submatrix W, stored in this node and received, is concatenated vertically according to the ascending order of p. pqThe second concatenated matrix is ​​obtained, and the first concatenated matrix and the second concatenated matrix are multiplied to obtain the resulting submatrix Y. pq .

[0109] In this embodiment, the first splicing matrix consists of Q first sub-matrices X pq The first concatenated matrix is ​​formed by paging together P second submatrices W. The number of columns in the first concatenated matrix is ​​equal to the number of columns in the first matrix X. pq The first submatrix is ​​constructed by concatenating the second submatrix, and the number of rows in the second concatenated matrix is ​​equal to the number of rows in the second matrix W. Since the number of columns in X×W is always equal to the number of rows in W during matrix multiplication, it is not necessary for the number of columns in the first submatrix to be equal to the number of rows in the second submatrix. This ensures that the number of columns in the first concatenated matrix is ​​equal to the number of rows in the second concatenated matrix, thus enabling the multiplication of the first and second concatenated matrices to obtain the resulting submatrix Y. pq .

[0110] In another exemplary embodiment of this invention, each node can calculate A after obtaining each pair of submatrices. a =X pq ×W pq According to P A a Calculate the resulting submatrix

[0111] Each pair of submatrices includes a first submatrix X. pq and a second submatrix W pq And X pq In this case, q equals W. pq In the case of p, for example, when p = 4, X 11 respectively and W 11 W 12 W 13 W 14 Pairing, X 13 respectively and W 31 W 32 W 33 W 34 Pairing, X 23 respectively and W 31 W 32 W 33 W 34 Pairing; only some paired submatrices are listed here, and other cases are deduced by analogy; a = 1, 2, ... P.

[0112] In this embodiment, obtaining a submatrix refers to reading the submatrix stored in this node, or receiving a submatrix from other nodes. Again, taking the node square matrix P=Q=4 shown in Figure 4 as an example, for instance, node E... 1,1 In the first cycle, a pair of submatrices X are read from this node.11 and W 11 Next, calculate A1 = X 11 ×W 11 In the second cycle, a pair of submatrices X are received. 12 and W 21 Next, calculate A2 = X 12 ×W 21 In the third cycle, a pair of submatrices X are received. 13 and W 31 Next, calculate A3 = X 13 ×W 31 In the fourth cycle, a pair of submatrices X are received. 14 and W 41 Next, calculate A4 = X 14 ×W 41 Then calculate the resulting submatrix Y. 11 = A1 + A2 + A3 + A4. The cases for other nodes follow the same logic, and will not be elaborated here.

[0113] In this embodiment, each node E is equivalent to p,q X can be divided into P periods. pq and W pq Multiplication calculations;

[0114] Node E p,q The result submatrix corresponding to this node is obtained based on the submatrix stored and received by this node, and may include:

[0115] Node E p,q In each of the P cycles, for the X transmitted in this cycle pq and W pq Perform matrix multiplication; sum the results of P periodic multiplications to obtain the result submatrix Y corresponding to this node. pq ;

[0116] One periodic transmission submatrix includes: the transmitted X pq and the W sent pq , or the X sent pq and received W pq , or the received X pq and the W sent pq , or the received X pq and received W pq .

[0117] In this embodiment, after acquiring the first and second submatrices in each cycle, the multiplication calculation for that cycle is performed. This multiplication calculation and the transmission of the next cycle can be parallelized, for example, by using a ping-pong operation. The temporary storage module is divided into two regions: the first region stores the submatrices being read and sent, as well as the submatrices being received, and the second region stores the submatrices undergoing multiplication calculation. After the transmission and calculation are completed, a switch is made. The submatrices read and received in the first region are ready for the next round of multiplication calculation, while the second region is used to store newly read and received submatrices. This saves time overall and is more efficient.

[0118] In this embodiment, the result submatrix Y of each node can be... pq The initial value is set to a zero matrix, and X is calculated after each cycle of multiplication. pq ×W pq The result of the multiplication is compared with the current result submatrix Y. pq Adding them together yields a new submatrix Y. pq ; until the result of the multiplication calculation in the Pth cycle is added to the result submatrix obtained in the (P-1)th cycle, the result submatrix Y is obtained. pq Alternatively, the results of the multiplication calculations in each cycle can be saved. After performing P cycles of multiplication calculations, the results of the P cycles of multiplication calculations can be added together to obtain the resulting submatrix Y. pq .

[0119] In this embodiment, the calculation for that cycle is performed after the transmission of one cycle, and the transmission of the next cycle is performed after the calculation is completed. This is illustrated by node E in the node matrix of P=Q=4 shown in Figure 4. 1,1 For example: After the transmission of the first cycle, the calculation of the first cycle is performed, based on the first sub-matrix X transmitted by this node to the nodes in the same row. 11 and the second submatrix W transmitted to the nodes in the same column 11 Perform multiplication calculation X 11 ×W 11 The calculation for the second cycle is performed after the transmission of the second cycle, that is, after receiving the first submatrix X. 12 Second submatrix W 21 Then, perform multiplication to calculate X. 12 ×W 21 The calculation for the third cycle is performed after the transmission of the third cycle, that is, after receiving the first submatrix X. 13 Second submatrix W 31 Then, perform multiplication to calculate X. 13 ×W 31 The calculation for the fourth cycle is performed after the transmission of the fourth cycle, that is, after receiving the first submatrix X. 14Second submatrix W 41 Then, perform multiplication to calculate X. 14 ×W 41 After each cycle of multiplication is completed, the result of the multiplication is accumulated into the result submatrix Y. pq Alternatively, after performing four cycles of multiplication, the resulting submatrix Y is calculated. 11 =X 11 ×W 11 +X 12 ×W 21 +X 13 ×W 31 +X 14 ×W 41 .

[0120] In the node matrix P=Q=4 shown in Figure 4, node E 2,3 For example: The calculation of the first cycle is performed after the transmission of the first cycle, that is, the calculation of the first cycle is performed after the node receives the first submatrix X. 21 Second submatrix W 13 Then, perform multiplication to calculate X. 21 ×W 13 The calculation for the second cycle is performed after the transmission of the second cycle, that is, after receiving the first submatrix X. 22 And send the second submatrix W 23 Then, perform multiplication to calculate X. 22 ×W 23 The calculation for the third cycle is performed after the transmission of the third cycle, that is, after sending the first submatrix X. 23 Second submatrix W 33 Then, perform multiplication to calculate X. 23 ×W 33 The calculation for the fourth cycle is performed after the transmission of the fourth cycle, that is, after receiving the first submatrix X. 24 Second submatrix W 43 Then, perform multiplication to calculate X. 24 ×W 43 After each cycle of multiplication is completed, the result of the multiplication is accumulated into the result submatrix Y. pq Alternatively, after performing four cycles of multiplication, the resulting submatrix Y is calculated. 23 =X 21 ×W 13 +X 22 ×W 23 +X 23 ×W 33 +X 24 ×W 43 .

[0121] In one exemplary implementation, each node Ep,q Store I×J first submatrices and H×K second submatrices respectively, where the number of columns in the first submatrices equals the number of rows in the second submatrices, and J×Q=H×P; each node E p,q Each corresponds to one of I×K result submatrices;

[0122] Node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:

[0123] Node E p,q The node sends multiple first submatrices and second submatrices stored in this node in multiple batches. Each time, at least one first submatrice is sent to the other Q-1 nodes in the same row as this node, and at least one second submatrice is sent to the other P-1 nodes in the same column as this node.

[0124] Node E p,q The resulting submatrix for this node, calculated based on the submatrix stored and received by this node, may include:

[0125] Node E p,q Perform matrix multiplication on the first and second submatrices transmitted each time, and accumulate the matrix multiplication results into the corresponding result submatrix in this node;

[0126] The first and second sub-matrixes in a single transmission include: the first sub-matrix to be transmitted and the second sub-matrix to be transmitted, or the first sub-matrix to be transmitted and the second sub-matrix to be received, or the first sub-matrix to be received and the second sub-matrix to be transmitted, or the first sub-matrix to be received and the second sub-matrix to be received.

[0127] In this embodiment, when a node stores multiple first sub-matrices, the multiple first sub-matrices are arranged in an array. When a node stores multiple second sub-matrices, the multiple second sub-matrices are arranged in an array. I and J are integers greater than 1, or one of I and J can be 1 (if both are 1, then the node stores only one first sub-matrice). H and K are integers greater than 1, or one of H and K can be 1 (if both are 1, then the node stores only one second sub-matrice).

[0128] In this embodiment, if the first matrix is ​​divided into R rows and S columns of first submatrices X ij The second matrix is ​​divided into S rows and T columns of second submatrices W. hkIf the first and second submatrices are evenly distributed to P×Q nodes, then we have I=R / P, J=S / Q, H=S / P, K=T / Q; if R / P, S / Q, S / P, and T / Q are not integers, they can be rounded up to become I, J, H, and K respectively.

[0129] In this embodiment, the P×Q nodes can send data multiple times in parallel, or different nodes can send data at staggered times. For example, the nodes in the first row send data first, then the nodes in the second row send data first, and after all the nodes in all rows have sent data, the nodes in the first row send data second, then the nodes in the second row send data second, and so on. This embodiment does not limit the sending order of the nodes or whether they send data in parallel, as long as all the first submatrices in each node can be sent to other nodes in the same row, and all the second submatrices can be sent to other nodes in the same column.

[0130] In this embodiment, the node can perform matrix multiplication on the paired submatrixes after each submatrix transmission is completed; the transmission of submatrixes includes four cases: sending the first submatrix and the second submatrix, receiving the first submatrix and the second submatrix, sending the first submatrix and receiving the second submatrix, and receiving the first submatrix and sending the second submatrix.

[0131] In this embodiment, the number of first submatrices and second submatrices in a node are not necessarily the same; in one example, P=2, Q=4, R=4, S=8, T=8, that is, the node array includes 2×4 nodes 10, and each node has 2×2 first submatrices X. ij and 4×2 second submatrices W hk As shown in Figure 5; each node will yield 2×2 result submatrices.

[0132] In this example, to ensure that submatrices can be paired after each transmission, each node can send one first submatrix and two second submatrices at a time. The submatrix sent in the dashed box in Figure 5 is the first submatrix sent. For example, taking node E... 1,1 For example, during the first transmission, X is sent. 11 and W 11 and W 31 Received from node E 2,1 W sent 51 and W 71 and node E 1,2 E 1,3 E 1,4 Send X 13 X 15 X 17 During the second transmission, X is sent. 12 and W 21 and W 41Received from node E 2,1 W sent 61 and W 81 and node E 1,2 E 1,3 E 1,4 Send X 14 X 16 X 18 During the third transmission, X is sent. 21 and W 11 and W 31 Received from node E 2,1 W sent 51 and W 71 and node E 1,2 E 1,3 E 1,4 Send X 23 X 25 X 27 During the fourth transmission, X is sent. 22 and W 21 and W 41 Received from node E 2,1 W sent 61 and W 81 and node E 1,2 E 1,3 E 1,4 Send X 24 X 26 X 28 In the fifth to eighth transmissions, the order of transmitting the first submatrix is ​​the same as in the first to fourth transmissions, except that the second submatrix is ​​transmitted instead of the second column (W). 11 To W 81 Change to W 12 To W 82 ).

[0133] In this example, during computation, the saved and received submatrices can be paired up each time, for example, with node E. 1,1 For example, after the first transmission, X is performed. 11 ×W 11 X 13 ×W 31 X 15 ×W 51 X 17 ×W 71 Accumulate into the resulting submatrix Y 11 After the second transmission, X is performed. 12 ×W 21 X 14 ×W 41 X 16 ×W 61 X18 ×W 81 Accumulate into the resulting submatrix Y 11 After the third transmission, X is performed. 21 ×W 11 X 23 ×W 31 X 25 ×W 51 X 27 ×W 71 Accumulate into the resulting submatrix Y 21 After the fourth transmission, X is performed. 22 ×W 21 X 24 ×W 41 X 26 ×W 61 X 28 ×W 81 Accumulate into the resulting submatrix Y 21 The multiplication calculations performed after the fifth to eighth transmissions are similar, except that W in the formula... 11 To W 81 Change to W 12 To W 82 The resulting submatrix was changed to Y. 12 and Y 22 .

[0134] As can be seen, what a node needs to do is process the entire row of the first submatrix within the array of the first submatrix (e.g., for node E). 1,1 , is X 11 To X 18 The whole line, and X 21 To X 28 The entire row), and the entire column to which the stored second submatrix belongs in the second submatrix array (e.g., for node E). 1,1 , is W 11 To W 81 The matrix multiplication between columns is divided into I×J×K = 8 transmissions and calculations.

[0135] In other embodiments, a node can send multiple first submatrices stored at once to other nodes in the same row, and send multiple second submatrices stored at once to other nodes in the same column. This can be handled as if a node has only one first submatrice and one second submatrice, which is equivalent to treating multiple first submatrices as a whole and multiple second submatrices as a whole.

[0136] In other embodiments, a node can wait until it receives the first submatrix sent by all other nodes in the same row and all other second submatrixes in the same column, then concatenate the saved and received first submatrixes according to their positions in the first matrix, and concatenate the saved and received second submatrixes according to their positions in the second matrix. Then, it can perform matrix multiplication on the concatenated two matrices. For example, in the example shown in Figure 5, after all nodes have sent their submatrixes, the saved and received first submatrixes can be concatenated according to their positions in the first matrix, and the saved and received second submatrixes can be concatenated according to their positions in the second matrix; taking node E as an example... 1,1 For example, the first submatrices (4 saved, 12 received) are concatenated into a 2×8 matrix, and the second submatrices (8 saved, 8 received) are concatenated into an 8×2 matrix. The two concatenated matrices are then multiplied to obtain node E. 1,1 The corresponding result is a 2×2 submatrix.

[0137] On the other hand, this application also provides a matrix multiplication calculation method, implemented based on the data processing architecture described in any of the above embodiments. The matrix multiplication calculation method is shown in Figure 6 and includes:

[0138] S100, each node E of the data processing architecture p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node;

[0139] S200, each of the nodes E p,q The result submatrix corresponding to this node is calculated based on the submatrix stored in this node and the received submatrix, respectively;

[0140] The submatrix includes a first submatrix and a second submatrix; the result submatrix of P×Q nodes forms the first matrix X and the result matrix Y of matrix multiplication of the second matrix W; P×Q first submatrixes form the first matrix X; and P×Q second submatrixes form the second matrix W.

[0141] In this embodiment, the matrix multiplication calculation of Y = X × W can be achieved through steps S100 and S200.

[0142] In this embodiment, P×Q nodes can perform step S100 and step S200 multiple times; a single node can perform step S100 and step S200 once or multiple times. For a single node, step S100 can be performed first, followed by step S200, or vice versa. For all nodes, step S100 can be performed first, followed by step S200; alternatively, steps S100 and S200 can be performed alternately, for example, some nodes perform step S100 first, some or all nodes perform step S200, and then another group of nodes performs step S100 again, followed by some or all nodes performing step S200.

[0143] In this embodiment, each node E p,q It will receive the first submatrix sent by the other Q-1 nodes in the same row as this node, and the second submatrix sent by the other P-1 nodes in the same column as this node.

[0144] The matrix multiplication calculation method in this embodiment divides the multiplication calculation of large matrices into multiple parallel small matrix multiplication calculations, which greatly improves the calculation efficiency of large matrix multiplication, saves calculation time, and provides a foundation for accurate matrix multiplication calculation.

[0145] In one exemplary embodiment, step S100 performed by each node may include steps S110-S120:

[0146] The nodes E in different columns p,q The first sub-matrix is ​​sent at different time intervals, and the nodes E in the same column are sent respectively. p,q The first submatrix stored at this node is sent in parallel to the other Q-1 nodes in the same row; the nodes E in different rows... p,q The second sub-matrix is ​​sent at different time intervals, and the nodes E in the same row are... p,q The second submatrix stored in this node is sent in parallel to the other P-1 nodes in the same column;

[0147] Or, each node E p,q In parallel, the first submatrix stored in this node is sent to the other Q-1 nodes in the same row, and the second submatrix stored in this node is sent to the other P-1 nodes in the same column.

[0148] In other embodiments, P×Q nodes can be sent sequentially node by node, or the sending batches can be divided without following rows and columns.

[0149] In one exemplary embodiment, step S100 performed by each node may include steps S110-S120:

[0150] S110: Broadcast the first submatrix stored in this node to the other Q-1 nodes in the same row as this node;

[0151] S120: Broadcast the second submatrix stored in this node to the other P-1 nodes in the same column as this node.

[0152] Steps S110 and S120 can be executed in any order and in parallel. Furthermore, when a node stores multiple first and second submatrices, steps S110 and S120 can be executed multiple times. Different nodes can execute steps S110 and S120 in parallel or in batches according to a predetermined order.

[0153] In this embodiment, each first submatrix broadcasts to each other only between nodes in the same row, and each second submatrix broadcasts to each other only between nodes in the same column.

[0154] In other embodiments, other transmission methods, not limited to broadcasting, can be used to send the first submatrix to other nodes in the same row and the second submatrix to other nodes in the same column.

[0155] In one exemplary embodiment, step S200 may include:

[0156] Each node E p,q After receiving all the first sub-matrices stored by Q-1 other nodes in the same row and all the second sub-matrices stored by P-1 other nodes in the same column, the first sub-matrices stored by this node and the received first sub-matrices are concatenated according to the order of the first sub-matrices in the first matrix to obtain a first concatenated matrix. The second sub-matrices stored by this node and the received second sub-matrices are concatenated according to the order of the second sub-matrices in the second matrix to obtain a second concatenated matrix. Matrix multiplication is performed on the first concatenated matrix and the second concatenated matrix to obtain the result sub-matrix corresponding to this node.

[0157] In another exemplary embodiment, the first matrix is ​​divided into R rows and S columns of first submatrices X. ij The second matrix is ​​divided into S rows and T columns of second submatrices W. hk ; i represents the first submatrix X ij The index of the row in the first matrix, j, represents the first submatrix X. ij The column number in the first matrix; h represents the second submatrix W. hk In the second matrix, the row number k represents the first submatrix W. hk The column index in the second matrix; i = 1, 2, ..., R, j = 1, 2, ..., S, h = 1, 2, ..., S, k = 1, 2, ..., T;

[0158] Step S200 may include: each of the nodes E p,q In the saved and received first and second sub-matrices, the first sub-matrix X that makes j and h equal. ij Second submatrix W hk As a pair of submatrices, perform matrix multiplication on each pair of submatrices, and sum the matrix multiplication results corresponding to the same result submatrix.

[0159] In other embodiments, after all transmissions are completed, the submatrices can be multiplied one by one and then summed to obtain the result submatrix. Alternatively, the transmitted submatrices can be concatenated each time and then matrix multiplied, with the results of each matrix multiplication summed to obtain the result submatrix.

[0160] In one exemplary embodiment, each node E p,q Each of the first submatrices X can be stored separately. pq and a second submatrix W pq When P = Q, P × Q nodes can send the first submatrix X in P cycles. pq Second submatrix W pq Step S100 may include:

[0161] P×Q nodes send the first submatrix X in P cycles. pq Second submatrix W pq Each node E p,q In the q-th period, store the first submatrix X at this node. pq Send the data to the other Q-1 nodes in the same row as this node; in the p-th cycle, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.

[0162] In this embodiment, step S100 can be considered as including P steps S130. Taking the u-th cycle as an example, step S130 includes: node E p,q＝u The first submatrix X stored in this node pq Send to the other Q-1 nodes in the same row as this node; Node E p＝u,q The second submatrix W stored in this node pq Send them to the other P-1 nodes in the same column as this node;

[0163] Where u = 1, 2, ..., P, that is, P × Q nodes execute step S130 a total of P times.

[0164] In one exemplary embodiment of this invention, step S200 may include:

[0165] Each node E p,q In each of the P cycles, for the X transmitted in this cycle pq and W pq Perform matrix multiplication calculations; accumulate the results of P periodic multiplication calculations to obtain the result submatrix corresponding to this node;

[0166] Among them, X is transmitted in one cycle. pq and W pq Includes: X sent pq and the W sent pq , or the X sent pq and received W pq , or the received X pq and the W sent pq , or the received X pq and received W pq .

[0167] In other embodiments, the batches for sending submatrices can be divided in other ways, not limited to sending by row or by column, or they can be sent node by node; the order in which different batches or nodes send submatrices is not limited;

[0168] In other embodiments, P and Q may not be equal.

[0169] In one exemplary embodiment, each node E p,q Each node E can store I×J first submatrices and H×K second submatrices respectively; p,q Each can correspond to one of I×K result submatrices;

[0170] Step S100 may include:

[0171] Each of the nodes E p,q The node sends multiple first submatrices and multiple second submatrices stored in this node in multiple batches. Each time, at least one first submatrice is sent to the other Q-1 nodes in the same row as this node, and at least one second submatrice is sent to the other P-1 nodes in the same column as this node.

[0172] Step S200 may include:

[0173] Each of the nodes E p,q Perform matrix multiplication on the first and second submatrices transmitted each time, and accumulate the matrix multiplication results into the corresponding result submatrix in this node;

[0174] The first and second sub-matrixes in a single transmission include: the first sub-matrix to be transmitted and the second sub-matrix to be transmitted, or the first sub-matrix to be transmitted and the second sub-matrix to be received, or the first sub-matrix to be received and the second sub-matrix to be transmitted, or the first sub-matrix to be received and the second sub-matrix to be received.

[0175] The following two examples illustrate this.

[0176] Example 1

[0177] Example 1 is a first matrix divided into P×Q first submatrices and a second matrix divided into P×Q second submatrices. Each node stores the information of one first submatrix and one second submatrix. In Example 1, P = Q, and the data is sent and calculated in P cycles. In each cycle, only one row of nodes sends the second submatrix and only one column of nodes sends the first submatrix. After each cycle, the submatrix multiplication calculation for that cycle is performed.

[0178] As shown in Figure 4, the node array in Example 1 is a square matrix of nodes with P = Q = 4, and each node stores a first submatrix X. pq and a second submatrix W pq This corresponds to a result submatrix Y. pq The first submatrix X is sent by 4×4 nodes in 4 cycles. pq Second submatrix W pq :

[0179] First cycle: Node E 1,1 The first submatrix X stored in this node 11 Send to E respectively 1,2 E 1,3 and E 1,4 Node E 2,1 The first submatrix X stored in this node 21 Send to E respectively 2,2 E 2,3 and E 2,4 Node E 3,1 The first submatrix X stored in this node 31 Send to E respectively 3,2 E 3,3 and E 3,4 Node E 4,1 The first submatrix X stored in this node 41 Send to E respectively 4,2 E 4,3 and E 4,4 Node E 1,1 The second submatrix W stored in this node 11 Send to E respectively 2,1 E 3,1 and E 4,1Node E 1,2 The second submatrix W stored in this node 12 Send to E respectively 2,2 E 3,2 and E 4,2 Node E 1,3 The second submatrix W stored in this node 13 Send to E respectively 2,3 E 3,3 and E 4,3 Node E 1,4 The second submatrix W stored in this node 14 Send to E respectively 2,4 E 3,4 and E 4,4 .

[0180] Second cycle: Node E 1,2 The first submatrix X stored in this node 12 Send to E respectively 1,1 E 1,3 and E 1,4 Node E 2,2 The first submatrix X stored in this node 22 Send to E respectively 2,1 E 2,3 and E 2,4 Node E 3,2 The first submatrix X stored in this node 32 Send to E respectively 3,1 E 3,3 and E 3,4 Node E 4,2 The first submatrix X stored in this node 42 Send to E respectively 4,1 E 4,3 and E 4,4 Node E 2,1 The second submatrix W stored in this node 21 Send to E respectively 1,1 E 3,1 and E 4,1 Node E 2,2 The second submatrix W stored in this node 22 Send to E respectively 1,2 E 3,2 and E 4,2 Node E 2,3 The second submatrix W stored in this node 23 Send to E respectively 1,3 E 3,3 and E 4,3 Node E 2,4 The second submatrix W stored in this node 24 Send to E respectively1,4 E 3,4 and E 4,4 .

[0181] Third cycle: Node E 1,3 The first submatrix X stored in this node 13 Send to E respectively 1,1 E 1,2 and E 1,4 Node E 2,3 The first submatrix X stored in this node 23 Send to E respectively 2,1 E 2,2 and E 2,4 Node E 3,3 The first submatrix X stored in this node 33 Send to E respectively 3,1 E 3,2 and E 3,4 Node E 4,3 The first submatrix X stored in this node 43 Send to E respectively 4,1 E 4,2 and E 4,4 Node E 3,1 The second submatrix W stored in this node 31 Send to E respectively 1,1 E 2,1 and E 4,1 Node E 3,2 The second submatrix W stored in this node 32 Send to E respectively 1,2 E 2,2 and E 4,2 Node E 3,3 The second submatrix W stored in this node 33 Send to E respectively 1,3 E 2,3 and E 4,3 Node E 3,4 The second submatrix W stored in this node 34 Send to E respectively 1,4 E 2,4 and E 4,4 .

[0182] Fourth cycle: Node E 1,4 The first submatrix X stored in this node 14 Send to E respectively 1,1 E 1,2 and E 1,3 Node E 2,4 The first submatrix X stored in this node 24 Send to E respectively 2,1 E2,2 and E 2,3 Node E 3,4 The first submatrix X stored in this node 34 Send to E respectively 3,1 E 3,2 and E 3,3 Node E 4,4 The first submatrix X stored in this node 44 Send to E respectively 4,1 E 4,2 and E 4,3 Node E 4,1 The second submatrix W stored in this node 41 Send to E respectively 1,1 E 2,1 and E 3,1 Node E 3,2 The second submatrix W stored in this node 42 Send to E respectively 1,2 E 2,2 and E 3,2 Node E 4,3 The second submatrix W stored in this node 43 Send to E respectively 1,3 E 2,3 and E 3,3 Node E 4,4 The second submatrix W stored in this node 44 Send to E respectively 1,4 E 2,4 and E 3,4 .

[0183] In one calculation method of Example 1, after the fourth cycle of transmission is completed, each node can perform steps S210-S230 respectively:

[0184] S210: Arrange the Q first submatrices X in ascending order of q. pq Perform horizontal concatenation to obtain the first concatenation matrix; for example, for node E 1,1 In other words, what we get is X in sequence. 11 X 12 X 13 X 14 The first concatenated matrix obtained by horizontal concatenation is the first row of the first matrix X; the first concatenated matrix obtained by the other nodes in the first row of the node array is also the first row of the first matrix X; similarly, each node in the second row of the node array obtains the second row of the first matrix X, each node in the third row obtains the third row of the first matrix X, and each node in the fourth row obtains the fourth row of the first matrix X.

[0185] S220: Arrange the P second submatrices W in ascending order of p. pq Perform vertical concatenation to obtain the second concatenated matrix; for example, for node E 1,1 In other words, what we get is W in sequence. 11 W 21 W 31 W 41 The second concatenated matrix obtained by vertically splicing is the first column of the second matrix W; similarly, the second concatenated matrix obtained by the other nodes in the first column of the node array is also the first column of the second matrix W; each node in the second column of the node array obtains the second column of the second matrix W, each node in the third column obtains the third column of the second matrix W, and each node in the fourth column obtains the fourth column of the second matrix W.

[0186] S230: Perform multiplication on the first concatenated matrix and the second concatenated matrix, that is, each node performs multiplication on the first concatenated matrix and the second concatenated matrix obtained by this node to obtain the result submatrix Y. pq .

[0187] The order of steps S210 and S220 is not important.

[0188] For this calculation method, the above-mentioned method of sending in four cycles can be adopted, or the method of sending in parallel by 4×4 nodes can be adopted, that is, the above four cycles are carried out in the same time period, without any order.

[0189] In Example 1, the first matrix X can be an M-row, L-column matrix, and the second matrix W can be an L-row, N-column matrix; M is an integer multiple of P, and the first submatrix X... pq The number of rows is M / P; N is an integer multiple of Q, and the second submatrix W pq The number of columns is N / Q; L is a common multiple of P and Q, and the first submatrix X pq The number of columns is L / Q, and the second submatrix W pq The number of rows is L / P; after step S210, the number of rows of the first concatenated matrix is ​​M / P, and the number of columns is Q×L / Q=L; after step S220, the number of rows of the second concatenated matrix is ​​P×L / P=L, and the number of columns is N / Q; after step S230, the resulting submatrix Y... pq The number of rows is M / P, and the number of columns is N / Q.

[0190] In Example 1, the resulting submatrix Y with P×Q nodes pqThe resulting matrix Y is constructed by concatenating the data. The number of rows in the resulting matrix Y is P×M / P=M, and the number of columns is Q×N / Q=N, which is the same as the resulting matrix Y obtained by the conventional matrix calculation method.

[0191] In Example 1, the resulting submatrix Y is calculated using a concatenation method. pq The method is applicable to node matrices of any structure, including rectangular node matrices with unequal P and Q, as well as square node matrices with equal P and Q; it is applicable to cases where submatrices are uniformly distributed among nodes, as well as cases where submatrices are not uniformly distributed; it is applicable to cases where nodes consist of only one first submatrix and one second submatrix, as well as cases where nodes consist of multiple first submatrices and multiple second submatrices.

[0192] In another calculation method of Example 1, step S200 may include P steps S240 and one step S250:

[0193] S240: Determine if a pair of submatrices has been obtained; if a pair of submatrices has been obtained, calculate A. a =X pq ×W pq ; wherein each pair of submatrices includes a first submatrix X pq Second submatrix W pq And in each pair of submatrices X pq In this case, q equals W. pq p in

[0194] S250: Based on P A's a Calculate the resulting submatrix Where a = 1, 2, ..., P.

[0195] In Example 1, each node uses only the pair of submatrices it has already obtained (read from or received from its memory child nodes) as the basis for each calculation of A. a =X pq ×W pq The condition is that steps S240 and S250 can be performed once after each transmission is completed.

[0196] Taking the u-th cycle as an example, X is performed based on the submatrix transmitted by this node in this cycle. pq ×W pq Multiplication calculations;

[0197] After each cycle or four cycles of transmission are completed, the resulting submatrix Y can be calculated based on the multiplication result. pq Calculation;

[0198] One periodic transmission submatrix includes: the transmitted X pq and the W sentpq , or the X sent pq and received W pq , or the received X pq and the W sent pq , or the received X pq and received W pq .

[0199] In Example 1, the node can initialize the resulting submatrix Y. pq Set as a zero matrix, meaning that X is calculated after each cycle of multiplication. pq ×W pq The result of the multiplication is compared with the current result submatrix Y. pq The addition continues until the result of the multiplication in the Pth cycle is added to the result submatrix obtained in the (P-1)th cycle, to obtain the result submatrix Y. pq Alternatively, after a node completes P cycles of multiplication, the results of the P cycles of multiplication are summed to obtain the resulting submatrix Y. pq .

[0200] In Example 1, matrix multiplication can be performed after each cycle of transmission; that is, each node performs the calculation over four cycles, including:

[0201] The calculation of the first cycle: This is performed after the first cycle is sent, at node E. 1,1 Calculate X 11 ×W 11 Node E 1,2 Calculate X 11 ×W 12 Node E 1,3 Calculate X 11 ×W 13 Node E 1,4 Calculate X 11 ×W 14 Node E 2,1 Calculate X 21 ×W 11 Node E 2,2 Calculate X 21 ×W 12 Node E 2,3 Calculate X 21 ×W 13 Node E 2,4 Calculate X 21 ×W 14 Node E 3,1 Calculate X 31 ×W 11 Node E 3,2 Calculate X 31 ×W 12 Node E3,3 Calculate X 31 ×W 13 Node E 3,4 Calculate X 31 ×W 14 Node E 4,1 Calculate X 41 ×W 11 Node E 4,2 Calculate X 41 ×W 12 Node E 4,3 Calculate X 41 ×W 13 Node E 4,4 Calculate X 41 ×W 14 .

[0202] The calculation of the second cycle: This is performed after the second cycle is sent, at node E. 1,1 Calculate X 12 ×W 21 Node E 1,2 Calculate X 12 ×W 22 Node E 1,3 Calculate X 12 ×W 23 Node E 1,4 Calculate X 12 ×W 24 Node E 2,1 Calculate X 22 ×W 21 Node E 2,2 Calculate X 22 ×W 22 Node E 2,3 Calculate X 22 ×W 23 Node E 2,4 Calculate X 22 ×W 24 Node E 3,1 Calculate X 32 ×W 21 Node E 3,2 Calculate X 32 ×W 22 Node E 3,3 Calculate X 32 ×W 23 Node E 3,4 Calculate X 32 ×W 24 Node E 4,1 Calculate X 42 ×W 21 Node E 4,2 Calculate X 42 ×W 22Node E 4,3 Calculate X 42 ×W 23 Node E 4,4 Calculate X 42 ×W 24 .

[0203] The calculation of the third cycle: This is performed after the third cycle is sent, at node E. 1,1 Calculate X 13 ×W 31 Node E 1,2 Calculate X 13 ×W 32 Node E 1,3 Calculate X 13 ×W 33 Node E 1,4 Calculate X 13 ×W 34 Node E 2,1 Calculate X 23 ×W 31 Node E 2,2 Calculate X 23 ×W 32 Node E 2,3 Calculate X 23 ×W 33 Node E 2,4 Calculate X 23 ×W 34 Node E 3,1 Calculate X 33 ×W 31 Node E 3,2 Calculate X 33 ×W 32 Node E 3,3 Calculate X 33 ×W 33 Node E 3,4 Calculate X 33 ×W 34 Node E 4,1 Calculate X 43 ×W 31 Node E 4,2 Calculate X 43 ×W 32 Node E 4,3 Calculate X 43 ×W 33 Node E 4,4 Calculate X 43 ×W 34 .

[0204] The calculation of the fourth cycle: This is performed after the fourth cycle is sent, at node E. 1,1 Calculate X 14 ×W41 Node E 1,2 Calculate X 14 ×W 42 Node E 1,3 Calculate X 14 ×W 43 Node E 1,4 Calculate X 14 ×W 44 Node E 2,1 Calculate X 24 ×W 41 Node E 2,2 Calculate X 24 ×W 42 Node E 2,3 Calculate X 24 ×W 43 Node E 2,4 Calculate X 24 ×W 44 Node E 3,1 Calculate X 34 ×W 41 Node E 3,2 Calculate X 34 ×W 42 Node E 3,3 Calculate X 34 ×W 43 Node E 3,4 Calculate X 34 ×W 44 Node E 4,1 Calculate X 44 ×W 41 Node E 4,2 Calculate X 44 ×W 42 Node E 4,3 Calculate X 44 ×W 43 Node E 4,4 Calculate X 44 ×W 44 .

[0205] Each node obtains its corresponding result submatrix by either accumulating the multiplication results into the result submatrix after each cycle of calculation, or by adding the multiplication results of all cycles after four cycles of calculation.

[0206] Example 2

[0207] Example 2 shows the case where each node stores multiple first submatrices and multiple second submatrices. In Example 2, P = Q. All nodes send data each time, but each time only the first submatrices and the second submatrices stored by the node are sent. After each transmission, the transmitted submatrices are multiplied.

[0208] Taking the 4×4 node matrix shown in Figure 7 as an example, the first matrix X is divided into 8×8 first submatrices, and the second matrix W is divided into 8×8 second submatrices; each node's memory child node 40 stores 2×2 first submatrices and 2×2 second submatrices; one matrix multiplication is completed in the following nine steps:

[0209] Step 1, Node E p,q Send the saved X (2p-1)(2q-1) Send the saved W to other nodes in the same industry. (2p-1)(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 11 Send W to other nodes in the same industry 11 Give it to other nodes in the same column, such as E. 3,2 Send X 53 Send W to other nodes in the same industry. 53 Send to other nodes in the same column. That is, each node sends the first row and first column of the first 2×2 first submatrix and the first row and first column of the second 2×2 submatrix stored in this node.

[0210] The submatrix to be sent and the submatrix to be received are stored in the temporary storage module 31 of the node. After the first step of sending, the submatrix stored in the temporary storage module 31 of each node is shown in Figure 7.

[0211] In the second step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the first step (i.e., the sub-matrices stored in the temporary storage module 31 in Figure 7), and accumulates them into the resulting sub-matrix Y. (2p-1)(2q-1) In the middle; for example, E 1,1 Calculate X 11 ×W 11 X 13 ×W 31 X 15 ×W 51 X 17 ×W 71 The result submatrix Y is then added to the temporary storage module 31. 11 middle.

[0212] As can be seen, the first submatrix used for calculations of nodes in the same row is the same, and the second submatrix used for calculations of nodes in the same column is the same.

[0213] During the calculation, node E p,q Send the saved X (2p-1)2q Send the saved W to other nodes in the same industry. 2p(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 12Send W to other nodes in the same industry. 21 Give it to other nodes in the same column; for example, E. 2,4 Send X 38 Send W to other nodes in the same industry. 47 Send to other nodes in the same column. That is, each node sends the first row and second column of the first 2×2 first submatrix and the second row and first column of the second 2×2 second submatrix stored in this node.

[0214] In the third step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the second step, and accumulates the results into the resulting sub-matrix Y. (2p-1)(2q-1) In the middle; for example, E 1,1 Calculate X 12 ×W 21 X 14 ×W 41 X 16 ×W 61 X 18 ×W 81 The result submatrix Y is then added to the temporary storage module 31. 11 In the middle; after the accumulation is complete, the resulting submatrix Y is... 11 Saved to memory child node 40. It can be seen that after the calculations in steps two and three, E... 1,1 The resulting submatrix Y 11 It is equivalent to the result of multiplying the first row of the first matrix X and the first column of the second matrix W.

[0215] During the calculation, node E p,q Send the saved X (2p-1)(2q-1) Send the saved W to other nodes in the same industry. (2p-1)2q Give it to other nodes in the same column; for example, E. 1,1 Send X 11 Send W to other nodes in the same industry. 12 Give it to other nodes in the same column; for example, E. 2,3 Send X 35 Send W to other nodes in the same industry. 36 Send to other nodes in the same column. That is, each node sends the first row and first column of the 2×2 first submatrix and the first row and second column of the 2×2 second submatrix stored in this node.

[0216] In the fourth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the third step, and accumulates the results into the resulting sub-matrix Y. (2p-1)2q In the middle; for example, E 1,1 Calculate X 11 ×W 12 X 13 ×W 32 X15 ×W 52 X 17 ×W 72 The result submatrix Y is then added to the temporary storage module 31. 12 middle.

[0217] During the calculation, node E p,q Send the saved X (2p-1)2q Send the saved W to other nodes in the same industry. 2p2q Give it to other nodes in the same column; for example, E. 1,1 Send X 12 Send W to other nodes in the same industry. 22 Give it to other nodes in the same column; for example, E. 2,3 Send X 36 Send W to other nodes in the same industry. 46 Send to other nodes in the same column. That is, each node sends the first row and second column of the first 2×2 first submatrix and the second row and second column of the second 2×2 second submatrix stored by the node.

[0218] In the fifth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the fourth step, and accumulates the results into the resulting sub-matrix Y. (2p-1)2q In the middle; for example, E 1,1 Calculate X 12 ×W 22 X 14 ×W 42 X 16 ×W 62 X 18 ×W 82 The result submatrix Y is then added to the temporary storage module 31. 12 In the middle; after the accumulation is complete, the resulting submatrix Y is... 12 Saved to memory child node 40. It can be seen that after the calculations in steps four and five, E... 1,1 The resulting submatrix Y 12 It is equivalent to the result of multiplying the first row of the first matrix X and the second column of the second matrix W.

[0219] During the calculation, node E p,q Send the saved X 2p(2q-1) Send the saved W to other nodes in the same industry. (2p-1)(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 21 Send W to other nodes in the same industry. 11 Give it to other nodes in the same column; for example, E. 4,1 Send X 81 Send W to other nodes in the same industry. 71Send to other nodes in the same column. That is, each node sends the second row and first column of the 2×2 first submatrix stored by the node, and the first row and first column of the 2×2 second submatrix.

[0220] In the sixth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the fifth step, and accumulates the results into the resulting sub-matrix Y. 2p(2q-1) In the middle; for example, E 1,1 Calculate X 21 ×W 11 X 23 ×W 31 X 25 ×W 51 X 27 ×W 71 The result submatrix Y is then added to the temporary storage module 31. 21 middle.

[0221] During the calculation, node E p,q Send the saved X 2p2q Send the saved W to other nodes in the same industry. 2p(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 22 Send W to other nodes in the same industry. 21 Give it to other nodes in the same column; for example, E. 3,3 Send X 66 Send W to other nodes in the same industry. 65 Send to other nodes in the same column. That is, each node sends the second row and second column of the 2×2 first submatrix stored in its own memory, and the second row and first column of the 2×2 second submatrix.

[0222] In the seventh step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the sixth step, and accumulates the results into the sub-matrix Y. 2p(2q-1) In the middle; for example, E 1,1 Calculate X 22 ×W 21 X 24 ×W 41 X 26 ×W 61 X 28 ×W 81 The result submatrix Y is then added to the temporary storage module 31. 21 In the middle; after the accumulation is complete, the resulting submatrix Y is... 21 Saved to memory child node 40. It can be seen that after the calculations in steps six and seven, E... 1,1 The resulting submatrix Y 21It is equivalent to the result of multiplying the second row of the first sub-matrix array divided by the first matrix X and the first column of the second sub-matrix array divided by the second matrix W.

[0223] During the calculation, node E p,q Send the saved X 2p(2q-1) Send the saved W to other nodes in the same industry. (2p-1)2q Give it to other nodes in the same column; for example, E. 1,1 Send X 21 Send W to other nodes in the same industry. 12 To other nodes in the same column, each node sends the second row and first column of the 2×2 first submatrix and the first row and second column of the 2×2 second submatrix stored in its own data.

[0224] In the eighth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the seventh step, and accumulates the results into the sub-matrix Y. 2p2q In the middle; for example, E 1,1 Calculate X 21 ×W 12 X 23 ×W 32 X 25 ×W 52 X 27 ×W 72 The result submatrix Y is then added to the temporary storage module 31. 22 middle.

[0225] During the calculation, node E p,q Send the saved X 2p2q Send the saved W to other nodes in the same industry. 2p2q Give it to other nodes in the same column; for example, E. 1,1 Send X 22 Send W to other nodes in the same industry. 22 To other nodes in the same column, each node sends the second row and second column of the 2×2 first submatrix and the second row and second column of the 2×2 second submatrix stored in its own memory.

[0226] In the ninth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the eighth step, and accumulates the results into the sub-matrix Y. 2p2q In the middle; for example, E 1,1 Calculate X 22 ×W 22 X 24 ×W 42 X 26 ×W 62 X 28 ×W 82The result submatrix Y is then added to the temporary storage module 31. 22 In the middle; after the accumulation is complete, the resulting submatrix Y is... 22 Saved to memory child node 40. It can be seen that after the calculations in steps eight and nine, E... 1,1 The resulting submatrix Y 22 It is equivalent to the result of multiplying the second row of the first matrix X and the second column of the second matrix W.

[0227] After the above nine steps, the matrix multiplication of the first matrix X and the second matrix W can be completed. Each node yields a 2×2 result submatrix (which can be viewed as a result subarray). These result subarrays are concatenated according to the position of each node in the node array to obtain the matrix multiplication result matrix Y of the first matrix X and the second matrix W. That is, the result matrix Y can be viewed as an array composed of 8×8 result submatrices or 4×4 result subarrays, with the first row being Y. 11 Y 12 Y 13 Y 14 Y 15 Y 16 Y 17 Y 18 The last line is Y 81 Y 82 Y 83 Y 84 Y 85 Y 86 Y 87 Y 88 The same logic applies to other rows.

[0228] In Example 2, each node can read and send the submatrix required for the next calculation in parallel while performing multiplication. All nodes perform sending and matrix multiplication in parallel, which can greatly improve the efficiency of matrix multiplication.

[0229] In Example 2, the sending process from step one to step eight is not limited to the above order and can be arbitrarily rearranged. Correspondingly, each step involves calculating the multiplication of the first and second submatrices transmitted in the previous step (including sending and receiving), and accumulating this multiplication into the corresponding result submatrix. For example, if the transmitted submatrix is ​​the first submatrix in row r and column s of each node, and the second submatrix in row s and column t of each node, then it is accumulated into the result submatrix in row r and column t of that node. During each multiplication calculation, nodes in the same row use the same first submatrix, and nodes in the same column use the same second submatrix.

[0230] Furthermore, embodiments of this application also provide a neural network computation method, the method comprising:

[0231] Each layer in the neural network model executes steps S300-S500, as shown in Figure 8:

[0232] S300: Using the input matrix of the current layer as the first matrix X and the weight matrix of the current layer as the second matrix W, perform matrix multiplication Y = X × W according to the matrix multiplication calculation method described in the above embodiment;

[0233] S400: Calculate the result matrix Y for the current layer based on Y. * ;

[0234] S500: Y * The input is fed into the next layer of the neural network model as the first matrix X of the next layer, or Y is used as the input. * As the result of neural network calculations.

[0235] In this embodiment, the neural network model can be a multi-layer deep learning model, such as the Transformer model, Linear-Time Sequence Modeling with Selective State Space, TTT (Test-Time Training), LSTM (Long Short-Term Memory), VGGNet (Deep Convolutional Neural Network), or any other large model structure or deep learning model. The above deep learning models are all illustrative examples and are not intended to limit this application. They will not be described in detail here.

[0236] In this embodiment, each layer of the neural network model can be computed by the data processing architecture described in the above embodiment, and each node of the node array jointly performs multi-layer computations of the neural network model.

[0237] In this embodiment, before performing the neural network calculation method, the weight matrix of each layer of the neural network model can be divided into P×Q second sub-matrices according to the structure of the node array (i.e., a total of P rows and Q columns), and the second sub-matrix of each layer is stored in the memory sub-node of the node array; when performing the calculation of a certain layer of the neural network, the calculation sub-node can read the second sub-matrix divided from the weight matrix of the corresponding layer from the memory sub-node and send it.

[0238] In this embodiment, the input matrix of the neural network model is used as the first matrix X of the first layer of the neural network model. Before performing the neural network calculation method, the first matrix X can be divided into P×Q first sub-matrices according to the structure of the node array and stored in the memory sub-nodes or temporary storage module of the node array.

[0239] In this embodiment, during step S400, different neural network models have different result matrices Y for calculating the current layer. * Methods, for example, directly setting Y as the result matrix Y of the current layer. * Alternatively, Y can be added to a bias matrix to obtain the result matrix Y of the current layer. * The neural network model yields a result matrix Y at each layer. * It is formed by concatenating the result submatrices from P×Q nodes.

[0240] In this embodiment, taking the F-layer neural network model as an example, the component Y calculated by the first layer... * The resulting submatrices are stored in P×Q memory child nodes or P×Q temporary storage modules of the node array, serving as the first submatrix of the second layer; the composition Y calculated in the second layer... * The resulting submatrices are stored in P×Q memory child nodes or P×Q temporary storage modules of the node array, serving as the first submatrix of the 3rd layer...; the composition Y calculated in the Fth layer... * The P×Q submatrices are stored in the P×Q memory subnodes or P×Q temporary storage modules of the node array, respectively, as the output matrix of the neural network model.

[0241] In the neural network calculation method of this embodiment, in the neural network calculation of the current layer, each node E p,q The calculated result matrix Y * It can be used as the input matrix for the next layer of the neural network, and the resulting matrix Y * By storing data within this node without requiring input / output operations, the computational efficiency of deep neural network computation methods is improved. Furthermore, by dividing the large matrix multiplication computation of each layer of the neural network into parallel small matrix multiplication computations, the computational efficiency of deep neural networks is further improved, providing an efficient solution for large-scale deep learning computations.

[0242] Furthermore, embodiments of this application also provide a chip, including: the data processing architecture provided in any of the above embodiments.

[0243] In one exemplary embodiment, the chip includes:

[0244] The storage die includes P×Q memory sub-nodes; the P×Q memory sub-nodes respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied;

[0245] The logic die includes P×Q computational sub-nodes, each corresponding to one of the P×Q memory sub-nodes. Each computational sub-node is configured to read a first sub-matrix from its corresponding memory sub-node and send it to the other Q-1 computational sub-nodes in the same row; read a second sub-matrix from its corresponding memory sub-node and send it to the other P-1 computational sub-nodes in the same column; and calculate a corresponding result sub-matrix based on the sub-matrix stored in the corresponding memory sub-node and the received sub-matrix, and save it in the corresponding memory sub-node. The sub-matrix includes a first sub-matrix and a second sub-matrix; the result sub-matrixes stored in the P×Q memory sub-nodes form the result matrix.

[0246] The storage die and the logic die are stacked and bonded together.

[0247] In this embodiment, the result submatrices stored in the P×Q memory child nodes can be directly concatenated according to the position of the memory child nodes in the storage die 60 (i.e., the row and column number) to obtain the final result matrix Y of the matrix multiplication of the first matrix X and the second matrix W; for example, assuming that each memory child node stores a result submatrix Y. pq Then the result submatrix Y in P×Q memory child nodes pq We can concatenate them to obtain Y[M][N], that is: Y[M][N] can be regarded as a result submatrix array, where the first row is Y 11 Y 12 , ..., Y 1Q The second line is Y. 21 Y 22 , ..., Y 2Q The last line is Y. P1 Y P2 , ..., Y PQ And so on. When each memory child node stores multiple result submatrices, the multiple result submatrices in the node form a result subarray. The result subarrays stored in multiple memory child nodes are concatenated according to the positions of the memory child nodes in the storage array to obtain Y[M][N].

[0248] In one exemplary embodiment of this invention, as shown in Figures 9 and 10, the memory sub-nodes in the storage die 60 and the computing sub-nodes in the logic die 50 can be connected one-to-one in a vertical connection manner. This reduces the size of the data processing architecture, optimizes space, reduces wiring complexity, reduces electromagnetic interference between different nodes, facilitates maintenance and expansion upgrades, and makes it easy to achieve standardized design.

[0249] In other implementations, memory sub-nodes and compute sub-nodes can be located on the same plane and connected by circuitry; storage dies and logic dies can be physically integrated into a single die, for example, using ReRaam technology to directly integrate P×Q memory sub-nodes into P×Q compute sub-nodes in a one-to-one correspondence.

[0250] It will be understood by those skilled in the art that all or some of the steps, systems, or apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

Claims

1. A data processing architecture, comprising: A node array with P rows and Q columns; The P×Q nodes in the node array respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied. The node is configured to send the first submatrix stored by the node to the other Q-1 nodes in the same row as the node, send the second submatrix stored by the node to the other P-1 nodes in the same column as the node, and calculate the result submatrix corresponding to the node based on the submatrix stored and received by the node; wherein, the submatrix includes the first submatrix and the second submatrix; the result submatrixes of the P×Q nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W.

2. The data processing architecture as described in claim 1, wherein, Each of the nodes includes: a computing sub-node and a storage module; The storage module is configured to store the first submatrix, the second submatrix, and the result submatrix; The computational sub-nodes include: The processing module is configured to read the first submatrix and the second submatrix from the storage module; calculate the result submatrix corresponding to the current node based on the submatrix stored and received at the current node, and save it to the storage module; The row data communication module is configured to send the first sub-matrix read by the processing module to the other Q-1 nodes in the same row as this node, and to receive the first sub-matrix from the other Q-1 nodes in the same row. The column data communication module is configured to send the second sub-matrix read by the processing module to the other P-1 nodes in the same column as this node, and to receive the second sub-matrix from the other P-1 nodes in the same column.

3. The data processing architecture as described in claim 2, wherein, The storage module includes: a temporary storage module disposed in the computation sub-node, configured to store the first sub-matrix, the second sub-matrix, and the result sub-matrix; Alternatively, it may include: a temporary storage module set in the computing sub-node, and a memory sub-node connected to the computing sub-node; The memory sub-node is configured to store the first sub-matrix, the second sub-matrix, and the result sub-matrix; The temporary storage module is configured to temporarily store the first sub-matrix and the second sub-matrix read by the processing module from the memory sub-node, as well as the result sub-matrix calculated by the processing module; The processing module is further configured to save the result submatrix temporarily stored in the temporary storage module to the memory subnode.

4. The data processing architecture as described in claim 1, wherein: The nodes in different columns send the first sub-matrix at different time periods, and the nodes in the same column send the first sub-matrix in parallel; the nodes in different rows send the second sub-matrix at different time periods, and the nodes in the same row send the second sub-matrix in parallel. Alternatively, the P×Q nodes may transmit the first submatrix and the second submatrix in parallel.

5. The data processing architecture as described in any one of claims 1 to 4, wherein: The node obtains its corresponding result submatrix based on the submatrix stored and received by the node, including: After receiving all the first sub-matrices stored by Q-1 other nodes in the same row and all the second sub-matrices stored by P-1 other nodes in the same column, the node concatenates the first sub-matrices stored by itself and the received first sub-matrices according to the order of the first sub-matrices in the first matrix to obtain a first concatenated matrix. Then, it concatenates the second sub-matrices stored by itself and the received second sub-matrices according to the order of the second sub-matrices in the second matrix to obtain a second concatenated matrix. Finally, it performs matrix multiplication on the first concatenated matrix and the second concatenated matrix to obtain the result sub-matrice corresponding to the node.

6. The data processing architecture as described in any one of claims 1 to 4, wherein: The first matrix is ​​divided into R rows and S columns of first submatrices X. ij The second matrix is ​​divided into S rows and T columns of second submatrices W. hk ; i represents the first submatrix X ij The index of the row in the first matrix, j, represents the first submatrix X. ij The column number in the first matrix; h represents the second submatrix W. hk The row number k in the second matrix represents the second submatrix W. hk The column index in the second matrix; i = 1, 2, ..., R, j = 1, 2, ..., S, h = 1, 2, ..., S, k = 1, 2, ..., T; The node obtains its corresponding result submatrix based on the submatrix stored and received by the node, including: The node, in the stored and received sub-matrices, sets a first sub-matrix X in which j and h are equal. ij and a second submatrix W hk As a pair of submatrices; Perform matrix multiplication on each pair of submatrices obtained, and sum the matrix multiplication results corresponding to the same result submatrix.

7. The data processing architecture as described in claim 1, wherein: Each node E p,q Store a first submatrix X respectively pq and a second submatrix W pq Where p represents the p-th row in the node array, q represents the q-th column in the node array, p = 1, 2, ..., P, q = 1, 2, ..., Q; When P = Q, the P × Q nodes E p,q The first sub-matrix X is sent in P cycles. pq Second submatrix W pq ; The node E p,q The first submatrix X stored in this node pq Send the second submatrix W stored in this node to the other Q-1 nodes in the same row as this node. pq Send to the other P-1 nodes in the same column as this node, including: The node E p,q In the q-th period, store the first submatrix X at this node. pq Send the data to the other Q-1 nodes in the same row as this node; in the p-th cycle, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.

8. The data processing architecture as described in claim 7, wherein: The node calculates the corresponding result submatrix based on the submatrix stored and received by the node, including: For each of the P cycles, the node transmits X in that cycle. pq and W pq Perform matrix multiplication calculations; accumulate the results of P periodic multiplication calculations to obtain the result submatrix corresponding to this node; Among them, X is transmitted in one cycle. pq and W pq Includes: X sent pq and the W sent pq , or the X sent pq and received W pq , or the received X pq and the W sent pq , or the received X pq and received W pq .

9. The data processing architecture as described in any one of claims 1 to 4, wherein, Each node stores I×J first submatrices and H×K second submatrices, where the number of columns in the first submatrices is equal to the number of rows in the second submatrices, and J×Q=H×P; each node corresponds to I×K result submatrices. The node sends the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and sends the second submatrix stored in this node to the other P-1 nodes in the same column as this node, including: The node sends the multiple first sub-matrixes and second sub-matrixes stored in the node in multiple batches. Each time, at least one first sub-matrix is ​​sent to the other Q-1 nodes in the same row as the node, and at least one second sub-matrix is ​​sent to the other P-1 nodes in the same column as the node. The submatrix corresponding to this node is calculated based on the submatrix stored in this node and the submatrix received, including: The node performs matrix multiplication on the first and second sub-matrices transmitted each time, and accumulates the matrix multiplication results into the corresponding result sub-matrix in the node. The first and second sub-matrixes in a single transmission include: the first sub-matrix to be transmitted and the second sub-matrix to be transmitted, or the first sub-matrix to be transmitted and the second sub-matrix to be received, or the first sub-matrix to be received and the second sub-matrix to be transmitted, or the first sub-matrix to be received and the second sub-matrix to be received.

10. A matrix multiplication calculation method, said method being implemented based on the data processing architecture as described in any one of claims 1-9, comprising: Each node in the data processing architecture sends the first sub-matrix stored in this node to the other Q-1 nodes in the same row as this node, and sends the second sub-matrix stored in this node to the other P-1 nodes in the same column as this node; Each node calculates its corresponding result submatrix based on the submatrix stored in and received by the node. The submatrix includes a first submatrix and a second submatrix; the result submatrixes of P×Q nodes form the first matrix X and the result matrix Y of matrix multiplication of the second matrix W; the first submatrix stored by P×Q nodes forms the first matrix X; and the second submatrix stored by P×Q nodes forms the second matrix W.

11. A neural network computation method, the method comprising: Each layer in the neural network model performs the following steps: Using the input matrix of the current layer as the first matrix X and the weight matrix of the current layer as the second matrix W, perform matrix multiplication Y = X × W according to the matrix multiplication calculation method described in claim 10. Calculate the result matrix Y of the current layer based on Y. * ; Y * The input is fed into the next layer of the neural network model as the first matrix X of the next layer, or Y is used as the input. * As the result of neural network calculations.

12. A chip, comprising: The data processing architecture as described in any one of claims 1 to 9.

13. The chip as claimed in claim 12, wherein, The chip includes: The storage die includes P×Q memory sub-nodes; the P×Q memory sub-nodes respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied; The logic die includes P×Q computational sub-nodes, each corresponding one-to-one with one of the P×Q memory sub-nodes. Each computational sub-node is configured to read a first sub-matrix from its corresponding memory sub-node and send it to the other Q-1 computational sub-nodes in the same row; read a second sub-matrix from its corresponding memory sub-node and send it to the other P-1 computational sub-nodes in the same column; and calculate the corresponding result sub-matrix based on the sub-matrix stored in the corresponding memory sub-node and the received sub-matrix, and save it in the corresponding memory sub-node. The sub-matrix includes a first sub-matrix and a second sub-matrix. The result sub-matrixes stored in the P×Q memory sub-nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W. The storage die and the logic die are stacked and bonded together.

Images