Data processing architecture, chip, matrix multiplication and neural network calculation method
By using a distributed two-dimensional node array architecture, large matrices are divided into smaller matrices and computed in parallel on multiple nodes, solving the efficiency and cost problems of large-scale matrix multiplication and achieving efficient parallel matrix multiplication.
Patent Information
- Application Number
- CN202480002739.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-25
- Publication Date
- 2025-12-16
AI Technical Summary
In large-scale deep learning computations, the speed and efficiency of matrix multiplication are critical issues, especially given the enormous computational demands of large-scale matrices. Existing technologies struggle to achieve low-cost and fast matrix multiplication.
By adopting a distributed two-dimensional node array architecture, a large matrix is divided into multiple smaller sub-matrices, which are stored and computed on multiple nodes. Matrix multiplication is performed through communication between the same row and column, and parallel computation and data transmission are carried out between nodes, which reduces computation time and cost.
It enables parallel computation of large-scale matrix multiplication, reduces hardware costs, improves computational efficiency, and is easy to expand and maintain.
Smart Images

Figure CN121153043A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of data processing, and in particular to a data processing architecture, a chip, a matrix multiplication method and a neural network calculation method. BACKGROUND
[0002] Deep learning is a special form of neural network, which learns the complex representation of data by using a multi-layer neural network structure. Deep learning is widely used in large models, such as image recognition and generation, natural speech processing, speech recognition and synthesis, video generation, robot dialogue, intelligent driving, etc. It can be seen that deep learning has broad application prospects in the future information transmission field. However, it involves a large number of matrix multiplication calculations and large matrix dimensions. In the face of large-scale matrix multiplication calculation problems, improving the matrix multiplication calculation processing speed has become a technical problem to be solved. SUMMARY
[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0004] The embodiment of the present application provides a data processing architecture, comprising: a node array of P rows and Q columns; P×Q nodes in the node array respectively store a first sub-matrix obtained by dividing a first matrix X to be subjected to matrix multiplication, and a second sub-matrix obtained by dividing a second matrix W to be subjected to matrix multiplication.
[0005] The node is configured to send the first sub-matrix stored in the node to other Q-1 nodes in the same row as the node, respectively, send the second sub-matrix stored in the node to other P-1 nodes in the same column as the node, respectively, and calculate a result sub-matrix corresponding to the node according to the sub-matrix stored in the node and received; wherein the sub-matrix comprises the first sub-matrix and the second sub-matrix; and the result sub-matrices of the P×Q nodes constitute a result matrix Y of the matrix multiplication of the first matrix X and the second matrix W.
[0006] In another aspect, the embodiment of the present application provides a matrix multiplication calculation method, which is implemented based on the above-mentioned data processing architecture, comprising:
[0007] Each node of the data processing architecture sends the first sub-matrix stored in the node to other Q-1 nodes in the same row as the node, respectively, and sends the second sub-matrix stored in the node to other P-1 nodes in the same column as the node, respectively.
[0008] Each node calculates a result sub-matrix corresponding to the node according to the sub-matrix stored in the node and received, respectively.
[0009] The sub-matrices include the first sub-matrices and the second sub-matrices; a result matrix Y of matrix multiplication of a first matrix X and a second matrix W is composed of P*Q nodes, the first matrix X is composed of the first sub-matrices stored in the P*Q nodes, and the second matrix W is composed of the second sub-matrices stored in the P*Q nodes.
[0010] In another aspect, the embodiments of the present application provide a neural network calculation method, each layer in a neural network model performs the following steps:
[0011] Taking the input matrix of the current layer as the first matrix X and taking the weight matrix of the current layer as the second matrix W, the matrix multiplication Y=X*W is calculated according to the above matrix multiplication calculation method;
[0012] According to Y, the result matrix Y* of the current layer is calculated;
[0013] Y* is input to the next layer of the neural network model as the first matrix X of the next layer, or Y* is taken as the calculation result of the neural network.
[0014] In another aspect, the embodiments of the present application provide a chip, comprising: the above data processing architecture.
[0015] Other aspects can be apparent from consideration of the drawings and detailed description.
[0016] SUMMARY
[0017] The accompanying drawings are included to provide a further understanding of the present application, and constitute a part of the specification, illustrate the embodiments of the present application, and are used to explain the technical scheme of the present application, and do not constitute a limitation on the technical scheme of the present application.
[0018] Figure 1 It is a schematic diagram of the data processing architecture of the embodiments of the present application;
[0019] Figure 2 It is a schematic diagram of the layout of the data processing architecture in an exemplary embodiment;
[0020] Figure 3 It is a schematic diagram of the structure of the node in an exemplary embodiment;
[0021] Figure 4 It is a schematic diagram of a 4*4 node square in an example;
[0022] Figure 5 It is a schematic diagram of a 2*4 node array storing sub-matrices in an example;
[0023] Figure 6 It is a flowchart of the matrix multiplication calculation method of the embodiments of the present application;
[0024] Figure 7 A schematic diagram of a 4x4 square of nodes for one example;
[0025] Figure 8 A flowchart of a neural network computing method for embodiments of the application;
[0026] Figure 9 A schematic diagram of a chip in an example embodiment;
[0027] Figure 10 A schematic diagram of a cross-section of a chip in an example embodiment.
[0028] Detailed description
[0029] The present disclosure describes a number of embodiments, but the description is exemplary rather than limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the embodiments described in the present disclosure. Although a number of possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature of any embodiment can be used with any other feature or element from any other embodiment or implementation, or can replace any other feature or element in any other embodiment, unless specifically restricted, and the scope of the disclosure is not limited to the specific embodiments described herein.
[0030] The terms "first" and "second" and the like in the description of the present disclosure and in the drawings are used to distinguish different objects, or to distinguish different processing of the same object, and are not used to describe a specific order of the objects. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.
[0031] Generally speaking, in deep learning computing of large models, matrix multiplication calculation Y[M][N] = X[M][L] x W[L][N] is often performed, and matrix addition calculation is also performed after matrix multiplication, and the total amount of calculation is 2MLN multiplication and addition; since the computing task in a large model is usually complex, the dimensions L and N of the weight matrix W (full name weight matrix) are generally in the order of thousands to tens of thousands, and the value of M varies according to the difference of application scenarios and computing stages, and is generally between tens of thousands to hundreds of thousands, and the total amount of calculation is very large, so low-cost and fast matrix multiplication calculation has become an inevitable requirement for the development of large models.
[0032] Embodiments of the present application propose a data processing architecture 20, such as Figure 1As shown, comprising: a node array of P rows and Q columns; P×Q nodes 10 in the node array respectively store a first sub-matrix divided from a first matrix X to be subjected to matrix multiplication, and a second sub-matrix divided from a second matrix W to be subjected to matrix multiplication;
[0033] The node 10 is configured to send the first sub-matrix stored in the node to other Q-1 nodes in the same row as the node respectively, send the second sub-matrix stored in the node to other P-1 nodes in the same column as the node respectively, and calculate a result sub-matrix corresponding to the node according to the sub-matrices stored in the node and received; wherein the sub-matrices include the first sub-matrices and the second sub-matrices; and the result sub-matrices of the P×Q nodes constitute a result matrix Y of the matrix multiplication of the first matrix X and the second matrix W.
[0034] Herein, the node 10 is denoted as E p,q , and the subscript is used to indicate the position of the node in the node array; p represents the pth row in the node array, and q represents the qth column in the node array, p=1, 2, …, P, q=1, 2, …, Q.
[0035] In this embodiment, the data processing architecture 20 is a distributed two-dimensional node array architecture, and the node E p,q In the node E p,q , p represents the horizontal coordinate of the node in the node array, i.e., p represents that the node is located in the pth row of the node array; and q represents the vertical coordinate of the node in the node array, i.e., q represents that the node is located in the qth column of the node array.
[0036] In this embodiment, the P×Q nodes E p,q The corresponding result sub-matrices can be directly spliced according to the positions (i.e., the rows and the columns) of the corresponding nodes in the node array, so as to obtain the final result of the matrix multiplication of the first matrix and the second matrix; for example, assuming that each node E p,q corresponds to a result sub-matrix Y pq , the result sub-matrices Y pq in the P×Q nodes can be spliced to obtain Y[M][N], i.e., Y[M][N] can be regarded as a result sub-matrix array, wherein the first row is Y 11 , Y 12 , …, Y 1Q ; the second row is Y 21 , Y 22 , …, Y 2Q ; and the last row is Y P1 , Y P2 , …, Y PQAnd so on. When there are multiple result submatrices in each node, the multiple result submatrices in the node form a result subarray. The result subarrays corresponding to multiple nodes are concatenated according to the position of the corresponding node in the node array to obtain Y[M][N].
[0037] In this embodiment, the data processing architecture 20 has P×Q nodes E p,q It can directly or indirectly connect to other nodes E in the same row and column of the node array. p,q This allows for the transmission and reception of the first and second submatrices; P×Q nodes E p,q It can, but is not limited to, forming a two-dimensional mesh network or a dedicated data link, with each pair of adjacent nodes E p,q They can communicate point-to-point or broadcast by row and column.
[0038] In this embodiment, each node E in the data processing architecture 20 p,q Based on the first submatrix stored in this node, the first submatrix received from the other Q-1 nodes in the same row, the second submatrix stored in this node, and the second submatrix received from the other P-1 nodes in the same row, calculate one or more result submatrixes corresponding to this node. That is, calculate the corresponding result submatrix based on the first submatrix in the Q nodes in the same row and the second submatrix in the P nodes in the same column.
[0039] In this embodiment, P and Q may be equal or unequal.
[0040] In this embodiment, the first submatrix derived from the first matrix can be viewed as an array of first submatrixes, and the second submatrix derived from the second matrix can be viewed as an array of second submatrixes. If each node E p,q If only one first submatrix and one second submatrix are stored, then the first matrix is divided into P rows and Q columns of first submatrixes, and the second matrix is divided into P rows and Q columns of second submatrixes; each node E p,q The positions of the saved first submatrix and the saved second submatrix within the first matrix are identical to the position of the node in the node array. If each node E... p,q It stores multiple first submatrices and multiple second submatrices, and any node E p,q The position of the matrix formed by the multiple first sub-matrices stored in the first matrix, and the position of the matrix formed by the multiple second sub-matrices stored in the second matrix, are the same as the position of the node in the node array. Here, position refers to the row and column number of the node or sub-matrix array.
[0041] In this embodiment, a node E p,qEach node can store one or more first submatrices and one or more second submatrices. The size of each first submatrix and each second submatrix can be the same. If each node contains only one first submatrix and one second submatrix, then the number of rows in the first submatrix is M / P, the number of columns is L / Q, and the number of rows in the second submatrix is L / P, and the number of columns is N / Q.
[0042] In this embodiment, the first submatrix, divided by the first matrix, can be evenly distributed across P×Q nodes E. p,q In the second matrix, the second submatrix can be evenly distributed across P×Q nodes E. p,q In this case, each node has the same number of first submatrices and the same number of second submatrices; M and L are integer multiples of P, and L and N are integer multiples of Q. Alternatively, M and L do not necessarily have to be integer multiples of P, and L and N do not necessarily have to be integer multiples of Q. The number of first submatrices and the number of second submatrices in each node are not necessarily the same, but in the array of first submatrices, the total number of columns in a row of all elements in the first submatrices is equal to the total number of rows in a column of all elements in the second submatrices, that is, the number of columns in the first matrix is equal to the number of rows in the second matrix. Thus, for a distributed two-dimensional node array with a fixed number of rows and columns, this is applicable to matrix multiplication calculations of any structure, has high applicability, and is easy to promote and apply.
[0043] The data processing architecture 20 in this embodiment can divide two large matrices undergoing matrix multiplication into multiple relatively smaller sub-matrices, which are then distributed across multiple nodes for storage and computation. This allows each node to perform only the sub-matrix multiplication and store the resulting sub-matrices, significantly reducing costs. This data processing architecture provides a parallel architecture for the storage and arrangement of large-scale matrices, with each node E in the node array... p,q The system can compute submatrices in partial or full parallelism. Submatric transmission between nodes only requires communication within the same row and column. Each node is responsible for its own small matrix computation task, based on the first submatrix stored by Q nodes in the same row and the second submatrix stored by P nodes in the same column. This provides the hardware foundation for converting large-scale matrix multiplication operations into parallel computations of multiple small matrices, significantly reducing the time required for large-scale matrix multiplication operations. Furthermore, all nodes E in the node array... p,q The structure can be identical, and the number of nodes or columns can be increased according to actual needs. It is highly scalable, easy to implement standardized design and integration, and the performance between each node is consistent, which improves the reliability of the data processing architecture. When a single node fails, the faulty node can be replaced. It is easy to maintain, user-friendly and environmentally friendly, and reduces the maintenance cost of the data processing architecture.
[0044] In an example embodiment, each node comprises a computing sub-node and a storage module;
[0045] The storage module is configured to save the first sub-matrix, the second sub-matrix and the result sub-matrix;
[0046] The computing sub-node comprises:
[0047] The processing module is configured to read the first sub-matrix and the second sub-matrix from the storage module, calculate the result sub-matrix corresponding to the node according to the sub-matrix saved by the node and the received sub-matrix, and save the result sub-matrix into the storage module;
[0048] The row data communication module is configured to send the first sub-matrix read by the processing module to other Q-1 nodes in the same row as the node, and receive the first sub-matrix from other Q-1 nodes in the same row;
[0049] The column data communication module is configured to send the second sub-matrix read by the processing module to other P-1 nodes in the same column as the node, and receive the second sub-matrix from other P-1 nodes in the same column.
[0050] In an example embodiment of the present embodiment, the data processing architecture is as shown in Figure 2 The row data communication module and the column communication module can be integrated in a network module, and the nodes communicate with each other through the network module; the storage module can comprise a temporary storage module and a memory module, and each node sends and receives the sub-matrix through the network module and temporarily stores the sub-matrix in the temporary storage module for the processing module to perform matrix multiplication calculation, the multiplication result calculated is temporarily stored in the temporary storage module and finally saved in the memory module; the sub-matrix saved by a node can all be stored in the memory module, or a part of the sub-matrix can be stored in the cache and another part of the sub-matrix can be stored in the memory module.
[0051] In an example embodiment of the present embodiment, the storage module comprises a temporary storage module arranged in the computing sub-node and configured to save the first sub-matrix, the second sub-matrix and the result sub-matrix.
[0052] In the present embodiment, each node can only comprise a computing sub-node, and the first sub-matrix, the second sub-matrix and the result sub-matrix can all be saved in the temporary storage module, without arranging another memory sub-node.
[0053] In another example embodiment of the present embodiment, the storage module comprises a temporary storage module arranged in the computing sub-node and a memory sub-node connected to the computing sub-node;
[0054] The memory sub-node is configured to save the first sub-matrix, the second sub-matrix and the result sub-matrix;
[0055] The temporary storage module is configured to temporarily store the first sub-matrix and the second sub-matrix read out by the processing module from the memory sub-node, and the result sub-matrix calculated by the processing module;
[0056] The processing module is further configured to save the result sub-matrix temporarily stored in the temporary storage module to the memory sub-node.
[0057] As shown in the embodiment, Figure 3 Each node 10 can respectively include:
[0058] a calculation sub-node 30, and a memory sub-node 40 connected to the calculation sub-node 30;
[0059] The memory sub-node 40 is configured to save the first sub-matrix, the second sub-matrix and the result sub-matrix.
[0060] The calculation sub-node 30 includes:
[0061] a temporary storage module 31;
[0062] a processing module 32 configured to read the first sub-matrix and the second sub-matrix to be sent from the memory sub-node, and temporarily store them in the temporary storage module; calculate the result sub-matrix corresponding to the node according to the first sub-matrix and the second sub-matrix temporarily stored in the temporary storage module, and save it to the memory sub-node;
[0063] a row data communication module 33 configured to send the first sub-matrix temporarily stored in the temporary storage module 31 by the processing module 32 to other Q-1 nodes in the same row as the node, and receive the first sub-matrix from other Q-1 nodes in the same row, and temporarily store it in the temporary storage module 31;
[0064] a column data communication module 34 configured to send the second sub-matrix temporarily stored in the temporary storage module 31 by the processing module 32 to other P-1 nodes in the same column as the node, and receive the second sub-matrix from other P-1 nodes in the same column, and temporarily store it in the temporary storage module 31.
[0065] In the embodiment, the sub-matrix to be calculated by the processing module 32 is temporarily placed in the temporary storage module 31, and the sub-matrix to be calculated includes two cases, one is the sub-matrix to be sent out saved in the memory sub-node of the node, and the other is the sub-matrix received from other nodes. The processing module 32 can use part or all of the sub-matrices in the temporary storage module 31 when calculating. As for which sub-matrices to use, the processing module 32 can be instructed to select by programming or the like, and a sub-matrix can be uniquely identified by its name (such as x or w) and coordinates (indicated by the subscript of x or w in this document).
[0066] In this embodiment, the P x Q computing sub-nodes 30 and the P x Q memory sub-nodes 40 are connected one by one in a one-to-one correspondence to form a parallel memory access architecture, that is, each computing sub-node 30 has a dedicated memory access bandwidth when reading the first and second sub-matrices from the memory sub-nodes 40 and writing the result sub-matrices to the memory sub-nodes 40, so that the computing sub-nodes in the entire node array can read and write the memory sub-nodes in parallel, and the sub-matrices can be sent in parallel after being read in parallel, greatly improving the memory access efficiency and the efficiency of transmitting the sub-matrices, and reducing the demand for large-capacity memory and the hardware cost.
[0067] In this embodiment, the memory sub-nodes 40 can be DRAM, or one or more of SRAM, PCM, ReRAM, Flash, etc., connected with the computing sub-nodes 30 through the memory ports 41; the temporary storage module 31 can be, but is not limited to, SRAM, and the processing module 32 can be a matrix calculation circuit.
[0068] In this embodiment, the row data communication module 33 and the column data communication module 34 can share part of the hardware resources, for example, the row data communication module 33 can include a routing module and one or two first interfaces in the row direction, the first interfaces are connected to adjacent nodes in the same row, and the routing module performs operations such as encapsulation, routing, sending, receiving, and decapsulation of data packets; the column data communication module 34 can include the routing module and one or two second interfaces in the column direction, the second interfaces are connected to adjacent nodes in the same column. Or they can be independent data links.
[0069] In this embodiment, the hardware implementation form of the computing sub-nodes and the memory sub-nodes is not limited, for example, the P x Q computing sub-nodes 30 can form a logical array, which can be encapsulated in a logical die; the P x Q memory sub-nodes 40 can form a storage array, which can be encapsulated in a storage die, and the logical die and the storage die can be stacked and bonded. For another example, a processing chip with memory can be used as a node to form a chip array. For another example, different cores in a multi-core processor can be used as different computing sub-nodes. Or, for example, each computing sub-node and the corresponding memory sub-node can be directly connected in the same circuit and encapsulated in one die. Or, the computing sub-nodes and the memory sub-nodes can be bonded together or directly encapsulated in one chip.
[0070] In other embodiments, the memory sub-nodes can only save one of the first and second sub-matrices, and the other sub-matrix can be saved in the temporary storage module; for example, in the calculation of a neural network, the first sub-matrix as the input quantity of each layer is saved in the temporary storage module, and the second sub-matrix as the weight can be saved in the memory sub-node.
[0071] In other embodiments, one or more memory sub-nodes can be shared by multiple computing sub-nodes, each of which reads the first and second sub-matrices through different memory addresses and saves the calculated results in different memory addresses.
[0072] In other embodiments, a P×Q memory area arranged in P rows and Q columns can be used to save the first, second and result sub-matrices, and each computing sub-node can access the corresponding memory area through two-dimensional coordinates (e.g., the memory area in the i-th row and j-th column).
[0073] In an exemplary embodiment, the node E p,q The first sub-matrix saved by the node can be sent to the other Q-1 nodes in the same row as the node, and the second sub-matrix saved by the node can be sent to the other P-1 nodes in the same column as the node, which can include:
[0074] The node E p,q The first sub-matrix saved by the node can be broadcast to the other Q-1 nodes in the same row as the node, and the second sub-matrix saved by the node can be broadcast to the other P-1 nodes in the same column as the node, which can include: p,q p,q .
[0075] In this embodiment, the node uses the same row broadcast and the same column broadcast when performing the same row sending and the same column sending, and broadcasts the first sub-matrix to the other nodes in the same row and broadcasts the second sub-matrix to the other nodes in the same column.
[0076] In other embodiments, multiple data packets can be used for sending, i.e., each node sends Q-1 data packets to send the first sub-matrix to the other Q-1 nodes in the same row, and sends P-1 data packets to send the second sub-matrix to the other P-1 nodes in the same column; or the first sub-matrix can be forwarded along the row node by node in each row, and the second sub-matrix can be forwarded along the column node by node in each column.
[0077] The embodiments of the present application do not limit the form of sending the first and second sub-matrices, as long as the first sub-matrix can be sent to the other nodes in the same row and the second sub-matrix can be sent to the other nodes in the same column.
[0078] In an exemplary embodiment, the P×Q nodes E p,q The first sub-matrix and the second sub-matrix are transmitted in parallel, i.e., the P x Q nodes transmit the first sub-matrix in parallel, the P x Q nodes transmit the second sub-matrix in parallel, and each node transmits the first sub-matrix and the second sub-matrix in parallel. In this case, all nodes can transmit the saved first sub-matrix in parallel in the same time period, and transmit the saved second sub-matrix in parallel in the same time period, or in other words, the P x Q nodes transmit the first sub-matrix in the same time period, and transmit the second sub-matrix in the same time period.
[0079] In another exemplary embodiment, the P x Q nodes can be transmitted in a predetermined order in a plurality of predetermined batches, and only part of the nodes are transmitted in each batch.
[0080] In one implementation of the embodiment, the nodes E p,q The first sub-matrix can be transmitted in different time periods respectively, and the nodes E p,q The first sub-matrix is transmitted in parallel. p,q The second sub-matrix can be transmitted in different time periods respectively, and the nodes E p,q The second sub-matrix is transmitted in parallel.
[0081] In the embodiment, the P x Q nodes E p,q The transmission is performed column by column and row by row, i.e., the nodes in the same column transmit the first sub-matrix to the nodes in the same row in parallel, and the nodes in different columns transmit the first sub-matrix staggered; the nodes in the same row transmit the second sub-matrix to the nodes in the same column in parallel, and the nodes in different rows transmit the second sub-matrix staggered.
[0082] In one exemplary embodiment, each node can transmit and receive the sub-matrix in the same time period, i.e., the node can transmit the sub-matrix while receiving the sub-matrix. Each node can first transmit one kind of sub-matrix, and then transmit another kind of sub-matrix, for example, the node first transmits the first sub-matrix saved by the node, and then transmits the second sub-matrix saved by the node. Any node can transmit all the first sub-matrix or the second sub-matrix saved by the node at one time, or when the node saves a plurality of first sub-matrices or second sub-matrices, the node can transmit the first sub-matrices or the second sub-matrices in multiple times.
[0083] In one exemplary embodiment, the number of times or the period of transmitting the sub-matrix can be set according to the size of P and Q in the node array. For example, when P is twice Q, two rows and one column of nodes in the node array can be set to the transmitting state at each time of transmission or each period of transmission, or one column transmission is performed after two row transmissions; when P is equal to Q, one row and one column of nodes in the node array can be set to the transmitting state at each time of transmission or each period of transmission, or one column transmission is performed after one row transmission. In one exemplary embodiment, one node Ep,q Save a first submatrix X pq and a second submatrix W pq ;
[0084] Node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:
[0085] The node E p,q When the first preset condition is met, the first submatrix X stored in this node will be... pq Send the data to the other Q-1 nodes in the same row as this node; when the second preset condition is met, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.
[0086] In this embodiment, the first preset condition may be: reaching a first preset period, for example, sending when reaching period q, that is: when node E reaches the first period. p,1 Send the first submatrix X p1 When the second cycle is reached, node E p,2 Send the first submatrix X p2 The same logic applies to other cycles, and will not be elaborated here; alternatively, the first preset condition could be: after receiving the first sub-matrix sent by the predetermined node, this node sends the first sub-matrix stored by this node, for example, node E. 1,2 Received from node E 1,1 The first submatrix X sent 11 Then send the first submatrix X saved in this node. 12 Node E 2,3 Received from node E 2,2 The first submatrix X sent 22 Then send the first submatrix X saved in this node. 23 The same logic applies to other nodes, and will not be elaborated here.
[0087] In this embodiment, the second preset condition can be: reaching a second preset period, for example, sending when reaching period p, that is: when node E reaches the first period. 1,q Send the second submatrix W 1q When the second cycle is reached, node E 2,q Send the second submatrix W 2q The same logic applies to other periods, and will not be elaborated here; alternatively, the second preset condition could be: after receiving the second sub-matrix from the predetermined node, this node sends the second sub-matrix stored in this node, for example, node E. 1,2 Received from node E2,2 The second submatrix W sent 22 Then send the second submatrix W stored in this node. 12 Node E 2,3 Received from node E 3,3 The second submatrix W sent 33 Then send the second submatrix W stored in this node. 23 The same logic applies to other nodes, and will not be elaborated here.
[0088] In this embodiment, the first preset condition and the second preset condition are not limited to the situations listed above.
[0089] In one exemplary embodiment of this example, node E p,q The first submatrix X stored in this node can be saved after power-on or after receiving the command to start the operation. pq It automatically sends the second submatrix W stored in this node to the other Q-1 nodes in the same row as this node. pq Automatically sends to P-1 other nodes in the same column as this node; "automatic sending" means that no other conditions need to be judged, and the power-on or operation command is used as the first and second preset conditions, such as no need to judge the sending period or whether other nodes have completed sending.
[0090] In another exemplary embodiment of this invention, each node can first store the X data saved by its own node. pq Send to the same location, in all X pq After sending is complete, each node will then store the W data that it has saved. pq Send in the same column; or conversely, complete W first. pq Send X, then send X pq .
[0091] In other embodiments, the sending conditions and order of the first and second submatrices can be set according to requirements, as long as it is ensured that the first submatrices saved by each node can be sent to other nodes in the same row, and the second submatrices saved by each node can be sent to other nodes in the same column.
[0092] In one exemplary embodiment, node E p,q The result submatrix corresponding to this node is obtained based on the submatrix stored and received by this node, and may include:
[0093] Node E p,qAfter receiving all the first sub-matrices saved by the Q-1 other nodes in the same row and all the second sub-matrices saved by the P-1 other nodes in the same column, the first sub-matrices saved by the node and the received first sub-matrices are spliced according to the arrangement order of the first sub-matrices in the first matrix to obtain a first spliced matrix, the second sub-matrices saved by the node and the received second sub-matrices are spliced according to the arrangement order of the second sub-matrices in the second matrix to obtain a second spliced matrix, and matrix multiplication is performed on the first spliced matrix and the second spliced matrix to obtain the result sub-matrix corresponding to the node.
[0094] The arrangement order of the first sub-matrices in the first matrix can be the order from small to large of the coordinates representing the number of columns in the coordinates of the first sub-matrices, and the arrangement order of the second sub-matrices in the second matrix can be the order from small to large of the coordinates representing the number of rows in the coordinates of the second sub-matrices.
[0095] In another exemplary embodiment, splicing is not necessary; the first matrix is divided into R rows and S columns of first sub-matrices X ij , and the second matrix is divided into S rows and T columns of second sub-matrices W hk ; i represents the sequence number of the row to which the first sub-matrix X ij belongs in the first matrix, j represents the sequence number of the column to which the first sub-matrix X ij belongs in the first matrix; h represents the sequence number of the row to which the second sub-matrix W hk belongs in the second matrix, and k represents the sequence number of the column to which the second sub-matrix W hk belongs in the second matrix; i = 1, 2, …, R, j = 1, 2, …, S, h = 1, 2, …, S, and k = 1, 2, …, T.
[0096] In this embodiment, the node E p,q obtains the result sub-matrix corresponding to the node according to the sub-matrices saved by the node and the received sub-matrices, which can include:
[0097] The node E p,q , in the saved and received first sub-matrices and second sub-matrices, takes one first sub-matrix X ij and one second sub-matrix W hk as a pair of sub-matrices when j and h are equal, respectively performs matrix multiplication calculation on each pair of sub-matrices, and accumulates the matrix multiplication results corresponding to the same result sub-matrix.
[0098] In this embodiment, the number of columns of the first sub-matrix in a pair of sub-matrices is equal to the number of rows of the second sub-matrix.
[0099] In this embodiment, the first matrix can be regarded as an array of R rows and S columns of first sub-matrices X ijIn the i-th row and j-th column of the first submatrix array; the second matrix can be viewed as an S-row, T-column submatrix array, where the second submatrix W... hk In the h-th row and k-th column of the second submatrix array.
[0100] In one exemplary embodiment of this example, R = S = T = P = Q, the node array is a square matrix, and the total number of the first submatrix and the total number of the second submatrix are both equal to the total number of nodes. Each node stores one first submatrix and one second submatrix. The position or two-dimensional coordinates of the submatrix in the matrix are the same as the position or two-dimensional coordinates of the node in the node array. For example, if the node array is 2 rows and 3 columns, and the first matrix is divided into 2 rows and 3 columns of first submatrix, then the first submatrix in the first row and first column is located at E. 1,1 The first submatrix in the 2nd row and 3rd column is located in E. 2,3 The same applies to the second submatrix.
[0101] In other implementations, P and Q, R and S, and S and T need not be equal; it is sufficient that each first submatrix has a pairable second submatrix during calculation, and the number of columns of the first submatrix is equal to the number of rows of the paired second submatrix.
[0102] In one exemplary embodiment, each node E p,q Store a first submatrix X respectively pq and a second submatrix W pq When P = Q, P × Q nodes can send the first submatrix X in P cycles. pq Second submatrix W pq ;
[0103] In this embodiment, node E p,q Send the first submatrix stored in this node to the other Q-1 nodes in the same row as this node, and send the second submatrix stored in this node to the other P-1 nodes in the same column as this node. This can include:
[0104] Node E p,q In the q-th period, store the first submatrix X at this node. pq Send the data to the other Q-1 nodes in the same row as this node; in the p-th cycle, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.
[0105] In this embodiment, the node array is a square node array, with Figure 4 Taking the 4×4 node matrix with P=Q=4 as an example, the first submatrix X of the 4×4 node matrix is divided into 4 periods. pq Second submatrix W pqsending; the following takes several nodes in the 4x4 node square as an example to introduce the sending of the sub-matrix, for example, node E 1,1 In the first cycle, the first sub-matrix X 11 saved by the node is sent to node E 1,2 , E 1,3 , E 1,4 , respectively 11 In the second cycle, the second sub-matrix W 2,1 saved by the node is sent to node E 3,1 , E 4,1 , respectively 2,3 In the third cycle, the first sub-matrix X 23 saved by the node is sent to node E 1,3 , E 3,3 , E 4,3 , respectively 23 In the fourth cycle, the second sub-matrix W 2,1 saved by the node is sent to node E 2,2 , E 2,4 , respectively 4,2 In the second cycle, the first sub-matrix X 42 saved by the node is sent to node E 4,1 , E 4,3 , E 4,4 , respectively 42 In the fourth cycle, the second sub-matrix W 1,2 saved by the node is sent to node E 2,2 , E 3,2 , respectively
[0106] In this embodiment, the multiplication of the first sub-matrix and the second sub-matrix can be calculated once after the sending of each cycle is completed, or the sub-matrices can be paired or spliced after the transmission of the first sub-matrix and the second sub-matrix of all nodes is completed, and then the matrix multiplication calculation is performed.
[0107] In other embodiments, P and Q can not be equal, and the first sub-matrix and the second sub-matrix can not be transmitted according to the cycle; for example, the first sub-matrix saved by all nodes can be sent in turn first, and then the second sub-matrix saved by all nodes is sent in turn; for another example, the first sub-matrix and the second sub-matrix can be sent in parallel by all nodes.
[0108] In an exemplary embodiment of this embodiment, one node corresponds to one result sub-matrix Y pq , which can be obtained after Q-1 first sub-matrices X pq and P-1 second sub-matrices W pqThen, the first sub-matrix X saved by the node and received is transversely spliced according to the order of q from small to large to obtain a first spliced matrix pq The second sub-matrix W saved by the node and received is vertically spliced according to the order of p from small to large to obtain a second spliced matrix pq The first spliced matrix and the second spliced matrix are multiplied to obtain the result sub-matrix Y pq .
[0109] In the embodiment, the first spliced matrix is spliced by Q first sub-matrices X pq , and the column number of the first spliced matrix is equal to the column number of the first matrix X; the second spliced matrix is spliced by P second sub-matrices W pq , and the row number of the second spliced matrix is equal to the row number of the second matrix W; since the column number of X is equal to the row number of W when the matrix multiplication X×W is performed, the column number of the first sub-matrix is not required to be equal to the row number of the second sub-matrix, and the column number of the first spliced matrix is still ensured to be equal to the row number of the second spliced matrix, so that the multiplication of the first spliced matrix and the second spliced matrix can be performed to obtain the result sub-matrix Y pq .
[0110] In another exemplary embodiment of the embodiment, each node can calculate A a = X pq ×W pq after each pair of sub-matrices is obtained, and the result sub-matrix Y is calculated according to P A a .
[0111] wherein each pair of sub-matrices includes a first sub-matrix X pq and a second sub-matrix W pq , and q in X pq is equal to p in W pq , for example, in the case of P=4, X 11 is paired with W 11 , W 12 , W 13 , and W 14 , respectively, X 13 is paired with W 31 , W 32 , W 33 , and W 34 , respectively, X 23 is paired with W 31 , W 32 , W 33 , and W 34 , respectively; only some of the paired sub-matrices are listed here, and other cases are similar; a=1, 2, …, P.
[0112] In this embodiment, obtaining a sub-matrix refers to reading a sub-matrix stored in the node, or receiving a sub-matrix from another node, and so on. Figure 4 For example, the node E 1,1 In the first period, a pair of sub-matrices X 11 and W 11 are read from the node, and then A1=X 11 ×W 11 is calculated; in the second period, a pair of sub-matrices X 12 and W 21 are received, and then A2=X 12 ×W 21 is calculated; in the third period, a pair of sub-matrices X 13 and W 31 are received, and then A3=X 13 ×W 31 is calculated; in the fourth period, a pair of sub-matrices X 14 and W 41 are received, and then A4=X 14 ×W 41 is calculated; and then the result sub-matrix Y 11 =A1+A2+A3+A4 is calculated. The same applies to other nodes, and thus is not described herein.
[0113] In this embodiment, each node E p,q can perform multiplication of X pq and W pq in P periods;
[0114] The node E p,q obtains the result sub-matrix corresponding to the node according to the sub-matrices stored in the node and received by the node, and can include:
[0115] In each of the P periods, the node E p,q performs matrix multiplication of X pq and W pq transmitted in the period; and accumulates the results of the multiplication in the P periods to obtain the result sub-matrix Y pq corresponding to the node.
[0116] In each period, the sub-matrices include: transmitted X pq and transmitted W pq , or transmitted X pq and received W pq , or received X pq and transmitted W pq , or received X pq and received W pq .
[0117] In this embodiment, after acquiring the first and second submatrices in each cycle, the multiplication calculation for that cycle is performed. This multiplication calculation and the transmission of the next cycle can be parallelized, for example, by using a ping-pong operation. The temporary storage module is divided into two regions: the first region stores the submatrices being read and sent, as well as the submatrices being received, and the second region stores the submatrices undergoing multiplication calculation. After the transmission and calculation are completed, a switch is made. The submatrices read and received in the first region are ready for the next round of multiplication calculation, while the second region is used to store newly read and received submatrices. This saves time overall and is more efficient.
[0118] In this embodiment, the result submatrix Y of each node can be... pq The initial value is set to a zero matrix, and X is calculated after each cycle of multiplication. pq ×W pq The result of the multiplication is compared with the current result submatrix Y. pq Adding them together yields a new submatrix Y. pq The result submatrix Y is obtained by adding the result of the multiplication calculation in the Pth cycle to the result submatrix obtained in the (P-1)th cycle. pq Alternatively, the results of the multiplication calculations in each cycle can be saved. After performing P cycles of multiplication calculations, the results of the P cycles of multiplication calculations can be added together to obtain the resulting submatrix Y. pq .
[0119] In this embodiment, calculations for that cycle are performed after a transmission cycle, and the next cycle of transmission begins after the calculations are completed. Figure 4 The node E in the square matrix of nodes P=Q=4 shown 1,1 For example: After the transmission of the first cycle, the calculation of the first cycle is performed, based on the first sub-matrix X transmitted by this node to the nodes in the same row. 11 and the second submatrix W transmitted to nodes in the same column 11 Perform multiplication calculation X 11 ×W 11 The calculation for the second cycle is performed after the transmission of the second cycle, that is, after receiving the first submatrix X. 12 Second submatrix W 21 Then, perform multiplication to calculate X. 12 ×W 21 The calculation for the third cycle is performed after the transmission of the third cycle, that is, after receiving the first submatrix X. 13 Second submatrix W 31 Then, perform multiplication to calculate X. 13 ×W 31The calculation for the fourth cycle is performed after the transmission of the fourth cycle, that is, after receiving the first submatrix X. 14 Second submatrix W 41 Then, perform multiplication to calculate X. 14 ×W 41 After each cycle of multiplication is completed, the result of the multiplication is accumulated into the result submatrix Y. pq Alternatively, after performing four cycles of multiplication, the resulting submatrix Y is calculated. 11 =X 11 ×W 11 +X 12 ×W 21 +X 13 ×W 31 +X 14 ×W 41 .
[0120] by Figure 4 The node E in the square matrix of nodes P=Q=4 shown 2,3 For example: The calculation of the first cycle is performed after the transmission of the first cycle, that is, the calculation of the first cycle is performed after the node receives the first submatrix X. 21 Second submatrix W 13 Then, perform multiplication to calculate X. 21 ×W 13 The calculation for the second cycle is performed after the transmission of the second cycle, that is, after receiving the first submatrix X. 22 And send the second submatrix W 23 Then, perform multiplication to calculate X. 22 ×W 23 The calculation for the third cycle is performed after the transmission of the third cycle, that is, after sending the first submatrix X. 23 Second submatrix W 33 Then, perform multiplication to calculate X. 23 ×W 33 The calculation for the fourth cycle is performed after the transmission of the fourth cycle, that is, after receiving the first submatrix X. 24 Second submatrix W 43 Then, perform multiplication to calculate X. 24 ×W 43 After each cycle of multiplication is completed, the result of the multiplication is accumulated into the result submatrix Y. pq Alternatively, after performing four cycles of multiplication, the resulting submatrix Y is calculated. 23 =X 21 ×W 13 +X 22 ×W 23 +X 23 ×W 33 +X 24 ×W43 .
[0121] In an example implementation, each node E p,q stores I×J first sub-matrices and H×K second sub-matrices respectively, wherein the number of columns of the first sub-matrices is equal to the number of rows of the second sub-matrices, and J×Q=H×P; each node E p,q corresponds to I×K result sub-matrices respectively;
[0122] The node E p,q sending the first sub-matrices stored by the node to the other Q-1 nodes in the same row as the node and sending the second sub-matrices stored by the node to the other P-1 nodes in the same column as the node can include:
[0123] The node E p,q sending the first sub-matrices and the second sub-matrices stored by the node in multiple times, and sending at least one first sub-matrix to the other Q-1 nodes in the same row as the node and sending at least one second sub-matrix to the other P-1 nodes in the same column as the node in each time;
[0124] The node E p,q calculating the result sub-matrices corresponding to the node according to the sub-matrices stored by the node and the received sub-matrices can include:
[0125] The node E p,q performing matrix multiplication calculation on the first sub-matrices and the second sub-matrices transmitted in each time, and accumulating the matrix multiplication result into the corresponding result sub-matrices in the node;
[0126] The first sub-matrices and the second sub-matrices transmitted in each time include the transmitted first sub-matrices and the transmitted second sub-matrices, or the transmitted first sub-matrices and the received second sub-matrices, or the received first sub-matrices and the transmitted second sub-matrices, or the received first sub-matrices and the received second sub-matrices.
[0127] In the example implementation, when multiple first sub-matrices are stored in the node, the multiple first sub-matrices are arranged in the form of an array, and when multiple second sub-matrices are stored in the node, the multiple second sub-matrices are arranged in the form of an array; I and J are integers greater than 1, or one of I and J can be 1 (if both are 1, the node only stores one first sub-matrix); H and K are integers greater than 1, or one of H and K can be 1 (if both are 1, the node only stores one second sub-matrix).
[0128] In the example implementation, if the first matrix is divided into R rows and S columns of first sub-matrices X ij , and the second matrix is divided into S rows and T columns of second sub-matrices W hkIf the first and second submatrices are evenly distributed to P×Q nodes, then we have I=R / P, J=S / Q, H=S / P, K=T / Q; if R / P, S / Q, S / P, and T / Q are not integers, they can be rounded up to become I, J, H, and K respectively.
[0129] In this embodiment, the P×Q nodes can send data multiple times in parallel, or different nodes can send data at staggered times. For example, the nodes in the first row send data first, then the nodes in the second row send data first, and after all the nodes in all rows have sent data, the nodes in the first row send data second, then the nodes in the second row send data second, and so on. This embodiment does not limit the sending order of the nodes or whether they send data in parallel, as long as all the first submatrices in each node can be sent to other nodes in the same row, and all the second submatrices can be sent to other nodes in the same column.
[0130] In this embodiment, the node can perform matrix multiplication on the paired submatrixes after each submatrix transmission is completed; the transmission of submatrixes includes four cases: sending the first submatrix and the second submatrix, receiving the first submatrix and the second submatrix, sending the first submatrix and receiving the second submatrix, and receiving the first submatrix and sending the second submatrix.
[0131] In this embodiment, the number of first submatrices and second submatrices in a node are not necessarily the same; in one example, P=2, Q=4, R=4, S=8, T=8, that is, the node array includes 2×4 nodes 10, and each node has 2×2 first submatrices X. ij and 4×2 second submatrices W hk ,like Figure 5 As shown; each node will yield 2×2 result submatrices.
[0132] In this example, to ensure that the submatrices can be paired after each transmission, each node can send one first submatrix and two second submatrices each time. Figure 5 The submatrix within the dashed box represents the first transmission. For example, taking node E... 1,1 For example, during the first transmission, X is sent. 11 and W 11 and W 31 Received from node E 2,1 W sent 51 and W 71 and node E 1,2 E 1,3 E 1,4 Send X 13 X 15 X 17 During the second transmission, X is sent. 12 and W21 and W 41 , the receiving node E 2,1 sends W 61 and W 81 , and the node E 1,2 , E 1,3 , E 1,4 sends X 14 , X 16 , X 18 ; the third time, the sending node sends X 21 and W 11 and W 31 , the receiving node E 2,1 sends W 51 and W 71 , and the node E 1,2 , E 1,3 , E 1,4 sends X 23 , X 25 , X 27 ; the fourth time, the sending node sends X 22 and W 21 and W 41 , the receiving node E 2,1 sends W 61 and W 81 , and the node E 1,2 , E 1,3 , E 1,4 sends X 24 , X 26 , X 28 . The fifth to eighth times, the order of transmission of the first sub-matrix is the same as the first to fourth times, except that the second sub-matrix is changed to transmit the second column (W 11 to W 81 is changed to W 12 to W 82 ).
[0133] In this example, in the calculation, the saved and received sub-matrices can be paired each time, for example, taking the node E 1,1 as an example, after the first transmission, X 11 X W 11 , X 13 X W 31 , X 15 X W 51 , X 17 X W 71 are added to the result sub-matrix Y 11 ; after the second transmission, X 12 X W 21 , X 14 X W 41 , X 16XW 61 XW 18 XW 81 , accumulated to the result sub-matrix Y 11 ; after the third transmission, X 21 XW 11 , X 23 XW 31 , X 25 XW 51 , X 27 XW 71 , accumulated to the result sub-matrix Y 21 ; after the fourth transmission, X 22 XW 21 , X 24 XW 41 , X 26 XW 61 , X 28 XW 81 , accumulated to the result sub-matrix Y 21 . The multiplication calculation after the fifth to the eighth transmission is similar, only the W 11 to W 81 in the formula are replaced by W 12 to W 82 , and the result sub-matrix is replaced by Y 12 and Y 22 .
[0134] It can be seen that what a node has to do is the matrix multiplication between the whole row (for node E 1,1 , X 11 to X 18 whole row, and X 21 to X 28 whole row) of the saved first sub-matrix in the first sub-matrix array and the whole column (for node E 1,1 , W 11 to W 81 whole column) of the saved second sub-matrix in the second sub-matrix array, but is divided into IxJxK=8 times of transmission and calculation.
[0135] In other embodiments, a node can send the saved multiple first sub-matrices to other nodes in the same row at one time and send the saved multiple second sub-matrices to other nodes in the same column at one time, which can be handled by referring to the case where there is only one first sub-matrix and one second sub-matrix in a node, which is equivalent to regarding the multiple first sub-matrices as a whole and regarding the multiple second sub-matrices as a whole.
[0136] In other embodiments, after receiving all the first sub-matrices sent by other nodes in the same row and all the second sub-matrices sent by other nodes in the same column, the node can splice the saved first sub-matrices and the received first sub-matrices according to the positions of the first sub-matrices in the first matrix, splice the saved second sub-matrices and the received second sub-matrices according to the positions of the second sub-matrices in the second matrix, and then perform matrix multiplication on the two spliced matrices. For example, in the example shown in FIG. 8, after all the nodes send the sub-matrices, the saved first sub-matrices and the received first sub-matrices can be spliced according to the positions of the first sub-matrices in the first matrix, and the saved second sub-matrices and the received second sub-matrices can be spliced according to the positions of the second sub-matrices in the second matrix. Figure 5 For example, node E shown in FIG. 8 can splice the 16 first sub-matrices (4 saved and 12 received) into a 2x8 matrix, splice the 16 second sub-matrices (8 saved and 8 received) into an 8x2 matrix, and perform multiplication on the two spliced matrices to obtain a corresponding 2x2 result sub-matrix of node E. 1,1 For example, node E shown in FIG. 8 can splice the 16 first sub-matrices (4 saved and 12 received) into a 2x8 matrix, splice the 16 second sub-matrices (8 saved and 8 received) into an 8x2 matrix, and perform multiplication on the two spliced matrices to obtain a corresponding 2x2 result sub-matrix of node E. 1,1 For example, node E shown in FIG. 8 can splice the 16 first sub-matrices (4 saved and 12 received) into a 2x8 matrix, splice the 16 second sub-matrices (8 saved and 8 received) into an 8x2 matrix, and perform multiplication on the two spliced matrices to obtain a corresponding 2x2 result sub-matrix of node E.
[0137] On the other hand, the embodiments of the present application also provide a matrix multiplication calculation method, which is implemented based on the data processing architecture as described in any of the above embodiments. As shown in FIG. 10, the matrix multiplication calculation method comprises the following steps. Figure 6
[0138] S100, each node E of the data processing architecture p,q sends the first sub-matrices saved by the node to other Q-1 nodes in the same row, and sends the second sub-matrices saved by the node to other P-1 nodes in the same column;
[0139] S200, each node E of the data processing architecture p,q calculates a result sub-matrix corresponding to the node according to the first sub-matrices and the second sub-matrices saved by and received by the node;
[0140] In the embodiments, the matrix multiplication calculation Y=X×W can be implemented through steps S100 and S200.
[0141] In the embodiments, the matrix multiplication calculation Y=X×W can be implemented through steps S100 and S200.
[0142] In this embodiment, the P×Q nodes can perform step S100 in multiple times and perform step S200 in multiple times; one node can perform step S100 one or more times and perform step S200 one or more times. For one node, step S100 can be performed first and then step S200 can be performed, or vice versa. For all nodes, step S100 can be performed first and then step S200 can be performed; or step S100 and step S200 can be performed in an interleaved manner, such as some nodes perform step S100 first, some or all nodes perform step S200, then another part of nodes perform step S100 again, and some or all nodes perform step S200.
[0143] In this embodiment, each node E p,q will receive the first sub-matrix respectively sent by the other Q-1 nodes in the same row as the node, and receive the second sub-matrix respectively sent by the other P-1 nodes in the same column as the node.
[0144] The matrix multiplication calculation method of this embodiment divides the multiplication calculation of a large matrix into multiple small matrix multiplication calculations that can be performed in parallel, greatly improves the calculation efficiency of the multiplication of a large matrix, saves calculation time, and provides a basis for accurate matrix multiplication calculation.
[0145] In an exemplary embodiment, step S100 performed by each node can include steps S110-S120:
[0146] The nodes E p,q send the first sub-matrix respectively in different time periods, and the nodes E p,q in the same column send the first sub-matrix respectively in different time periods. p,q send the second sub-matrix respectively in different time periods, and the nodes E p,q in the same row send the second sub-matrix respectively in different time periods.
[0147] Alternatively, each node E p,q parallelly sends the first sub-matrix saved by the node to the other Q-1 nodes in the same row, and sends the second sub-matrix saved by the node to the other P-1 nodes in the same column.
[0148] In other embodiments, the P×Q nodes can send node by node in sequence, or not divide the sending batches in the manner of rows and columns.
[0149] In an exemplary embodiment, step S100 performed by each node can include steps S110-S120:
[0150] S110: broadcast the first sub-matrix stored by the node to other Q-1 nodes in the same row as the node;
[0151] S120: broadcast the second sub-matrix stored by the node to other P-1 nodes in the same column as the node.
[0152] Steps S110 and S120 can be executed in parallel, and when the first sub-matrix and the second sub-matrix stored by the node are both multiple, steps S110 and S120 can be executed multiple times. Different nodes can execute steps S110 and S120 in parallel or in batches according to a predetermined order.
[0153] In the embodiment, each first sub-matrix is only broadcast among nodes in the same row, and each second sub-matrix is only broadcast among nodes in the same column.
[0154] In other embodiments, other sending forms can be used, without being limited to the broadcast mode, to send the first sub-matrix to other nodes in the same row and the second sub-matrix to other nodes in the same column.
[0155] In an exemplary embodiment, step S200 can include:
[0156] Each node E p,q After receiving all the first sub-matrices stored by the Q-1 other nodes in the same row and all the second sub-matrices stored by the P-1 other nodes in the same column, the node E splices the first sub-matrices stored by the node E and received to obtain a first spliced matrix according to the arrangement order of the first sub-matrices in the first matrix, splices the second sub-matrices stored by the node E and received to obtain a second spliced matrix according to the arrangement order of the second sub-matrices in the second matrix, and performs matrix multiplication calculation on the first spliced matrix and the second spliced matrix to obtain the result sub-matrix corresponding to the node E.
[0157] In another exemplary embodiment, the first matrix is divided into R rows and S columns of first sub-matrices X ij , and the second matrix is divided into S rows and T columns of second sub-matrices W hk ; i represents the sequence number of the row to which the first sub-matrix X ij belongs in the first matrix, j represents the sequence number of the column to which the first sub-matrix X ij belongs in the first matrix; h represents the sequence number of the row to which the second sub-matrix W hk belongs in the second matrix, and k represents the sequence number of the column to which the second sub-matrix W hk belongs in the second matrix; i=1, 2, …, R, j=1, 2, …, S, h=1, 2, …, S, and k=1, 2, …, T.
[0158] Step S200 can include that each of the nodes E p,q In the saved and received first sub-matrix and second sub-matrix, the first sub-matrix X ij and the second sub-matrix W hk equal to j and h are taken as a pair of sub-matrices; and matrix multiplication is performed on each pair of obtained sub-matrices, and the matrix multiplication results corresponding to the same result sub-matrix are accumulated.
[0159] In other embodiments, the result sub-matrix can be obtained by performing multiplication on each pair of sub-matrices after the transmission is completed, or the result sub-matrix can be obtained by performing matrix multiplication on the transmitted sub-matrices after splicing each time, and accumulating the matrix multiplication results each time.
[0160] In an exemplary embodiment, each node E p,q can save a first sub-matrix X pq and a second sub-matrix W pq respectively; when P=Q, the P×Q nodes can send the first sub-matrix X pq and the second sub-matrix W pq in P periods, and step S100 can include:
[0161] The P×Q nodes send the first sub-matrix X pq and the second sub-matrix W pq in P periods; each node E p,q sends the first sub-matrix X pq saved by the node to the other Q-1 nodes in the same row as the node in the qth period; and sends the second sub-matrix W pq saved by the node to the other P-1 nodes in the same column as the node in the pth period.
[0162] In this embodiment, step S100 can be regarded as including P times of step S130, and taking the u th period as an example, step S130 includes that the node E p,q=u sends the first sub-matrix X pq saved by the node to the other Q-1 nodes in the same row as the node; and the node E p=u,q sends the second sub-matrix W pq saved by the node to the other P-1 nodes in the same column as the node.
[0163] Wherein, u=1, 2, …, P, that is, the P×Q nodes execute step S130 P times in total.
[0164] In an exemplary implementation of this embodiment, step S200 can include:
[0165] Each node E p,q In each of the P periods, for X pq and W pq performed matrix multiplication calculation; according to the results of the P period multiplication calculation, the result sub-matrix corresponding to the node is obtained by accumulation;
[0166] Wherein, X pq and W pq transmitted in one period include: transmitted X pq and transmitted W pq , or transmitted X pq and received W pq , or received X pq and transmitted W pq , or received X pq and received W pq .
[0167] In other embodiments, the batches of transmitted sub-matrices can be divided in other ways, which are not limited to row transmission or column transmission, or can be transmitted node by node; the order of transmission of different batches or nodes is not limited;
[0168] In other embodiments, P and Q can not be equal.
[0169] In an exemplary embodiment, each node E p,q can save I×J first sub-matrices and H×K second sub-matrices respectively; each node E p,q may correspond to I×K result sub-matrices respectively;
[0170] Step S100 can include:
[0171] Each of the nodes E p,q transmits the plurality of first sub-matrices and the plurality of second sub-matrices saved by the node in multiple times, at least one first sub-matrix is transmitted to other Q-1 nodes in the same row as the node each time, and at least one second sub-matrix is transmitted to other P-1 nodes in the same column as the node each time;
[0172] Step S200 can include:
[0173] Each of the nodes E p,q performs matrix multiplication calculation on the first sub-matrix and the second sub-matrix transmitted each time, and accumulates the matrix multiplication result into the corresponding result sub-matrix in the node;
[0174] The first and second sub-matrices of the one-time transmission include: a transmitted first sub-matrix and a transmitted second sub-matrix, or a transmitted first sub-matrix and a received second sub-matrix, or a received first sub-matrix and a transmitted second sub-matrix, or a received first sub-matrix and a received second sub-matrix.
[0175] Two examples are described below.
[0176] Example 1
[0177] Example 1 is a case where a first matrix is divided into P×Q first sub-matrices, a second matrix is divided into P×Q second sub-matrices, and each node respectively stores a first sub-matrix and a second sub-matrix; in Example 1, P=Q, and the sending and calculation are performed in P periods, in each period, only one row of nodes sends a second sub-matrix, and only one column of nodes sends a first sub-matrix, and after sending in each period, the sub-matrix multiplication calculation of the current period is performed.
[0178] As shown in Figure 4 , in Example 1, the node array is a P=Q=4 node square matrix, each node stores a first sub-matrix X pq and a second sub-matrix W pq , corresponding to a result sub-matrix Y pq , and the 4×4 nodes send the first sub-matrix X pq and the second sub-matrix W pq in 4 periods:
[0179] First period: node E 1,1 sends the first sub-matrix X 11 stored in the node to E 1,2 , E 1,3 , and E 1,4 , node E 2,1 sends the first sub-matrix X 21 stored in the node to E 2,2 , E 2,3 , and E 2,4 , node E 3,1 sends the first sub-matrix X 31 stored in the node to E 3,2 , E 3,3 , and E 3,4 , node E 4,1 sends the first sub-matrix X 41 stored in the node to E 4,2 , E 4,3 , and E 4,4 ; node E 1,1 sends the second sub-matrix W 11 stored in the node to E 2,1 , E 3,1 , and E4,1 Node E 1,2 The second sub-matrix W held by the node is sent to E 12 , E 2,2 , and E 3,2 , respectively. 4,2 Node E 1,3 The second sub-matrix W held by the node is sent to E 13 , E 2,3 , and E 3,3 , respectively. 4,3 Node E 1,4 The second sub-matrix W held by the node is sent to E 14 , E 2,4 , and E 3,4 , respectively. 4,4 .
[0180] Second cycle: Node E 1,2 The first sub-matrix X held by the node is sent to E 12 , E 1,1 , and E 1,3 , respectively. 1,4 Node E 2,2 The first sub-matrix X held by the node is sent to E 22 , E 2,1 , and E 2,3 , respectively. 2,4 Node E 3,2 The first sub-matrix X held by the node is sent to E 32 , E 3,1 , and E 3,3 , respectively. 3,4 Node E 4,2 The first sub-matrix X held by the node is sent to E 42 , E 4,1 , and E 4,3 , respectively. 4,4 Node E 2,1 The second sub-matrix W held by the node is sent to E 21 , E 1,1 , and E 3,1 , respectively. 4,1 Node E 2,2 The second sub-matrix W held by the node is sent to E 22 , E 1,2 , and E 3,2 , respectively. 4,2 Node E 2,3 The second sub-matrix W held by the node is sent to E 23 , E 1,3 , and E 3,3 , respectively. 4,3 Node E 2,4 The second sub-matrix W held by the node is sent to E24 send to E 1,4 , E 3,4 and E 4,4 , respectively.
[0181] Third cycle: node E 1,3 sends the first sub-matrix X 13 saved by the node to E 1,1 , E 1,2 and E 1,4 , respectively, node E 2,3 sends the first sub-matrix X 23 saved by the node to E 2,1 , E 2,2 and E 2,4 , respectively, node E 3,3 sends the first sub-matrix X 33 saved by the node to E 3,1 , E 3,2 and E 3,4 , respectively, node E 4,3 sends the first sub-matrix X 43 saved by the node to E 4,1 , E 4,2 and E 4,4 , respectively; node E 3,1 sends the second sub-matrix W 31 saved by the node to E 1,1 , E 2,1 and E 4,1 , respectively, node E 3,2 sends the second sub-matrix W 32 saved by the node to E 1,2 , E 2,2 and E 4,2 , respectively, node E 3,3 sends the second sub-matrix W 33 saved by the node to E 1,3 , E 2,3 and E 4,3 , respectively, node E 3,4 sends the second sub-matrix W 34 saved by the node to E 1,4 , E 2,4 and E 4,4 , respectively.
[0182] Fourth cycle: node E 1,4 sends the first sub-matrix X 14 saved by the node to E 1,1 , E 1,2 and E 1,3 , respectively, node E 2,4 sends the first sub-matrix X 24 saved by the node to E2,1 , E 2,2 and E 2,3 , node E 3,4 sends the first sub-matrix X 34 saved by the node to E 3,1 , E 3,2 and E 3,3 , respectively 4,4 sends the first sub-matrix X 44 saved by the node to E 4,1 , E 4,2 and E 4,3 , respectively; node E 4,1 sends the second sub-matrix W 41 saved by the node to E 1,1 , E 2,1 and E 3,1 , respectively 3,2 sends the second sub-matrix W 42 saved by the node to E 1,2 , E 2,2 and E 3,2 , respectively 4,3 sends the second sub-matrix W 43 saved by the node to E 1,3 , E 2,3 and E 3,3 , respectively 4,4 sends the second sub-matrix W 44 saved by the node to E 1,4 , E 2,4 and E 3,4 , respectively.
[0183] In one calculation manner of example 1, after the end of the fourth cycle, each node respectively performs steps S210-S230:
[0184] S210: according to the order of q from small to large, the Q first sub-matrices X pq are transversely spliced to obtain a first spliced matrix; for example, for node E 1,1 , the first spliced matrix obtained is X 11 , X 12 , X 13 , X 14 spliced in turn, that is, the first row of the first matrix X; the first spliced matrix obtained by other nodes in the first row of the node array is also the first row of the first matrix X; similarly, the second row of each node in the node array obtains the second row of the first matrix X, the third row of each node obtains the third row of the first matrix X, and the fourth row of each node obtains the fourth row of the first matrix X.
[0185] S220: Arrange the P second submatrices W in ascending order of p. pq Perform vertical concatenation to obtain the second concatenated matrix; for example, for node E 1,1 In other words, what we get is W in sequence. 11 W 21 W 31 W 41 The second concatenated matrix obtained by vertically splicing is the first column of the second matrix W; similarly, the second concatenated matrix obtained by the other nodes in the first column of the node array is also the first column of the second matrix W; each node in the second column of the node array obtains the second column of the second matrix W, each node in the third column obtains the third column of the second matrix W, and each node in the fourth column obtains the fourth column of the second matrix W.
[0186] S230: Perform multiplication on the first concatenated matrix and the second concatenated matrix, that is, each node performs multiplication on the first concatenated matrix and the second concatenated matrix obtained by this node to obtain the result submatrix Y. pq .
[0187] The order of steps S210 and S220 is not important.
[0188] For this calculation method, the above-mentioned method of sending in four cycles can be adopted, or the method of sending in parallel by 4×4 nodes can be adopted, that is, the above four cycles are carried out in the same time period, without any order.
[0189] In Example 1, the first matrix X can be an M-row, L-column matrix, and the second matrix W can be an L-row, N-column matrix; M is an integer multiple of P, and the first submatrix X... pq The number of rows is M / P; N is an integer multiple of Q, and the second submatrix W pq The number of columns is N / Q; L is a common multiple of P and Q, and the first submatrix X pq The number of columns is L / Q, and the second submatrix W pq The number of rows is L / P; after step S210, the number of rows of the first concatenated matrix is M / P, and the number of columns is Q×L / Q=L; after step S220, the number of rows of the second concatenated matrix is P×L / P=L, and the number of columns is N / Q; after step S230, the resulting submatrix Y... pq The number of rows is M / P, and the number of columns is N / Q.
[0190] In Example 1, the resulting submatrix Y with P×Q nodes pqThe result matrix Y is spliced, that is, the number of rows of the result matrix Y is P x M / P=M, and the number of columns is Q x N / Q=N, which is the same as the result matrix Y obtained by the conventional matrix calculation method.
[0191] In Example 1, the result sub-matrix Y is calculated by splicing pq The method is applicable to node matrices of any structure, including rectangular node matrices with P and Q not equal, and node square matrices with P and Q equal; both uniform distribution of sub-matrices in nodes and non-uniform distribution of sub-matrices; both cases where a node includes only one first sub-matrix and one second sub-matrix, and cases where a node includes multiple first sub-matrices and multiple second sub-matrices.
[0192] In another calculation method of Example 1, step S200 can include P times of step S240 and one time of step S250:
[0193] S240: Determine whether a pair of sub-matrices is obtained; when a pair of sub-matrices is obtained, calculate A a =X pq x W pq ; wherein each pair of sub-matrices includes a first sub-matrix X pq and a second sub-matrix W pq , and q in X pq is equal to p in W pq in each pair of sub-matrices;
[0194] S250: Calculate the result sub-matrix Y a from P A wherein a=1, 2, …, P.
[0195] In Example 1, each node only uses a pair of sub-matrices that it has obtained (read from a memory sub-node or received) as the condition for each calculation of A a =X pq x W pq . That is, step S240 and S250 can be performed after each transmission is completed.
[0196] Taking the u-th period as an example, the multiplication calculation of X pq x W pq is performed according to the sub-matrix transmitted by the node in the period;
[0197] The calculation of the result sub-matrix Y pq can be performed according to the multiplication calculation result after each period or after the end of transmission of 4 periods;
[0198] wherein the sub-matrix transmitted in one period includes X pq transmitted and Wpq , or the X pq and the received W pq , or the X pq and the sent W pq , or the X pq and the received W pq .
[0199] In Example 1, the node can set the initial result sub-matrix Y pq to a zero matrix, i.e. after each cycle of multiplication calculation X pq × W pq , the multiplication calculation result is added to the current result sub-matrix Y pq , until the multiplication calculation result of the Pth cycle is added to the result sub-matrix obtained in the (P-1)th cycle, to obtain the result sub-matrix Y pq . Alternatively, after the node performs the multiplication calculation of P cycles, the results of the P cycles of multiplication calculation are added to obtain the result sub-matrix Y pq .
[0200] In Example 1, the corresponding matrix multiplication calculation can be performed after each cycle of sending, i.e. each node performs the calculation in 4 cycles, including:
[0201] The calculation of the first cycle: performed after the sending in the first cycle, the node E 1,1 calculates X 11 × W 11 , the node E 1,2 calculates X 11 × W 12 , the node E 1,3 calculates X 11 × W 13 , the node E 1,4 calculates X 11 × W 14 ; the node E 2,1 calculates X 21 × W 11 , the node E 2,2 calculates X 21 × W 12 , the node E 2,3 calculates X 21 × W 13 , the node E 2,4 calculates X 21 × W 14 ; the node E 3,1 calculates X 31 × W 11 , the node E 3,2 calculates X 31 × W 12 , the node E3,3 Compute X 31 x W 13 , Node E 3,4 Compute X 31 x W 14 ; Node E 4,1 Compute X 41 x W 11 , Node E 4,2 Compute X 41 x W 12 , Node E 4,3 Compute X 41 x W 13 , Node E 4,4 Compute X 41 x W 14 .
[0202] Second cycle computation: after second cycle transmission, Node E 1,1 Compute X 12 x W 21 , Node E 1,2 Compute X 12 x W 22 , Node E 1,3 Compute X 12 x W 23 , Node E 1,4 Compute X 12 x W 24 ; Node E 2,1 Compute X 22 x W 21 , Node E 2,2 Compute X 22 x W 22 , Node E 2,3 Compute X 22 x W 23 , Node E 2,4 Compute X 22 x W 24 ; Node E 3,1 Compute X 32 x W 21 , Node E 3,2 Compute X 32 x W 22 , Node E 3,3 Compute X 32 x W 23 , Node E 3,4 Compute X 32 x W 24 ; Node E 4,1 Compute X 42 x W 21 , Node E 4,2 Compute X 42 x W 22, node E 4,3 Compute X 42 x W 23 , node E 4,4 Compute X 42 x W 24 .
[0203] Third cycle computation: performed after the third cycle transmission, node E 1,1 Compute X 13 x W 31 , node E 1,2 Compute X 13 x W 32 , node E 1,3 Compute X 13 x W 33 , node E 1,4 Compute X 13 x W 34 ; node E 2,1 Compute X 23 x W 31 , node E 2,2 Compute X 23 x W 32 , node E 2,3 Compute X 23 x W 33 , node E 2,4 Compute X 23 x W 34 ; node E 3,1 Compute X 33 x W 31 , node E 3,2 Compute X 33 x W 32 , node E 3,3 Compute X 33 x W 33 , node E 3,4 Compute X 33 x W 34 ; node E 4,1 Compute X 43 x W 31 , node E 4,2 Compute X 43 x W 32 , node E 4,3 Compute X 43 x W 33 , node E 4,4 Compute X 43 x W 34 .
[0204] Fourth cycle computation: performed after the fourth cycle transmission, node E 1,1 Compute X 14 x W41 Node E 1,2 Calculate X 14 ×W 42 Node E 1,3 Calculate X 14 ×W 43 Node E 1,4 Calculate X 14 ×W 44 Node E 2,1 Calculate X 24 ×W 41 Node E 2,2 Calculate X 24 ×W 42 Node E 2,3 Calculate X 24 ×W 43 Node E 2,4 Calculate X 24 ×W 44 Node E 3,1 Calculate X 34 ×W 41 Node E 3,2 Calculate X 34 ×W 42 Node E 3,3 Calculate X 34 ×W 43 Node E 3,4 Calculate X 34 ×W 44 Node E 4,1 Calculate X 44 ×W 41 Node E 4,2 Calculate X 44 ×W 42 Node E 4,3 Calculate X 44 ×W 43 Node E 4,4 Calculate X 44 ×W 44 .
[0205] Each node obtains the result sub-matrix corresponding to the node by the way of accumulating the multiplication result into the result sub-matrix after each period calculation, or by the way of adding all the period multiplication results after the four period calculations.
[0206] Example 2
[0207] In example 2, P=Q, all nodes transmit every time, but only part of the first sub-matrices saved by the node and part of the second sub-matrices saved by the node are transmitted every time, and multiplication calculation is performed on the transmitted sub-matrices after each transmission.
[0208] by Figure 7 Taking a 4×4 node matrix as an example, the first matrix X is divided into 8×8 first submatrices, and the second matrix W is divided into 8×8 second submatrices; each node's memory child node 40 stores 2×2 first submatrices and 2×2 second submatrices; one matrix multiplication is completed in the following nine steps:
[0209] Step 1, Node E p,q Send the saved X (2p-1)(2q-1) Send the saved W to other nodes in the same industry. (2p-1)(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 11 Send W to other nodes in the same industry. 11 Give it to other nodes in the same column, such as E. 3,2 Send X 53 Send W to other nodes in the same industry. 53 Send to other nodes in the same column. That is, each node sends the first row and first column of the first 2×2 first submatrix and the first row and first column of the second 2×2 submatrix stored in this node.
[0210] The submatrix to be sent and the submatrix to be received are stored in the temporary storage module 31 of each node. After the first step of sending, the submatrix stored in the temporary storage module 31 of each node is as follows: Figure 7 As shown in the image.
[0211] In the second step, each node processes the four first sub-matrices and four second sub-matrices transmitted in the first step (i.e., ... Figure 7 The submatrices stored in the temporary storage module 31 are subjected to matrix multiplication pairwise, and the results are accumulated into the resulting submatrix Y. (2p-1)(2q-1) In the middle; for example, E 1,1 Calculate X 11 ×W 11 X 13 ×W 31 X 15 ×W 51 X 17 ×W 71 The result submatrix Y is then added to the temporary storage module 31. 11 middle.
[0212] As can be seen, the first submatrix used for calculations of nodes in the same row is the same, and the second submatrix used for calculations of nodes in the same column is the same.
[0213] During the calculation, node E p,q Send the saved X (2p-1)2q Send the saved W to other nodes in the same industry. 2p(2q-1) Give it to other nodes in the same column; for example, E.1,1 Send X 12 to other nodes in the same row, send W 21 to other nodes in the same column; for example, E 2,4 sends X 38 to other nodes in the same row, sends W 47 to other nodes in the same column. That is, each node sends the first row second column of the 2x2 first sub-matrix and the second row first column of the 2x2 second sub-matrix saved in the node.
[0214] Third step, each node respectively performs matrix multiplication calculation on the four first sub-matrices and the four second sub-matrices transmitted in the second step in pairs, and accumulates to the result sub-matrix Y (2p-1)(2q-1) ; for example, E 1,1 calculates X 12 ×W 21 , X 14 ×W 41 , X 16 ×W 61 , X 18 ×W 81 , and accumulates to the result sub-matrix Y 11 in the temporary storage module 31. After the accumulation is completed, the result sub-matrix Y 11 is saved to the memory sub-node 40. It can be seen that after the calculation of the second step and the third step, the result sub-matrix Y 1,1 in E 11 is equivalent to the multiplication result of the first row in the first matrix X and the first column in the second matrix W.
[0215] In the calculation process, the node E p,q sends the saved X (2p-1)(2q-1) to other nodes in the same row, and sends the saved W (2p-1)2q to other nodes in the same column; for example, E 1,1 sends X 11 to other nodes in the same row, and sends W 12 to other nodes in the same column; for example, E 2,3 sends X 35 to other nodes in the same row, and sends W 36 to other nodes in the same column. That is, each node sends the first row first column of the 2x2 first sub-matrix and the first row second column of the 2x2 second sub-matrix saved in the node.
[0216] Fourth step, each node respectively performs matrix multiplication calculation on the four first sub-matrices and the four second sub-matrices transmitted in the third step in pairs, and accumulates to the result sub-matrix Y (2p-1)2q ; for example, E 1,1 calculates X 11 ×W 12 , X13 ×W 32 X 15 ×W 52 X 17 ×W 72 The result submatrix Y is then added to the temporary storage module 31. 12 middle.
[0217] During the calculation, node E p,q Send the saved X (2p-1)2q Send the saved W to other nodes in the same node group. 2p2q Give it to other nodes in the same column; for example, E. 1,1 Send X 12 Send W to other nodes in the same industry. 22 Give it to other nodes in the same column; for example, E. 2,3 Send X 36 Send W to other nodes in the same industry 46 Send to other nodes in the same column. That is, each node sends the first row and second column of the first 2×2 first submatrix and the second row and second column of the second 2×2 second submatrix stored by the node.
[0218] In the fifth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the fourth step, and accumulates the results into the resulting sub-matrix Y. (2p-1)2q In the middle; for example, E 1,1 Calculate X 12 ×W 22 X 14 ×W 42 X 16 ×W 62 X 18 ×W 82 The result submatrix Y is then added to the temporary storage module 31. 12 In the middle; after the accumulation is complete, the resulting submatrix Y is... 12 Saved to memory child node 40. It can be seen that after the calculations in steps four and five, E... 1,1 The resulting submatrix Y 12 It is equivalent to the result of multiplying the first row of the first matrix X and the second column of the second matrix W.
[0219] During the calculation, node E p,q Send the saved X 2p(2q-1) Send the saved W to other nodes in the same node group. (2p-1)(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 21 Send W to other nodes in the same industry 11 Give it to other nodes in the same column; for example, E. 4,1 Send X 81Send W to other nodes in the same industry 71 Send to other nodes in the same column. That is, each node sends the second row and first column of the 2×2 first submatrix stored by the node, and the first row and first column of the 2×2 second submatrix.
[0220] In the sixth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the fifth step, and accumulates the results into the resulting sub-matrix Y. 2p(2q-1) In the middle; for example, E 1,1 Calculate X 21 ×W 11 X 23 ×W 31 X 25 ×W 51 X 27 ×W 71 The result submatrix Y is then added to the temporary storage module 31. 21 middle.
[0221] During the calculation, node E p,q Send the saved X 2p2q Send the saved W to other nodes in the same node group. 2p(2q-1) Give it to other nodes in the same column; for example, E. 1,1 Send X 22 Send W to other nodes in the same industry 21 Give it to other nodes in the same column; for example, E. 3,3 Send X 66 Send W to other nodes in the same industry 65 Send to other nodes in the same column. That is, each node sends the second row and second column of the 2×2 first submatrix stored in its own memory, and the second row and first column of the 2×2 second submatrix.
[0222] In the seventh step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the sixth step, and accumulates the results into the sub-matrix Y. 2p(2q-1) In the middle; for example, E 1,1 Calculate X 22 ×W 21 X 24 ×W 41 X 26 ×W 61 X 28 ×W 81 The result submatrix Y is then added to the temporary storage module 31. 21 In the middle; after the accumulation is complete, the resulting submatrix Y is... 21 Saved to memory child node 40. It can be seen that after the calculations in steps six and seven, E... 1,1 The resulting submatrix Y 21It is equivalent to the result of multiplying the second row of the first sub-matrix array divided by the first matrix X and the first column of the second sub-matrix array divided by the second matrix W.
[0223] During the calculation, node E p,q Send the saved X 2p(2q-1) Send the saved W to other nodes in the same node group. (2p-1)2q Give it to other nodes in the same column; for example, E. 1,1 Send X 21 Send W to other nodes in the same industry 12 To other nodes in the same column, each node sends the second row and first column of the 2×2 first submatrix and the first row and second column of the 2×2 second submatrix stored in its own data.
[0224] In the eighth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the seventh step, and accumulates the results into the sub-matrix Y. 2p2q In the middle; for example, E 1,1 Calculate X 21 ×W 12 X 23 ×W 32 X 25 ×W 52 X 27 ×W 72 The result submatrix Y is then added to the temporary storage module 31. 22 middle.
[0225] During the calculation, node E p,q Send the saved X 2p2q Send the saved W to other nodes in the same node group. 2p2q Give it to other nodes in the same column; for example, E. 1,1 Send X 22 Send W to other nodes in the same industry 22 To other nodes in the same column, each node sends the second row and second column of the 2×2 first submatrix and the second row and second column of the 2×2 second submatrix stored in its own memory.
[0226] In the ninth step, each node performs matrix multiplication on each pair of the four first sub-matrices and four second sub-matrices transmitted in the eighth step, and accumulates the results into the sub-matrix Y. 2p2q In the middle; for example, E 1,1 Calculate X 22 ×W 22 X 24 ×W 42 X 26 ×W 62 X 28 ×W 82and accumulated into the result sub-matrix Y in the temporary module 31 22 and accumulated into the result sub-matrix Y in the temporary module 31 22 saved into the memory sub-node 40. It can be seen that after the calculation of the eighth step and the ninth step, the result sub-matrix Y 1,1 saved into the memory sub-node 40. It can be seen that after the calculation of the eighth step and the ninth step, the result sub-matrix Y 22 is the multiplication result of the second row in the first matrix X and the second column in the second matrix W.
[0227] After the above nine steps, the matrix multiplication calculation of the first matrix X and the second matrix W can be completed, and each node obtains 2x2 result sub-matrices (which can be regarded as a result sub-array), and the result sub-arrays are spliced according to the positions of each node in the node array to obtain the matrix multiplication result matrix Y of the first matrix X and the second matrix W. That is, the result matrix Y can be regarded as an array composed of 8x8 result sub-matrices or 4x4 result sub-arrays, the first row is Y 11 , Y 12 , Y 13 , Y 14 , Y 15 , Y 16 , Y 17 , Y 18 ; the last row is Y 81 , Y 82 , Y 83 , Y 84 , Y 85 , Y 86 , Y 87 , Y 88 , and the other rows can be similarly obtained.
[0228] In example 2, when each node performs multiplication calculation, the reading and sending of the sub-matrix required for the next step can be performed in parallel, and all nodes perform sending and matrix multiplication in parallel, so that the efficiency of matrix multiplication can be greatly improved.
[0229] In example 2, the sending process of the above first step to the eighth step is not limited to the above order, and the order can be arbitrarily changed. Correspondingly, each step is to calculate the multiplication of the first sub-matrix and the second sub-matrix transmitted (including sending and receiving) in the previous step, and to accumulate into the corresponding result sub-matrix. For example, if the first sub-matrix of the rth row and the s th column of each node and the second sub-matrix of the s th row and the t th column are transmitted, then the result sub-matrix of the rth row and the t th column of the node is accumulated. When each step performs multiplication calculation, the first sub-matrix used by the nodes in the same row for calculation is the same, and the second sub-matrix used by the nodes in the same column for calculation is the same.
[0230] In another aspect, the embodiment of the present application further provides a neural network calculation method, and the method comprises:
[0231] Each layer in the neural network model performs steps S300-S500, as shown in Figure 8 :
[0232] S300: Taking the input matrix of the current layer as the first matrix X and the weight matrix of the current layer as the second matrix W, performing matrix multiplication calculation Y = X x W according to the matrix multiplication calculation method described in the above embodiment;
[0233] S400: Calculating the result matrix Y of the current layer according to Y * ;
[0234] S500: Inputting Y * to the next layer of the neural network model as the first matrix X of the next layer, or inputting Y * as the calculation result of the neural network.
[0235] In this embodiment, the neural network model can be a multi-layer deep learning model, which can be any one of a Transformer model, Linear-Time Sequence Modeling with Selective State Space, TTT (Test-Time Training), LSTM (Long Short-Term Memory), VGGNet (Deep Convolutional Neural Network), etc. The above deep learning models are exemplary descriptions and are not limited by the present application. This paper will not be described one by one.
[0236] In this embodiment, each layer calculation of the neural network model can be performed by the data processing architecture described in the above embodiment, and each node of the node array collectively performs multi-layer calculation of the neural network model.
[0237] In this embodiment, before performing the neural network calculation method, the weight matrix of each layer of the neural network model can be divided into P x Q second sub-matrices according to the structure of the node array (i.e., P rows and Q columns), and each second sub-matrix of each layer is saved in the memory sub-node of the node array; when calculating a certain layer of the neural network, the calculation sub-node can read the second sub-matrix divided from the weight matrix of the corresponding layer from the memory sub-node for sending.
[0238] In this embodiment, the input matrix of the neural network model is taken as the first matrix X of the first layer of the neural network model, and before performing the neural network calculation method, the first matrix X can be divided into P x Q first sub-matrices according to the structure of the node array, and saved in the memory sub-node or the temporary storage module of the node array.
[0239] In the embodiment, when step S400 is performed, different neural network models have different calculations of the result matrix Y * of the current layer, for example, Y is directly set as the result matrix Y * of the current layer, or Y is added with a bias matrix (BiasMatrix) to obtain the result matrix Y * of the current layer; the result matrix Y * of each layer of the neural network model is obtained by splicing the result sub-matrices in the P*Q nodes.
[0240] In the embodiment, taking the neural network model of the Fth layer as an example, the result sub-matrices constituting Y * calculated by the 1st layer are respectively stored in the P*Q memory sub-nodes or the P*Q temporary storage modules of the node array, as the first sub-matrices of the 2nd layer; the result sub-matrices constituting Y * calculated by the 2nd layer are respectively stored in the P*Q memory sub-nodes or the P*Q temporary storage modules of the node array, as the first sub-matrices of the 3rd layer; the P*Q sub-matrices constituting Y * calculated by the Fth layer are respectively stored in the P*Q memory sub-nodes or the P*Q temporary storage modules of the node array, as the output matrix of the neural network model.
[0241] In the neural network calculation method of the embodiment, the result matrix Y p,q calculated by each node E * in the neural network calculation of the current layer can be used as the input matrix of the neural network calculation of the next layer, and the result matrix Y * is stored in the node, without input and output operations, thereby improving the calculation efficiency of the deep neural network calculation method, and dividing the large matrix multiplication calculation of each layer of the neural network into parallel small matrix multiplication calculations, thereby improving the deep neural network calculation efficiency and providing an efficient solution for large-scale deep learning calculation.
[0242] In another aspect, the embodiment of the application further provides a chip, including the data processing architecture provided in any of the above embodiments.
[0243] In an example embodiment, the chip includes:
[0244] The memory die includes P*Q memory sub-nodes; the P*Q memory sub-nodes respectively store the first sub-matrices divided from the first matrix X to be subjected to matrix multiplication and the second sub-matrices divided from the second matrix W to be subjected to matrix multiplication;
[0245] The logic die comprises P×Q computing sub-nodes corresponding to the P×Q memory sub-nodes; each computing sub-node is configured to read a first sub-matrix from the corresponding memory sub-node, respectively send to other Q-1 computing sub-nodes in the same row, read a saved second sub-matrix from the corresponding memory sub-node, respectively send to other P-1 computing sub-nodes in the same column, and calculate a corresponding result sub-matrix according to the saved sub-matrix and the received sub-matrix in the corresponding memory sub-node, and save the result sub-matrix in the corresponding memory sub-node; wherein the sub-matrix comprises the first sub-matrix and the second sub-matrix; and the result sub-matrices saved in the P×Q memory sub-nodes form a result matrix.
[0246] The memory die is stacked with the logic die and is bonded.
[0247] In the embodiment, the result sub-matrices saved in the P×Q memory sub-nodes can be directly spliced according to the positions (i.e. the row and the column) of the memory sub-nodes in the memory die 60, and the final result matrix Y of the matrix multiplication of the first matrix X and the second matrix W can be obtained; for example, it is assumed that each memory sub-node saves one result sub-matrix Y pq , and the result sub-matrices Y pq in the P×Q memory sub-nodes can be spliced to obtain Y[M][N], i.e. Y[M][N] can be regarded as a result sub-matrix array, wherein the first row is Y 11 , Y 12 , …, Y 1Q ; the second row is Y 21 , Y 22 , …, Y 2Q ; the last row is Y P1 , Y P2 , …, Y PQ ; and the like. In the case that each memory sub-node saves multiple result sub-matrices, the multiple result sub-matrices are a result sub-array in the node, and the result sub-arrays saved in the multiple memory sub-nodes are spliced according to the positions of the memory sub-nodes in the memory array, and Y[M][N] can be obtained.
[0248] In an exemplary embodiment of the embodiment, as shown in Figure 9 and Figure 10 , the memory sub-nodes in the memory die 60 and the computing sub-nodes in the logic die 50 can be connected one by one in a vertical connection manner. In this way, the size of the data processing architecture can be reduced, the space optimization is realized, the wiring complexity is reduced, the electromagnetic interference between different nodes is reduced, the maintenance and expansion and upgrading are facilitated, and the standardized design is easy to realize.
[0249] In other embodiments, the memory sub-nodes and the computing sub-nodes can be located on the same plane and connected by circuitry; the memory dies and the logic dies can be combined into one die in physical structure, such as using ReRam technology to integrate P x Q memory sub-nodes in one-to-one correspondence to P x Q computing sub-nodes.
[0250] Those of ordinary skill in the art will appreciate that all or certain steps in the methods disclosed above and the functional modules / units in the systems and apparatuses can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units referred to in the above description can not necessarily correspond to the division between physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components working together. Certain components or all components can be implemented as software executed by a processor, such as a digital signal processor or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Further, it is common in the art for a person of ordinary skill to appreciate that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and can include any information delivery media.
Claims
1. A data processing architecture, comprising: A node array with P rows and Q columns; The P×Q nodes in the node array respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied. The node is configured to send the first submatrix stored by the node to the other Q-1 nodes in the same row as the node, send the second submatrix stored by the node to the other P-1 nodes in the same column as the node, and calculate the result submatrix corresponding to the node based on the submatrix stored and received by the node; wherein, the submatrix includes the first submatrix and the second submatrix; the result submatrixes of the P×Q nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W.
2. The data processing architecture as described in claim 1, wherein, Each of the nodes includes: a computing sub-node and a storage module; The storage module is configured to store the first submatrix, the second submatrix, and the result submatrix; The computational sub-nodes include: The processing module is configured to read the first submatrix and the second submatrix from the storage module; calculate the result submatrix corresponding to the current node based on the submatrix stored and received at the current node, and save it to the storage module; The row data communication module is configured to send the first sub-matrix read by the processing module to the other Q-1 nodes in the same row as this node, and to receive the first sub-matrix from the other Q-1 nodes in the same row. The column data communication module is configured to send the second sub-matrix read by the processing module to the other P-1 nodes in the same column as this node, and to receive the second sub-matrix from the other P-1 nodes in the same column.
3. The data processing architecture as described in claim 2, wherein, The storage module includes: a temporary storage module disposed in the computation sub-node, configured to store the first sub-matrix, the second sub-matrix, and the result sub-matrix; Alternatively, it may include: a temporary storage module set in the computing sub-node, and a memory sub-node connected to the computing sub-node; The memory sub-node is configured to store the first sub-matrix, the second sub-matrix, and the result sub-matrix; The temporary storage module is configured to temporarily store the first sub-matrix and the second sub-matrix read by the processing module from the memory sub-node, as well as the result sub-matrix calculated by the processing module; The processing module is further configured to save the result submatrix temporarily stored in the temporary storage module to the memory subnode.
4. The data processing architecture as described in claim 1, wherein: The nodes in different columns send the first sub-matrix at different time periods, and the nodes in the same column send the first sub-matrix in parallel; the nodes in different rows send the second sub-matrix at different time periods, and the nodes in the same row send the second sub-matrix in parallel. Alternatively, the P×Q nodes may transmit the first submatrix and the second submatrix in parallel.
5. The data processing architecture as described in any one of claims 1 to 4, wherein: The node obtains its corresponding result submatrix based on the submatrix stored and received by the node, including: After receiving all the first sub-matrices stored by Q-1 other nodes in the same row and all the second sub-matrices stored by P-1 other nodes in the same column, the node concatenates the first sub-matrices stored by itself and the received first sub-matrices according to the order of the first sub-matrices in the first matrix to obtain a first concatenated matrix. Then, it concatenates the second sub-matrices stored by itself and the received second sub-matrices according to the order of the second sub-matrices in the second matrix to obtain a second concatenated matrix. Finally, it performs matrix multiplication on the first concatenated matrix and the second concatenated matrix to obtain the result sub-matrice corresponding to the node.
6. The data processing architecture as described in any one of claims 1 to 4, wherein: The first matrix is divided into R rows and S columns of first submatrices X. ij The second matrix is divided into S rows and T columns of second submatrices W. hk ; i represents the first submatrix X ij The index of the row in the first matrix, j, represents the first submatrix X. ij The column number in the first matrix; h represents the second submatrix W. hk The row number k in the second matrix represents the second submatrix W. hk The column index in the second matrix; i = 1, 2, ..., R, j = 1, 2, ..., S, h = 1, 2, ..., S, k = 1, 2, ..., T; The node obtains its corresponding result submatrix based on the submatrix stored and received by the node, including: The node, in the stored and received sub-matrices, sets a first sub-matrix X in which j and h are equal. ij and a second submatrix W hk As a pair of submatrices; Perform matrix multiplication on each pair of submatrices obtained, and sum the matrix multiplication results corresponding to the same result submatrix.
7. The data processing architecture as described in claim 1, wherein: Each node E p,q Store a first submatrix X respectively pq and a second submatrix W pq Where p represents the p-th row in the node array, q represents the q-th column in the node array, p = 1, 2, ..., P, q = 1, 2, ..., Q; When P = Q, the P × Q nodes E p,q The first sub-matrix X is sent in P periods. pq Second submatrix W pq The node E p,q The first submatrix X stored in this node pq Send the second submatrix W stored in this node to the other Q-1 nodes in the same row as this node. pq Send to the other P-1 nodes in the same column as this node, including: node E p,q In the q-th period, store the first submatrix X at this node. pq Send the data to the other Q-1 nodes in the same row as this node; in the p-th cycle, send the second submatrix W stored in this node. pq Send them to the other P-1 nodes in the same column as this node.
8. The data processing architecture as described in claim 7, wherein: The node calculates the corresponding result submatrix based on the submatrix stored and received by the node, including: For each of the P cycles, the node transmits X in that cycle. pq and W pq Perform matrix multiplication calculations; accumulate the results of P periodic multiplication calculations to obtain the result submatrix corresponding to this node; Among them, X is transmitted in one cycle. pq and W pq Includes: X sent pq and the W sent pq , or the X sent pq and received W pq , or the received X pq and the W sent pq , or the received X pq and received W pq .
9. The data processing architecture as described in any one of claims 1 to 4, wherein, Each node stores I×J first submatrices and H×K second submatrices, where the number of columns in the first submatrices is equal to the number of rows in the second submatrices, and J×Q=H×P; each node corresponds to I×K result submatrices. The node sends its first submatrix to the other Q-1 nodes in the same row as the node, and sends its second submatrix to the other P-1 nodes in the same column as the node, including: The node sends the multiple first sub-matrixes and second sub-matrixes stored in the node in multiple batches. Each time, at least one first sub-matrix is sent to the other Q-1 nodes in the same row as the node, and at least one second sub-matrix is sent to the other P-1 nodes in the same column as the node. The submatrix corresponding to this node is calculated based on the submatrix stored in this node and the submatrix received, including: The node performs matrix multiplication on the first and second sub-matrices transmitted each time, and accumulates the matrix multiplication results into the corresponding result sub-matrix in the node. The first and second sub-matrixes in a single transmission include: the first sub-matrix to be transmitted and the second sub-matrix to be transmitted, or the first sub-matrix to be transmitted and the second sub-matrix to be received, or the first sub-matrix to be received and the second sub-matrix to be transmitted, or the first sub-matrix to be received and the second sub-matrix to be received.
10. A matrix multiplication calculation method, said method being implemented based on the data processing architecture as described in any one of claims 1-9, comprising: Each node in the data processing architecture sends the first sub-matrix stored in this node to the other Q-1 nodes in the same row as this node, and sends the second sub-matrix stored in this node to the other P-1 nodes in the same column as this node; Each node calculates its corresponding result submatrix based on the submatrix stored in and received by the node. The submatrix includes a first submatrix and a second submatrix; the result submatrixes of P×Q nodes form the first matrix X and the result matrix Y of matrix multiplication of the second matrix W; the first submatrix stored by P×Q nodes forms the first matrix X; and the second submatrix stored by P×Q nodes forms the second matrix W.
11. A neural network computation method, the method comprising: Each layer in the neural network model performs the following steps: Using the input matrix of the current layer as the first matrix X and the weight matrix of the current layer as the second matrix W, perform matrix multiplication Y = X × W according to the matrix multiplication calculation method described in claim 10. Calculate the result matrix Y of the current layer based on Y. * ; Y * The input is fed into the next layer of the neural network model as the first matrix X of the next layer, or Y is used as the input. * As the result of neural network calculations.
12. A chip, comprising: The data processing architecture as described in any one of claims 1 to 9.
13. The chip as claimed in claim 12, wherein, The chip includes: The storage die includes P×Q memory sub-nodes; the P×Q memory sub-nodes respectively store the first sub-matrix obtained by partitioning the first matrix X to be multiplied, and the second sub-matrix obtained by partitioning the second matrix W to be multiplied; The logic die includes P×Q computational sub-nodes, each corresponding one-to-one with one of the P×Q memory sub-nodes. Each computational sub-node is configured to read a first sub-matrix from its corresponding memory sub-node and send it to the other Q-1 computational sub-nodes in the same row; read a second sub-matrix from its corresponding memory sub-node and send it to the other P-1 computational sub-nodes in the same column; and calculate the corresponding result sub-matrix based on the sub-matrix stored in the corresponding memory sub-node and the received sub-matrix, and save it in the corresponding memory sub-node. The sub-matrix includes a first sub-matrix and a second sub-matrix. The result sub-matrixes stored in the P×Q memory sub-nodes form the result matrix Y of matrix multiplication of the first matrix X and the second matrix W. The storage die and the logic die are stacked and bonded together.
