Co-directional systolic array circuit and data processing device based on distributed memory

Abstract

The present invention relates to a distributed memory-based, unidirectional systolic array circuit and data processing device, comprising: at least one systolic array unit, each unit comprising nodes from the first to the Nth rows and nodes from the first to the Mth columns, each node comprising an upper row register, a left row register, a right row register, a distributed memory, a calculation unit, and a cumulative sum register, wherein the upper row register is used to receive data of a first input matrix or first data and transmit the first data, the left row register is used to receive first or second data and transmit second data, the right row register is used to receive second or third data and transmit third data, the distributed memory is used to store and transmit data of the second input matrix, the calculation unit is used to receive data of the second input matrix, third and fourth data, and transmit fifth data after calculation, and the cumulative sum register is used to receive fifth data and transmit fourth data. Thus, operation speed is accelerated and symmetry is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuit design, and in particular to a distributed memory-based unidirectional systolic array circuit and a data processing device. Background Art

[0002] In matrix multiplication, the number of rows (columns) in one matrix is ​​equal to the number of columns (rows) in another matrix. A dot product is performed on each column (row) of the first matrix with each row (column) of the second matrix to produce a new matrix. In integrated circuit design, matrix multiplication can be implemented using circuits, typically serially executed by a central processing unit (CPU). However, this array circuit is slow, necessitating optimized design.

[0003] In related technologies, parallel computing is usually performed using methods such as systolic arrays based on application-specific integrated circuits (ASICs) such as tensor processing units (TPUs) to speed up computing.

[0004] However, the above technical means will increase the asymmetry of the systolic array and the routing cost when the data of another input matrix and the data of the output matrix are input and output from different directions, which needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a distributed memory-based co-directional systolic array circuit and device to address the problems of low symmetry and high routing cost when using TPUs and traditional systolic arrays to implement matrix multiplication operations in related technologies. This accelerates the operation speed of matrix multiplication, improves the symmetry of the systolic array, and reduces the routing cost.

[0006] To achieve the above objectives, a first embodiment of the present invention provides a distributed memory-based systolic array circuit, comprising:

[0007] At least one systolic array unit, each systolic array unit includes nodes from the first to the Nth rows and nodes from the first to the Mth columns, where N and M are both integers, N≤M, and each node includes: an upstream register, a left row register, a right row register, a distributed memory, a computing unit, and an accumulated sum register, wherein:

[0008] The upstream register is used to receive data of the first input matrix and send the first data to the upstream register of the node above the current node, wherein when the current node is the K-th row node and the current node is the N-K+1-th column node, the first data is simultaneously sent to the left row register of the current node, K≤N;

[0009] The left row register is used to receive second data sent by the left row register of the node to the right of the current node, and send the second data to the left row register of the node to the left of the current node, and when the current node is in the first column, send the second data to the right row register of the current node at the same time;

[0010] The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the node to the right of the current node and the computing unit of the current node;

[0011] The distributed memory is used to store data of a second input matrix and send the data of the second input matrix to the computing unit of the current node;

[0012] a computing unit, configured to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node, and the fourth data of the cumulative sum register of the node above the current node, and send the computing result as the fifth data to the cumulative sum register of the current node;

[0013] The cumulative sum register is used to receive a calculation result of the calculation unit of the current node, wherein when the current node is a first-row node, the calculation result of the calculation unit of the current node is sent to a unit below the current systolic array as data of an output matrix.

[0014] According to an embodiment of the present invention, if N is greater than or equal to 4, the upstream register does not exist at the node in the N+1th to Mth columns where the current node is.

[0015] According to one embodiment of the present invention, the calculation unit includes:

[0016] a multiplier, configured to multiply the data in the right row register of the current node and the data in the distributed memory of the current node;

[0017] An adder is used to add the product obtained by the multiplier and the data in the cumulative sum register of the node above the current node.

[0018] According to an embodiment of the present invention, the input direction and the output direction of each systolic array unit are the same.

[0019] According to an embodiment of the present invention, the distributed memory is at least one of a dynamic random access memory, a static random access memory, a register or other memories.

[0020] According to an embodiment of the present invention, a distributed memory-based, co-directional systolic array circuit is proposed. By arranging distributed memory within each systolic array, when input matrix data enters the systolic array from the bottom, computation starts at the left node. Upon reaching a diagonal line of the systolic array, the input data turns leftward until it reaches the leftmost node, where computation restarts. The computation then propagates rightward node by node until it reaches the rightmost node. All nodes passed through participate in the matrix multiplication computation, and the accumulated sum is accumulated in the opposite direction of the input data entering the array and in the same direction as the output data leaving the array. Thus, by employing parallel computing and a co-directional input-output design approach, the matrix multiplication operation speed is accelerated, the symmetry of the systolic array is improved, and routing costs are reduced.

[0021] To achieve the above-mentioned object, another embodiment of the present invention provides a data processing device, comprising the above-mentioned distributed memory-based co-directional systolic array circuit.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments with reference to the accompanying drawings, in which:

[0024] Figure 1 A schematic structural diagram of a distributed memory-based unidirectional systolic array circuit according to an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a distributed memory-based co-directional systolic array data flow according to a specific embodiment of the present invention;

[0026] Figure 3 The figure is a schematic diagram of the structure of a computing unit in a systolic array node according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present invention, but are not to be construed as limiting the present invention.

[0028] The following describes a distributed memory-based systolic array circuit and a data processing device according to an embodiment of the present invention with reference to the accompanying drawings.

[0029] Figure 1 The diagram is a structural diagram of a distributed memory-based unidirectional systolic array circuit according to an embodiment of the present invention.

[0030] like Figure 1 As shown, the distributed memory-based unidirectional systolic array circuit includes:

[0031] At least one systolic array unit, each systolic array unit includes nodes from the first to the Nth row, nodes from the first to the Mth column, N and M are both integers, N≤M, each node includes: an upstream register, a left row register, a right row register, a distributed memory, a computing unit and a cumulative sum register, the upstream register is used to receive data of a first input matrix, and send the first data to the upstream register of the node above the current node, wherein when the current node is the Kth row node and the current node is the N-K+1th column node, the first data is simultaneously sent to the left row register of the current node, K≤N; the left row register is used to receive second data sent by the left row register of the node to the right of the current node, and send the second data to the left row register of the node to the left of the current node, and when the current node is the first column, the second data is simultaneously sent to the right row register of the current node. The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the node to the right of the current node and the calculation unit of the current node; the distributed memory is used to store the data of a second input matrix, and send the data of the second input matrix to the calculation unit of the current node; the calculation unit is used to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node and the fourth data of the cumulative sum register of the node above the current node, and send the calculation result as the fifth data to the cumulative sum register of the current node; the cumulative sum register is used to receive the calculation result of the calculation unit of the current node, wherein, when the current node is the first row node, the calculation result of the calculation unit of the current node is sent to the unit below the current systolic array as the data of the output matrix.

[0032] Among them, the data of the first input matrix is ​​the data received by the upstream register of the first row node, the data of the second input matrix is ​​the data sent by the memory, the first data is the data sent by the upstream register, the second data is the data sent by the left row register, the third data is the data sent by the right row register, the fourth data is the data sent by the cumulative sum register, the fifth data is the data sent by the calculation unit, and the data of the output matrix is ​​the data sent by the cumulative sum register of the first row node.

[0033] Specifically, if Figure 2 As shown, Figure 2 This diagram illustrates data flow in a distributed memory-based, co-directional systolic array according to a specific embodiment of the present invention. Input matrix data enters the systolic array from below but must reach a node perpendicular to the direction of entry to initiate computation. In this example, computation begins at the upper left node, but computation can be extended to the right. Input data entering from below turns left upon reaching a diagonal of the systolic array, initiating computation upon reaching a node on the left. It then propagates rightward, node by node, until it reaches a node on the right, participating in computations in each node it passes through. The direction of cumulative summation is opposite to the direction the input data enters the array and in the same direction as the output data leaves the array. In this embodiment of the present invention, the cumulative sum propagates downward.

[0034] For example, Figure 1 When the row and column size of the systolic array reaches 4×4 or larger, the systolic array circuit has 11 types of nodes, including:

[0035] The first type of node is the lower left corner node, such as node (1,1). The upper register receives the data of the input matrix and sends the data to the upper register of the node above. The left register receives the data of the left register of the right node and sends the data to the right register of the current node. The right register receives the data of the left register of the current node and sends the data to the right register of the right node and the computation unit of the current node. The distributed memory sends the data to the computation unit of the current node. The computation unit receives the data of the right register of the current node, the distributed memory of the current node, and the cumulative sum register of the node above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives the data of the computation unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0036] The second type of node is the lower non-corner node, such as nodes (1,2) and (1,3). The upper row register receives data from the input matrix and sends the data to the upper row register of the node above. The left row register receives data from the left row register of the right node and sends the data to the left row register of the left node. The right row register receives data from the right row register of the left node and sends the data to the right row register of the right node and the computation unit of the current node. The distributed memory sends data to the computation unit of the current node. The computation unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the node above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0037] The third type of node is the lower right corner node, such as node (1,4). The upper row register receives the data of the input matrix and sends the data to the upper row register of the upper node and the left row register of the current node. The left row register receives the data of the upper row register of the current node and sends the data to the left row register of the left node. The right row register receives the data of the right row register of the left node and sends the data to the calculation unit of the current node. The distributed memory sends the data to the calculation unit of the current node. The calculation unit receives the data of the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives the data of the calculation unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0038] The fourth type of node is the left non-corner node, such as nodes (2,1) and (3,1). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. The left register receives data from the left register of the right node and sends the data to the right register of the current node. The right register receives data from the left register of the current node and sends the data to the right register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives data from the right register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0039] The fifth type of node is the lower left non-edge node, such as node (2,2). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. The left row register receives data from the left row register of the right node and sends the data to the left row register of the left node. The right row register receives data from the right row register of the left node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0040] The sixth type of node is a non-corner turning node on the diagonal line, such as nodes (2,3) and (3,2). The upstream register receives data from the upstream register of the node below and sends the data to the upstream register of the node above and the left register of the current node. The left register receives data from the left register of the right node and sends the data to the left register of the left node. The right register receives data from the right register of the left node and sends the data to the right register of the right node and the computation unit of the current node. The distributed memory sends the data to the computation unit of the current node. The computation unit receives data from the right register of the current node, the distributed memory of the current node, and the cumulative sum register of the node above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the computation unit of the node below.

[0041] The seventh type of node is the upper right non-edge node, such as node (3,3). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. There is no left register. The right register receives data from the right register of the left node and sends the data to the right register of the right node and the computation unit of the current node. The distributed memory sends data to the computation unit of the current node. The computation unit receives data from the right register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the computation unit of the lower node.

[0042] The eighth type of node is a right non-corner node, such as nodes (2,4) and (3,4). The upstream register receives data from the upstream register of the node below and sends the data to the upstream register of the node above. There is no left register. The right register receives data from the right register of the node to the left and sends the data to the computation unit of the current node. The distributed memory sends the data to the computation unit of the current node. The computation unit receives data from the right register of the current node, the distributed memory of the current node, and the cumulative sum register of the node above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the computation unit of the node below.

[0043] The ninth type of node is the upper left corner node, such as node (4,1). The upper register receives data from the upper register of the lower node and sends the data to the left register of the current node. The left register receives data from the upper register of the current node and sends the data to the right register of the current node. The right register receives data from the right register of the left node and sends the data to the right register of the right node and the computation unit of the current node. The distributed memory sends data to the computation unit of the current node. The computation unit receives the right register of the current node, the distributed memory of the current node, and zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the computation unit of the lower node.

[0044] The tenth type of node is a non-corner node on the upper edge, such as nodes (4,2) and (4,3). The upper register receives data from the upper register of the node below. There is no left register. The right register receives data from the right register of the left node and sends the data to the right register of the right node and the computation unit of the current node. The distributed memory sends data to the computation unit of the current node. The computation unit receives the right register of the current node, the distributed memory of the current node, and zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computation unit of the current node and sends the data to the computation unit of the node below.

[0045] The eleventh node is the top-right node, such as node (4,4). The up-row register receives data from the up-row register of the node below. There is no left-row register. The right-row register receives data from the right-row register of the left node and sends the data to the computational unit of the current node. The distributed memory sends data to the computational unit of the current node. The computational unit receives the right-row register of the current node, the distributed memory of the current node, and zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computational unit of the current node and sends the data to the computational unit of the node below.

[0046] It should be noted that when M and N are equal to 3, that is, when the row and column scale of the systolic array is 3×3, there is no fifth type of node (lower left non-edge node) and seventh type of node (upper right non-edge node); when M and N are equal to 2, that is, when the row and column scale of the systolic array is 2×2, the systolic array only has the lower left corner node, lower right corner node, upper left corner node, and upper right corner node.

[0047] In some embodiments, the total number of rows and the total number of columns of nodes of each systolic array unit are equal.

[0048] Furthermore, in some embodiments, if N is greater than or equal to 4, there is no upstream register at the current node that is a node in the N+1th to Mth columns.

[0049] It should be noted that the systolic array of the embodiment of the present invention can be extended to systolic arrays of any dimension. The distributed memory is distributed inside each systolic array, storing the data of one input matrix (i.e., the data of the second input matrix), the data of another input matrix (i.e., the data of the first input matrix), and the data of the output matrix, which are input and output from the same direction of the systolic array. Figure 1 In the 4×4 systolic array shown in FIG, the data of the first input matrix and the data of the output matrix are input and output from the bottom direction of the systolic array, but the data can be extended to the other three directions.

[0050] Optionally, in some embodiments, the computing unit includes: a multiplier and an adder, wherein the multiplier is used to multiply the data in the right row register of the current node and the distributed memory of the current node; and the adder is used to add the product obtained by the multiplier to the data in the accumulator register of the node above the current node.

[0051] Specifically, if Figure 3 As shown, Figure 3 FIG2 is a schematic diagram of the structure of a computing unit in a systolic array node according to a specific embodiment of the present invention, wherein data in the right row register of the current node and the distributed memory of the current node are multiplied in a multiplier, the obtained product is added to the data from the accumulation register (or zero input) of the upper node, and the sum is written to the accumulated sum register of the current node.

[0052] Optionally, in some embodiments, the distributed memory is a dynamic random access memory, a static random access memory, a register or other memories.

[0053] Specifically, dynamic random access memory (DRAM) can store data dynamically and maintain data through periodic refreshes; static random access memory (SRAM) can quickly store and read data without requiring refresh operations; and registers can store data at ultra-high speeds but have smaller capacities. The distributed memory in the embodiments of the present invention can be DRAM, DRAM, registers, or other memory, or a combination of the above, without limitation.

[0054] According to an embodiment of the present invention, a distributed memory-based, co-directional systolic array circuit is proposed. By arranging distributed memory within each systolic array, when input matrix data enters the systolic array from the bottom, computation is initiated at a node on one side. Upon reaching a diagonal line of the systolic array, the input data turns leftward until it reaches the leftmost node, where computation is initiated again. The input data propagates rightward node by node until it reaches the rightmost node. All nodes passed through participate in the matrix multiplication computation, and the accumulated sum is accumulated in the opposite direction of the input data entering the array and in the same direction as the output data leaving the array. Thus, by employing parallel computing and a co-directional input-output design approach, the matrix multiplication operation speed is accelerated, the symmetry of the systolic array is improved, and routing costs are reduced.

[0055] An embodiment of the present invention further provides a data processing device, comprising the above-mentioned distributed memory-based unidirectional systolic array circuit.

[0056] According to an embodiment of the present invention, a data processing device arranges distributed memory within each systolic array. When input matrix data enters the systolic array from the bottom, computation is initiated at a node on one side. Upon reaching a diagonal of the systolic array, the input data turns leftward until it reaches the leftmost node, where computation is initiated again. The computation then propagates rightward, node by node, until it reaches the rightmost node. All nodes passed through participate in the matrix multiplication computation, and the accumulated sum is accumulated in the opposite direction of the input data entering the array and in the same direction as the output data leaving the array. Thus, by employing parallel computing and a co-directional input-output design approach, the matrix multiplication operation speed is accelerated, the symmetry of the systolic array is improved, and routing costs are reduced.

[0057] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0058] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0059] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A distributed memory-based systolic array circuit, characterized in that: include: At least one systolic array unit, each systolic array unit includes nodes from the first to the Nth rows and nodes from the first to the Mth columns, where N and M are both integers, N≤M, and each node includes: an upstream register, a left row register, a right row register, a distributed memory, a computing unit, and an accumulated sum register, wherein: The upstream register is used to receive data of the first input matrix and send the first data to the upstream register of the node above the current node, wherein when the current node is the K-th row node and the current node is the N-K+1-th column node, the first data is simultaneously sent to the left row register of the current node, K≤N; The left row register is used to receive second data sent by the left row register of the node to the right of the current node, and send the second data to the left row register of the node to the left of the current node, and when the current node is in the first column, send the second data to the right row register of the current node at the same time; The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the node to the right of the current node and the computing unit of the current node; The distributed memory is used to store data of a second input matrix and send the data of the second input matrix to the computing unit of the current node; a computing unit, configured to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node, and the fourth data of the cumulative sum register of the node above the current node, and send the computing result as the fifth data to the cumulative sum register of the current node; The cumulative sum register is used to receive a calculation result of the calculation unit of the current node, wherein when the current node is a first-row node, the calculation result of the calculation unit of the current node is sent to a unit below the current systolic array as data of an output matrix.

2. The distributed memory-based systolic array circuit according to claim 1, characterized in that: If N is greater than or equal to 4, the upstream register does not exist at the node in the N+1th to Mth columns where the current node is located.

3. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The total number of rows and the total number of columns of nodes of each systolic array unit are equal.

4. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The calculation unit includes: a multiplier, configured to multiply the data in the right row register of the current node and the data in the distributed memory of the current node; An adder is used to add the product obtained by the multiplier and the data in the cumulative sum register of the node above the current node.

5. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The input direction and output direction of each systolic array unit are the same.

6. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The distributed memory is at least one of a dynamic random access memory, a static random access memory, a register or other memories.

7. A data processing device, characterized in that: include: A distributed memory-based unidirectional systolic array circuit according to any one of claims 1 to 6.

Images