Unidirectional systolic array circuit based on distributed memory and data processing device

Abstract

The invention relates to a same-direction systolic array circuit based on a distributed memory and a data processing device, and the circuit comprises at least one systolic array unit, each unit comprises first to Nth row nodes and first to Mth column nodes, each node comprises a first node, a second node, a third node, a fourth node, a fifth node and a sixth node, comprising an uplink register, a left-row register, a right-row register, a distributed memory, a computing unit and an accumulation sum register, the uplink register is used for receiving data of a first input matrix or first data and sending the first data, the left-row register is used for receiving the first data or second data and sending the second data, and the right-row register is used for receiving the second data or the second data; the right row register is used for receiving second or third data and sending the third data, the distributed memory is used for storing and sending data of a second input matrix, the calculation unit is used for receiving the data of the second input matrix, the third data and the fourth data and sending fifth data after calculation, and the accumulation sum register is used for receiving the fifth data and sending the fifth data after calculation. And sending the fourth data. Therefore, the operation speed is increased, and the symmetry is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of integrated circuit design, and in particular to a distributed memory-based unidirectional systolic array circuit and a data processing device. Background Art

[0002] In matrix multiplication, the number of rows (columns) of one matrix is ​​equal to the number of columns (rows) of another matrix, and each column (row) of the former matrix is ​​dot-producted with each row (column) of the latter matrix to obtain a new matrix. In the field of integrated circuit design, matrix multiplication can be implemented through circuits, usually by serially executing matrix multiplication through a central processing unit (CPU), but the operation speed of this array circuit is slow, so it needs to be optimized.

[0003] In the related art, usually, a tensor processing unit (TPU) and other application-specific integrated circuits (ASIC) are used to perform parallel computing using methods such as a systolic array to speed up the calculation.

[0004] However, the above technical means will increase the asymmetry of the systolic array and the routing cost when the data of another input matrix and the data of the output matrix are input and output from different directions, which needs to be solved urgently. Summary of the invention

[0005] The present invention provides a distributed memory-based co-directional systolic array circuit and device to solve the problems of low symmetry and high routing cost when TPU and traditional systolic array are used to implement matrix multiplication operations in the related art, thereby accelerating the operation speed of matrix multiplication, improving the symmetry of the systolic array, and reducing the routing cost.

[0006] To achieve the above object, a first embodiment of the present invention provides a distributed memory-based systolic array circuit, comprising:

[0007] At least one systolic array unit, each systolic array unit includes nodes from the first to the Nth row, nodes from the first to the Mth column, N and M are both integers, N≤M, each node includes: an upper row register, a left row register, a right row register, a distributed memory, a computing unit and an accumulated sum register, wherein,

[0008] The upper register is used to receive data of the first input matrix, and send the first data to the upper register of the node above the current node, wherein when the current node is the Kth row node and the current node is the N-K+1th column node, the first data is simultaneously sent to the left row register of the current node, K≤N;

[0009] The left row register is used to receive the second data sent by the left row register of the right node of the current node, and send the second data to the left row register of the left node of the current node, and when the current node is in the first column, send the second data to the right row register of the current node at the same time;

[0010] The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the right node of the current node and the computing unit of the current node;

[0011] The distributed memory is used to store data of a second input matrix and send the data of the second input matrix to the computing unit of the current node;

[0012] A calculation unit, configured to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node, and the fourth data of the cumulative sum register of the node above the current node, and send the calculation result as the fifth data to the cumulative sum register of the current node;

[0013] The cumulative sum register is used to receive the calculation result of the calculation unit of the current node, wherein when the current node is the first row node, the calculation result of the calculation unit of the current node is sent to the bottom of the current systolic array unit as the data of the output matrix.

[0014] According to an embodiment of the present invention, if N is greater than or equal to 4, the upstream register does not exist in the node where the current node is in the N+1th to Mth columns.

[0015] According to one embodiment of the present invention, the computing unit comprises:

[0016] A multiplier, used for multiplying the data in the right row register of the current node and the distributed memory of the current node;

[0017] An adder is used to add the product obtained by the multiplier to the data of the cumulative sum register of the node above the current node.

[0018] According to an embodiment of the present invention, the input direction and the output direction of each systolic array unit are the same.

[0019] According to an embodiment of the present invention, the distributed memory is at least one of a dynamic random access memory, a static random access memory, a register or other memories.

[0020] According to the distributed memory-based unidirectional systolic array circuit proposed in the embodiment of the present invention, by arranging distributed memory inside each systolic array, when the data of the input matrix enters the systolic array from the bottom direction, the calculation is started from the left node, and the input data turns to the left direction when it reaches a diagonal line of the systolic array, and the calculation is started after reaching the leftmost node, and propagates to the right node by node until it reaches the rightmost node. At the same time, all the passed nodes participate in the matrix multiplication calculation, and the cumulative sum direction is opposite to the direction of the input data entering the systolic array and the same as the direction of the output data leaving the systolic array. Therefore, by adopting parallel computing and unidirectional output design methods, the operation speed of matrix multiplication is accelerated, the symmetry of the systolic array is improved, and the routing cost is reduced.

[0021] To achieve the above object, another aspect of the present invention provides a data processing device, including the above-mentioned distributed memory-based co-directional systolic array circuit.

[0022] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 A schematic diagram of the structure of a distributed memory-based systolic array circuit provided by one embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a distributed memory-based co-directional systolic array data flow according to a specific embodiment of the present invention;

[0026] Figure 3 The figure is a schematic diagram of the structure of a computing unit in a systolic array node according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0027] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0028] A distributed memory-based systolic array circuit and a data processing device according to an embodiment of the present invention will be described below with reference to the accompanying drawings. First, a distributed memory-based systolic array circuit according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0029] Figure 1 It is a structural schematic diagram of a distributed memory-based unidirectional systolic array circuit provided by an embodiment of the present invention.

[0030] like Figure 1 As shown, the distributed memory-based unidirectional systolic array circuit includes:

[0031] At least one systolic array unit, each systolic array unit includes first to N-th row nodes, first to M-th column nodes, N and M are both integers, N≤M, each node includes: an upper row register, a left row register, a right row register, a distributed memory, a computing unit and a cumulative sum register, the upper row register is used to receive data of a first input matrix, and send the first data to the upper row register of the node above the current node, wherein when the current node is the K-th row node and the current node is the N-K+1-th column node, the first data is simultaneously sent to the left row register of the current node, K≤N; the left row register is used to receive second data sent by the left row register of the right node of the current node, and send the second data to the left row register of the left node of the current node, and when the current node is the first column, the second data is simultaneously sent to the right row register of the current node The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the node to the right of the current node and the calculation unit of the current node; the distributed memory is used to store the data of a second input matrix, and send the data of the second input matrix to the calculation unit of the current node; the calculation unit is used to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node and the fourth data of the cumulative sum register of the node above the current node, and send the calculation result as the fifth data to the cumulative sum register of the current node; the cumulative sum register is used to receive the calculation result of the calculation unit of the current node, wherein, when the current node is the first row node, the calculation result of the calculation unit of the current node is sent to the bottom of the current systolic array unit as the data of the output matrix.

[0032] Among them, the data of the first input matrix is ​​the data received by the upstream register of the first row node, the data of the second input matrix is ​​the data sent by the memory, the first data is the data sent by the upstream register, the second data is the data sent by the left row register, the third data is the data sent by the right row register, the fourth data is the data sent by the cumulative sum register, the fifth data is the data sent by the calculation unit, and the data of the output matrix is ​​the data sent by the cumulative sum register of the first row node.

[0033] Specifically, if Figure 2 As shown, Figure 2 The schematic diagram of the data flow of a distributed memory-based unidirectional systolic array according to a specific embodiment of the present invention is shown in FIG. The data of the input matrix enters the systolic array from the bottom, but needs to reach a node on the side perpendicular to the entry direction to start the calculation. Here, the example starts the calculation from the node on the left, but it can be extended to the right. The input data entering from the bottom direction turns to the left direction when it reaches a diagonal line of the systolic array, and starts the calculation after reaching the node on the left, and propagates to the right node by node until it reaches the node on the right and ends, and participates in the calculation in the passed nodes. The cumulative sum direction is opposite to the direction of the input data entering the systolic array and the same as the direction of the output data leaving the systolic array. In the embodiment of the present invention, the cumulative sum propagates downward.

[0034] For example, Figure 1 When the row and column size of the systolic array reaches 4×4 or more, the systolic array circuit has 11 types of nodes, including:

[0035] The first type of node is the lower left corner node, such as node (1,1). The upper row register receives the data of the input matrix and sends the data to the upper row register of the upper node. The left row register receives the data of the left row register of the right node and sends the data to the right row register of the current node. The right row register receives the data of the left row register of the current node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends the data to the computing unit of the current node. The computing unit receives the data of the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives the data of the computing unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0036] The second type of node is the lower non-corner node, such as nodes (1,2) and (1,3). The upper row register receives the data of the input matrix and sends the data to the upper row register of the upper node. The left row register receives the data of the left row register of the right node and sends the data to the left row register of the left node. The right row register receives the data of the right row register of the left node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends the data to the computing unit of the current node. The computing unit receives the data of the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives the data of the computing unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0037] The third type of node is the lower right corner node, such as node (1,4). The upper row register receives the data of the input matrix and sends the data to the upper row register of the upper node and the left row register of the current node. The left row register receives the data of the upper row register of the current node and sends the data to the left row register of the left node. The right row register receives the data of the right row register of the left node and sends the data to the calculation unit of the current node. The distributed memory sends the data to the calculation unit of the current node. The calculation unit receives the data of the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives the data of the calculation unit of the current node and sends the data to the bottom of the systolic array as the data of the output matrix.

[0038] The fourth type of node is the left non-corner node, such as nodes (2,1) and (3,1). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. The left row register receives data from the left row register of the right node and sends the data to the right row register of the current node. The right row register receives data from the left row register of the current node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0039] The fifth type of node is the lower left non-edge node, such as node (2,2). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. The left row register receives data from the left row register of the right node and sends the data to the left row register of the left node. The right row register receives data from the right row register of the left node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends the data to the computing unit of the current node. The computing unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0040] The sixth type of node is a turning node that is not a corner on the diagonal line, such as nodes (2,3) and (3,2). The upstream register receives data from the upstream register of the lower node, and sends the data to the upstream register of the upper node and the left row register of the current node. The left row register receives data from the left row register of the right node, and sends the data to the left row register of the left node. The right row register receives data from the right row register of the left node, and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends the data to the computing unit of the current node. The computing unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0041] The seventh type of node is the upper right non-edge node, such as node (3,3). The upstream register receives data from the upstream register of the lower node and sends the data to the upstream register of the upper node. There is no left row register. The right row register receives data from the right row register of the left node and sends the data to the right row register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives data from the right row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0042] The eighth type of node is the non-corner node on the right, such as nodes (2,4) and (3,4). The up-row register receives data from the up-row register of the lower node and sends the data to the up-row register of the upper node. There is no left-row register. The right-row register receives data from the right-row register of the left node and sends the data to the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives data from the right-row register of the current node, the distributed memory of the current node, and the cumulative sum register of the upper node, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0043] The ninth type of node is the upper left corner node, such as node (4,1). The upper register receives data from the upper register of the lower node and sends the data to the left register of the current node. The left register receives data from the upper register of the current node and sends the data to the right register of the current node. The right register receives data from the right register of the left node and sends the data to the right register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives the right register of the current node, the distributed memory of the current node, and the zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0044] The tenth type of node is the upper non-corner node, such as nodes (4,2) and (4,3). The upper register receives data from the upper register of the lower node. There is no left register. The right register receives data from the right register of the left node, and sends the data to the right register of the right node and the computing unit of the current node. The distributed memory sends data to the computing unit of the current node. The computing unit receives the right register of the current node, the distributed memory of the current node, and the zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the computing unit of the current node and sends the data to the computing unit of the lower node.

[0045] The eleventh node is the upper right corner node, such as node (4,4). The upper register receives data from the upper register of the lower node. There is no left register. The right register receives data from the right register of the left node and sends the data to the calculation unit of the current node. The distributed memory sends data to the calculation unit of the current node. The calculation unit receives the right register of the current node, the distributed memory of the current node, and the zero input from above, and sends the calculation result to the cumulative sum register of the current node. The cumulative sum register receives data from the calculation unit of the current node and sends the data to the calculation unit of the lower node.

[0046] It should be noted that when M and N are equal to 3, that is, when the row and column scale of the systolic array is 3×3, there is no fifth type of node (lower left non-edge node) and seventh type of node (upper right non-edge node); when M and N are equal to 2, that is, when the row and column scale of the systolic array is 2×2, the systolic array only has the lower left corner node, the lower right corner node, the upper left corner node and the upper right corner node.

[0047] In some embodiments, the total number of rows and the total number of columns of nodes of each systolic array unit are equal.

[0048] Further, in some embodiments, if N is greater than or equal to 4, there is no upstream register when the current node is a node in the N+1th to Mth columns.

[0049] It should be noted that the systolic array of the embodiment of the present invention can be extended to a systolic array of any dimension. The distributed memory is distributed inside each systolic array, and stores data of an input matrix (i.e., data of the second input matrix), data of another input matrix (i.e., data of the first input matrix), and data of the output matrix, which are input and output from the same direction of the systolic array. Figure 1 In the 4×4 systolic array illustrated in FIG, the data of the first input matrix and the data of the output matrix are input and output from the bottom direction of the systolic array, but this can be extended to the other three directions.

[0050] Optionally, in some embodiments, the computing unit includes: a multiplier and an adder, wherein the multiplier is used to multiply the data in the right row register of the current node and the distributed memory of the current node; and the adder is used to add the product obtained by the multiplier to the data in the accumulator register of the node above the current node.

[0051] Specifically, if Figure 3 As shown, Figure 3 A schematic diagram of the structure of a computing unit in a systolic array node according to a specific embodiment of the present invention is provided, wherein the data in the right row register of the current node and the distributed memory of the current node are multiplied in the multiplier, the obtained product is added with the data from the accumulated register (or zero input) of the upper node, and the sum is written into the accumulated sum register of the current node.

[0052] Optionally, in some embodiments, the distributed memory is a dynamic random access memory, a static random access memory, a register or other memories.

[0053] Specifically, dynamic random access memory (DRAM) can store data dynamically and maintain data through periodic refresh; static random access memory (SRAM) can store and read data quickly without refresh operation; registers can store data at an ultra-high rate but have a small capacity. The distributed memory of the embodiment of the present invention can be a dynamic random access memory or a dynamic random access memory or a register or other memory, or a combination of the above memories, which is not limited here.

[0054] According to the distributed memory-based unidirectional systolic array circuit proposed in the embodiment of the present invention, by arranging distributed memory inside each systolic array, when the data of the input matrix enters the systolic array from the bottom direction, the calculation is started from a node on one side, and the input data turns to the left direction when it reaches a diagonal line of the systolic array, and the calculation is started after reaching the leftmost node, and propagates to the right node by node until it reaches the rightmost node, and all the passed nodes participate in the matrix multiplication calculation, and the cumulative sum direction is opposite to the direction of the input data entering the systolic array, and the same as the direction of the output data leaving the systolic array. Therefore, by adopting parallel computing and unidirectional output design methods, the operation speed of matrix multiplication is accelerated, the symmetry of the systolic array is improved, and the routing cost is reduced.

[0055] An embodiment of the present invention further provides a data processing device, comprising the above-mentioned distributed memory-based co-directional systolic array circuit.

[0056] According to the data processing device proposed in the embodiment of the present invention, by arranging a distributed memory inside each systolic array, when the data of the input matrix enters the systolic array from the bottom, the calculation is started from a node on one side, and the input data turns to the left direction when it reaches a diagonal line of the systolic array, and the calculation is started after reaching the leftmost node, and propagates to the right node by node until it reaches the rightmost node, and all the nodes passed by participate in the matrix multiplication calculation, and the cumulative sum direction is opposite to the direction of the input data entering the systolic array and the same as the direction of the output data leaving the systolic array. Therefore, by adopting parallel computing and the same-direction output design method, the operation speed of matrix multiplication is accelerated, the symmetry of the systolic array is improved, and the routing cost is reduced.

[0057] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0058] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0059] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A distributed memory-based systolic array circuit, characterized in that: include: At least one systolic array unit, each systolic array unit includes nodes from the first to the Nth row, nodes from the first to the Mth column, N and M are both integers, N≤M, each node includes: an upper row register, a left row register, a right row register, a distributed memory, a computing unit and an accumulated sum register, wherein, The upper register is used to receive data of the first input matrix, and send the first data to the upper register of the node above the current node, wherein when the current node is the Kth row node and the current node is the N-K+1th column node, the first data is simultaneously sent to the left row register of the current node, K≤N; The left row register is used to receive the second data sent by the left row register of the right node of the current node, and send the second data to the left row register of the left node of the current node, and when the current node is in the first column, send the second data to the right row register of the current node at the same time; The right row register is used to receive the second data sent by the left row register of the current node, and send the third data to the right row register of the right node of the current node and the computing unit of the current node; The distributed memory is used to store data of a second input matrix and send the data of the second input matrix to the computing unit of the current node; A calculation unit, configured to receive the third data of the right row register of the current node, the data of the second input matrix of the distributed memory of the current node, and the fourth data of the cumulative sum register of the node above the current node, and send the calculation result as the fifth data to the cumulative sum register of the current node; The cumulative sum register is used to receive the calculation result of the calculation unit of the current node, wherein when the current node is the first row node, the calculation result of the calculation unit of the current node is sent to the bottom of the current systolic array unit as the data of the output matrix.

2. The distributed memory-based systolic array circuit according to claim 1, characterized in that: If N is greater than or equal to 4, the upstream register does not exist in the node where the current node is in the N+1th to Mth columns.

3. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The total number of rows and the total number of columns of nodes of each systolic array unit are equal.

4. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The computing unit comprises: A multiplier, used for multiplying the data in the right row register of the current node and the distributed memory of the current node; An adder is used to add the product obtained by the multiplier to the data of the cumulative sum register of the node above the current node.

5. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The input direction and the output direction of each systolic array unit are the same.

6. The distributed memory-based systolic array circuit according to claim 1, characterized in that: The distributed memory is at least one of a dynamic random access memory, a static random access memory, a register or other memories.

7. A data processing device, characterized in that: include: A unidirectional systolic array circuit based on a distributed memory as described in any one of claims 1-6.

Images