A systolic array unit suitable for fully connected layers and its systolic array structure

Abstract

The present invention discloses a systolic array unit suitable for a fully connected layer and a systolic array structure thereof. The systolic array unit comprises: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator. Through the above structural combination, the systolic array unit of the present invention can simultaneously store two sets of weight data and perform multiplication and accumulation calculations with the same input data. Compared with the single-set data calculation method of a conventional systolic matrix, the systolic matrix constructed based on the systolic array unit of the present invention is more suitable for processing matrix operations in a fully connected layer of a neural network. Since it can simultaneously calculate two sets of multiplication and accumulation operations, the operation speed of the fully connected layer can be improved.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a systolic array unit suitable for a fully connected layer and a systolic array structure thereof. Background Art

[0002] With the rise of artificial intelligence, deep learning is being used more and more in various fields, such as image recognition.

[0003] In deep learning neural networks, the computation speed of the fully connected layer is one of the factors that affect the processing efficiency of the neural network. To improve the computational efficiency of the fully connected layer, the systolic matrix method is currently often used for calculation.

[0004] In conventional systolic matrix calculations, the conventional systolic array input method is as follows Figure 1 The input data matrix is ​​shown in Table 1, and the weight matrix is ​​shown in Table 2.

[0005] Table 1

[0006] a11 a12 a13 a21 a22 a23 a31 a23 a33

[0007] Table 2

[0008] w11 w12 w13 w21 w22 w23 w31 w23 w33

[0009] from Figure 1 As shown in Tables 1 and 2, in a conventional systolic matrix, each systolic array unit can only receive one weight data of the same set of inputs at a time, and each systolic array unit only receives one input data per cycle. Therefore, only a single calculation can be performed per cycle, which significantly reduces the computational efficiency of the fully connected layer. Summary of the Invention

[0010] To address the low computational efficiency of the fully connected layer described above, the present invention provides a systolic array unit and its corresponding systolic array structure suitable for fully connected layers. Each systolic array unit can simultaneously store two sets of corresponding weight data, allowing two multiplication-addition operations to be performed in parallel on the same input data. Because two sets of multiplication-accumulation operations can be performed simultaneously, this structure can simultaneously process calculations for two output channels, significantly improving computational speed during fully connected layer calculations.

[0011] To solve the above technical problems, the first aspect of the present invention discloses a systolic array unit suitable for a fully connected layer, wherein the systolic array unit includes: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator; wherein,

[0012] The input register and the first weight register are commonly connected to the first multiplier, and the first multiplier and the first partial sum register are commonly connected to the first accumulator; the input register and the second weight register are commonly connected to the second multiplier, and the second multiplier and the second partial sum register are commonly connected to the second accumulator;

[0013] The input register receives input data input to the systolic array unit;

[0014] The first weight register and the second weight register each store a weight data, and the weight data stored in the first weight register and the weight data stored in the second weight register are in the same position of the weights of different output channels of the same layer;

[0015] The weight data stored in the first weight register is multiplied by the input data, the result of which is input into the first accumulator, added to the first partial sum in the first partial sum register, and the result of the addition is output to the next systolic array unit adjacent in the vertical direction;

[0016] The weight data stored in the second weight register is multiplied by the input data, and the result is input into the second accumulator, added to the second partial sum in the second partial sum register, and the result of the addition is output to the next systolic array unit adjacent in the vertical direction.

[0017] A second aspect of the present invention discloses a systolic array structure suitable for a fully connected layer, wherein the systolic array structure is mainly composed of N systolic array units according to claim 1; wherein N is a positive integer and N ≥ 2;

[0018] In two longitudinally adjacent systolic array units, a first weight register of the upper systolic array unit is connected to a first weight register of the lower systolic array unit, a second weight register of the upper systolic array unit is connected to a second weight register of the lower systolic array unit, a first accumulator of the upper systolic array unit is connected to a first partial sum register of the lower systolic array unit, and a second accumulator of the upper systolic array unit is connected to a second partial sum register of the lower systolic array unit; and data transmitted between the two units includes two weight data and two accumulated data.

[0019] In two laterally adjacent systolic array units, the input register of the upper systolic array unit is connected to the input register of the lower systolic array unit, and the data transmitted between the two is input data.

[0020] Preferably, two weight data entering the same systolic array unit are located at the same position in their respective weight matrices.

[0021] In a third aspect of the present invention, a computing structure suitable for a fully connected layer is disclosed, comprising: input storage, weight storage, and output cache, such as a systolic array structure suitable for a fully connected layer as described in any one of claims 2-3 above; wherein,

[0022] The input storage, the weight storage, and the output buffer are respectively connected to the systolic array structure;

[0023] During calculation of the fully connected layer using the systolic array structure, the systolic array structure loads two sets of weight data from the weight storage; after the weights are loaded, the input storage transmits the input data in parallel by row to the systolic array structure for calculation, with each row being one clock cycle later than the previous row;

[0024] In the systolic array structure, after the input data is multiplied by the two weight data in the current systolic array unit, it is added with the corresponding partial sum from the previous systolic array unit and the addition results are respectively transmitted to the next systolic array unit. The array unit in the last row transmits the two final accumulated results to the output buffer for subsequent calculation.

[0025] Preferably, the input storage is connected to the first column of systolic array units in the systolic array structure.

[0026] Preferably, the weight storage is connected to the first row of systolic array units in the systolic array structure.

[0027] A fourth aspect of the present invention discloses a fully connected layer, including a computing structure suitable for a fully connected layer as described in the aforementioned solution.

[0028] The fifth aspect of the present invention discloses a neural network structure, including a fully connected layer as described in the above scheme.

[0029] In a sixth aspect of the present invention, a processing device is disclosed, comprising a neural network structure as described in the aforementioned scheme.

[0030] Through one or more technical solutions of the present invention, the present invention has the following beneficial effects or advantages:

[0031] The present invention discloses a systolic array unit suitable for a fully connected layer and a systolic array structure thereof, comprising: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator. Through the above structural combination, the systolic array unit of the present invention can simultaneously store two sets of weight data and perform multiplication and accumulation calculations with the same input data. Compared with the single-set data calculation method of a conventional systolic matrix, the systolic matrix constructed based on the systolic array unit of the present invention is more suitable for processing matrix operations in a fully connected layer of a neural network. Since it can simultaneously calculate two sets of multiplication and accumulation operations, the operation speed of the fully connected layer can be improved.

[0032] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be construed as limiting the present invention. The same reference numerals are used throughout the accompanying drawings to denote the same components.

[0034] In the attached figure:

[0035] Figure 1 A schematic diagram of the input method of a conventional systolic array is shown;

[0036] Figure 2 A schematic structural diagram of a systolic array unit according to an embodiment of the present invention is shown;

[0037] Figure 3 A schematic diagram of an input method of a systolic array unit according to an embodiment of the present invention is shown;

[0038] Figure 4 A schematic diagram of a systolic array structure suitable for a fully connected layer according to one embodiment of the present invention is shown;

[0039] Figure 5 A schematic diagram of a computing structure according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0040] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0041] As is well known, the basic idea of ​​a systolic matrix is: in a matrix multiplication operation (A*B=Y), matrix B is fixed, allowing A to flow through the systolic matrix cells and continuously outputting Y; or, alternatively, A and B are allowed to flow through the systolic matrix cells, while the result Y is stored in the systolic matrix cells. Convolution operations typically adopt the former approach. Therefore, in actual convolution operations, the weights W must first be stored in the systolic matrix cells, and then the input image is processed in a specific manner before being input. The present invention builds on this foundation by constructing systolic array cells, allowing each systolic array cell to store an additional weight data item and perform an additional multiplication-addition operation, thereby improving the computational speed of the fully connected layer.

[0042] See Figure 2 , is a schematic diagram of the structure of the systolic array unit disclosed in an embodiment of the present invention. The systolic array unit includes: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator. Figure 2 In the embodiment, a systolic array unit is exemplarily composed of one input register, two weight registers, two partial sum registers, two multipliers, and two accumulators. However, it is worth noting that the systolic array unit can simultaneously store three, four, or even more weight data. However, in actual circuits, as the weight storage increases, the hardware overhead will also increase. Therefore, too many inputs will increase the cost. In the actual neural network operation process, not every fully connected layer has many output channels. Excessive weight storage may cause resource waste. Considering the above factors, this embodiment preferably calculates two weight data, but it does not constitute a limitation. However, the systolic array unit still only needs one input register to obtain the input data.

[0043] The systolic array unit of this embodiment is suitable for fully connected layer calculations.

[0044] In the systolic array unit, the input register and the first weight register are commonly connected to a first multiplier, the first multiplier and the first partial sum register are commonly connected to a first accumulator; the input register and the second weight register are commonly connected to a second multiplier, the second multiplier and the second partial sum register are commonly connected to a second accumulator;

[0045] In addition, the systolic array units are also connected to other systolic array units.

[0046] Specifically, see Figure 2-Figure 4 The input register is connected to the input register of the previous systolic array unit adjacent in the horizontal direction. The first weight register, the second weight register, the first partial sum register, and the second partial sum register are all connected to the previous systolic array unit adjacent in the vertical direction; and the first weight register, the second weight register, the first accumulator, and the second accumulator are all connected to the next systolic array unit adjacent in the vertical direction.

[0047] It is worth noting that if the systolic array unit is a first-row systolic array unit, the input register is connected to the input storage. If the systolic array unit is a first-row systolic array unit, the first weight register and the second weight register are connected to the weight storage. If the systolic array unit is a last-row systolic array unit, the first accumulator and the second accumulator are connected to the output buffer.

[0048] The implementation principle of the systolic array unit is introduced below.

[0049] The input register receives input data from the input systolic array unit.

[0050] The first weight register and the second weight register each store a weight data, which is obtained from the previous systolic array unit adjacent to the vertical direction, and the weight data stored in the first weight register and the weight data stored in the second weight register are in the same position of the weights of different output channels of the same layer.

[0051] The weight data stored in the first weight register is multiplied by the input data, and the obtained result is input into the first accumulator, added to the first partial sum in the first partial sum register, and the added result is output to the next systolic array unit adjacent in the vertical direction; wherein the first partial sum register obtains the first partial sum from the previous systolic array unit adjacent in the vertical direction.

[0052] The weight data stored in the second weight register is multiplied by the input data, and the obtained result enters the second accumulator, is added to the second partial sum in the second partial sum register, and the added result is output to the next systolic array unit adjacent in the vertical direction; wherein the second partial sum register obtains the second partial sum from the previous systolic array unit adjacent in the vertical direction.

[0053] See Figure 3 , is a schematic diagram of the input method of the systolic array unit. And Tables 3 and 4 are schematic diagrams of two sets of weight data of the systolic array unit.

[0054] Table 3

[0055] w11a w12a w13a w21a w22a w23a w31a w23a w33a

[0056] Table 4

[0057] w11b w12b w13b w21b w22b w23b w31b w23b w33b

[0058] from Figure 3 As can be seen from Tables 3 and 4, the systolic array unit of this embodiment can simultaneously store two sets of weight data and perform multiplication and accumulation calculations with the same input data. Compared with the single-set data calculation method of the conventional systolic matrix, the systolic array unit of this embodiment is more suitable for processing matrix operations of the fully connected layer of the neural network. Moreover, since it can simultaneously calculate two sets of multiplication and accumulation operations, it can improve the operation speed of the fully connected layer.

[0059] The above is an introduction to the structure of a systolic array unit, and numerous systolic array units constitute a complete systolic array.

[0060] Based on the same inventive concept, this embodiment also discloses a systolic array structure suitable for a fully connected layer, see Figure 4 The systolic array structure is mainly composed of N systolic array units described in the above embodiments. Wherein, N is a positive integer and N≥2. Figure 4 In the description, 9 systolic array units are used as an example, but this does not constitute a limit to the number.

[0061] In two vertically adjacent systolic array units, the first weight register of the preceding systolic array unit is connected to the first weight register of the following systolic array unit, the second weight register of the preceding systolic array unit is connected to the second weight register of the following systolic array unit, the first accumulator of the preceding systolic array unit is connected to the first partial sum register of the following systolic array unit, and the second accumulator of the preceding systolic array unit is connected to the second partial sum register of the following systolic array unit. Thus, the two vertically adjacent systolic array units are connected via four connecting lines, and the data transmitted by the four connecting lines are respectively: two weight data and two accumulated data; specifically, the transmitted data are respectively: the output of the first weight register of the vertically adjacent preceding systolic array unit, the output of the second weight register of the vertically adjacent preceding systolic array unit, the output of the first accumulator of the vertically adjacent preceding systolic array unit, and the output of the second accumulator of the vertically adjacent preceding systolic array unit.

[0062] In two laterally adjacent systolic array units, the input register of the upper systolic array unit is connected to the input register of the lower systolic array unit, and data transmitted between the two is: input data is transmitted using the input register.

[0063] It is worth noting that, in the systolic array structure of this embodiment, two weight data entering the same systolic array unit are located in the same position in their respective weight matrices.

[0064] This embodiment makes modifications to the original systolic array so that the systolic array can store two sets of weight data simultaneously and perform two independent multiplication and addition operations in one cycle, thereby effectively reducing the calculation time of the systolic array.

[0065] Based on the same inventive concept, the following embodiment discloses a computing structure suitable for a fully connected layer. Figure 5 , is a schematic diagram of a computing structure, including: input storage, weight storage, output cache, such as a systolic array structure suitable for a fully connected layer in the aforementioned embodiment; wherein,

[0066] The input storage, weight storage, and output buffer are each connected to the systolic array structure. Specifically, not shown in the figure, the input storage is connected to the first column of systolic array cells in the systolic array structure; the weight storage is connected to the first row of systolic array cells in the systolic array structure; and the output buffer is connected to the last row of systolic array cells in the systolic array structure.

[0067] In the process of calculating the fully connected layer using the systolic array structure, the systolic array structure loads two sets of weight data from the weight storage. Specifically, the weight storage transfers the two sets of weight data into the weight registers of the first row of systolic array units and transmits them in sequence. The two weight data entering the same systolic array unit have the same position in their respective weight matrices, and one systolic array can load two weight data. For the two stored weight data in the same array, they can be set as two sets of weights corresponding to different output channels of a fully connected layer of the same neural network, which will be multiplied and accumulated with the same input data. After the weights are loaded, the input storage transmits the input data in parallel by row to the systolic array structure for calculation, and each row is one clock cycle later than the previous row.

[0068] In the systolic array structure, after the input data is multiplied by the two weight data in the current systolic array unit, it is added with the corresponding partial sum from the previous systolic array unit and the addition results are passed to the next systolic array unit. The array unit in the last row passes the two final accumulated results to the output buffer for subsequent calculations.

[0069] The computing structure of this embodiment can process the calculation of two output channels simultaneously, and can significantly improve the computing speed when performing the calculation of the fully connected layer of the neural network.

[0070] Based on the same inventive concept, the following embodiment further discloses a fully connected layer, including a computing structure applicable to the fully connected layer described in the above embodiment.

[0071] Based on the same inventive concept, the following embodiment also discloses a neural network structure, including the fully connected layer described in the above embodiment.

[0072] Based on the same inventive concept, the following embodiment also discloses a processing device, including the neural network structure described in the above embodiment.

[0073] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0074] The present invention discloses a systolic array unit suitable for a fully connected layer and a systolic array structure thereof, comprising: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator. Through the above structural combination, the systolic array unit of the present invention can simultaneously store two sets of weight data and perform multiplication and accumulation calculations with the same input data. Compared with the single-set data calculation method of a conventional systolic matrix, the systolic matrix constructed based on the systolic array unit of the present invention is more suitable for processing matrix operations in a fully connected layer of a neural network. Since it can simultaneously calculate two sets of multiplication and accumulation operations, the operation speed of the fully connected layer can be improved.

[0075] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0076] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A systolic array unit suitable for a fully connected layer, characterized in that: The systolic array unit includes: an input register, a first weight register, a second weight register, a first partial sum register, a second partial sum register, a first multiplier, a second multiplier, a first accumulator, and a second accumulator; wherein, The input register and the first weight register are commonly connected to the first multiplier, and the first multiplier and the first partial sum register are commonly connected to the first accumulator; the input register and the second weight register are commonly connected to the second multiplier, and the second multiplier and the second partial sum register are commonly connected to the second accumulator; The input register receives input data input to the systolic array unit; The first weight register and the second weight register each store a weight data, and the weight data stored in the first weight register and the weight data stored in the second weight register are in the same position of the weights of different output channels of the same layer; The weight data stored in the first weight register is multiplied by the input data, the result of which is input into the first accumulator, added to the first partial sum in the first partial sum register, and the result of the addition is output to the next systolic array unit adjacent in the vertical direction; The weight data stored in the second weight register is multiplied by the input data, and the result is input into the second accumulator, added to the second partial sum in the second partial sum register, and the result of the addition is output to the next systolic array unit adjacent in the vertical direction.

2. A systolic array structure suitable for a fully connected layer, characterized in that: The systolic array structure is mainly composed of N systolic array units according to claim 1; wherein N is a positive integer and N ≥ 2; In two longitudinally adjacent systolic array units, a first weight register of the upper systolic array unit is connected to a first weight register of the lower systolic array unit, a second weight register of the upper systolic array unit is connected to a second weight register of the lower systolic array unit, a first accumulator of the upper systolic array unit is connected to a first partial sum register of the lower systolic array unit, and a second accumulator of the upper systolic array unit is connected to a second partial sum register of the lower systolic array unit; and data transmitted between the two units includes two weight data and two accumulated data. In two laterally adjacent systolic array units, the input register of the upper systolic array unit is connected to the input register of the lower systolic array unit, and the data transmitted between the two is input data.

3. The systolic array structure according to claim 2, wherein: The two weight data entering the same systolic array unit have the same position in their respective weight matrices.

4. A computing structure suitable for a fully connected layer, characterized in that: include: Input storage, weight storage, output cache, a systolic array structure suitable for a fully connected layer as described in any one of claims 2-3 above; wherein, The input storage, the weight storage, and the output buffer are respectively connected to the systolic array structure; During calculation of the fully connected layer using the systolic array structure, the systolic array structure loads two sets of weight data from the weight storage; after the weights are loaded, the input storage transmits the input data in parallel by row to the systolic array structure for calculation, with each row being one clock cycle later than the previous row; In the systolic array structure, after the input data is multiplied by the two weight data in the current systolic array unit, it is added with the corresponding partial sum from the previous systolic array unit and the addition results are respectively transmitted to the next systolic array unit. The array unit in the last row transmits the two final accumulated results to the output buffer for subsequent calculation.

5. The computing structure according to claim 4, wherein: The input storage is connected to a first column of systolic array cells in the systolic array structure.

6. The computing structure according to claim 4, wherein: The weight storage is connected to a first row of systolic array units in the systolic array structure.

7. A fully connected layer, characterized in that It comprises a computing structure suitable for a fully connected layer as described in any one of claims 4 to 6.

8. A neural network structure, characterized in that Comprising the fully connected layer as claimed in claim 7.

9. A processing device, characterized in that: Comprising the neural network structure as claimed in claim 8.

Images