Systolic array and control method thereof, accelerator and electronic equipment

Abstract

The invention discloses a systolic array and a control method thereof, an accelerator, electronic equipment and a systolic array, and the systolic array comprises a plurality of processing units which are regularly arranged and connected in a matrix form, and two boundary processing units which are located at the head part and the tail part of each column in the column direction of the systolic array, meanwhile, different weight data segments contained in corresponding column vectors in the weight matrix are loaded, and weight data in the loaded weight data segments are sequentially transmitted to corresponding processing units in the column according to the column direction; two boundary processing units located at the head and the tail of each row in the row direction of the systolic array respond to loaded to-be-calculated weight data and simultaneously load different input data segments contained in corresponding column vectors in an input matrix; and sequentially transmitting the loaded input data in the input data segment to each processing unit in the line according to the line direction, and carrying out calculation operation on the input data and the weight data which are loaded into the processing units so as to realize a calculation task.

Description

Technical Field

[0001] The present application relates to the application field of pulsating arrays, and in particular to a pulsating array and a control method thereof, an accelerator, and an electronic device. Background Art

[0002] As a hardware architecture for parallel computing, a systolic array is usually composed of multiple identical processing elements (PEs). It reuses input data to achieve efficient parallel computing. Therefore, it is widely used for hardware acceleration of convolutional neural networks to improve model training and inference speed.

[0003] In practical applications, in order to improve the speed of data loading in the systolic array and reduce the demand for storage resources, a fixed-weight systolic array is currently used to implement computing tasks. An R×R=4×4 systolic array is used as an example. Figure 1 The diagram shown is a diagram of a systolic array loading weight data and input data to implement convolution or matrix multiplication operations. Each operation takes 3R clock cycles. For computing tasks with large amounts of data, frequent data switching is required, resulting in long calculation times, reduced computing task execution efficiency, and a large waste of PE resources. Summary of the Invention

[0004] In order to solve the above problems, this application provides the following technical solutions:

[0005] In a first aspect, the present application provides a systolic array, comprising a plurality of processing units, wherein the plurality of processing units are regularly arranged and connected in a matrix form;

[0006] Two boundary processing units located at the head and tail of each column in the column direction of the systolic array are used to simultaneously load different weight data segments contained in the corresponding column vectors in the weight matrix, and sequentially transmit the weight data in the loaded weight data segments to the corresponding processing units in the column along the column direction;

[0007] two boundary processing units located at the head and tail of each row in the row direction of the systolic array, configured to, in response to the weight data to be calculated having been loaded, simultaneously load different input data segments contained in corresponding column vectors in the input matrix, and sequentially transmit the input data in the loaded input data segments to each processing unit in the row in the row direction;

[0008] The processing unit is configured to perform a calculation operation on the input data and the weight data loaded therein to achieve a calculation task;

[0009] The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

[0010] In a possible implementation, the method further includes:

[0011] a control unit, configured to be connected to at least a boundary processing unit located at an edge of the systolic array, and to determine, in response to a load instruction for each column vector in the weight matrix or the input matrix, two data segments into which the column vector to be loaded is divided, and to simultaneously transmit the two data segments to two boundary processing units located at the head and tail of a corresponding row or column in a corresponding row direction or column direction, respectively.

[0012] In one possible implementation, each of the processing units includes: a weight storage circuit, an input storage circuit, and a calculation circuit, wherein:

[0013] The weight storage circuit is configured to be connected to the calculation circuit, independently store the loaded weight data from different weight matrices, and send the weight data from the weight matrix currently to be calculated to the calculation circuit;

[0014] an input storage circuit, connected to the calculation circuit, for storing each input data loaded from the same input matrix, and sending the currently loaded input data to the calculation circuit;

[0015] The calculation circuit is used to perform calculation operations on the received corresponding weight data and input data.

[0016] In a possible implementation, when the systolic array further includes a control unit for controlling loading of the weight matrix and the input matrix, the control unit is further configured to:

[0017] In response to the data from the same matrix in the storage circuit being used up, the systolic array is controlled to load the data of the corresponding next matrix.

[0018] In one possible implementation, the weight storage circuit includes: a first weight register, a second weight register, and a first weight selector, wherein:

[0019] The first weight register is used to store first weight data from the first weight matrix;

[0020] The second weight register is used to store second weight data from a second weight matrix;

[0021] The first weight selector is used to be connected to the first weight register, the second weight register and the calculation circuit respectively, and in response to the control signal of the weight matrix to be calculated currently, select the weight register corresponding to the weight matrix to be calculated currently, read the weight data in the weight register and send it to the calculation circuit.

[0022] In a possible implementation, the weight storage circuit further includes: a third weight register and a second weight selector;

[0023] The third weight register is used to be connected to the first weight selector and the second weight selector respectively, and store third weight data from a third weight matrix;

[0024] The second weight selector is used to be connected to the first weight register, the second weight register and the third weight register respectively, and in response to the control signal of the next weight matrix to be calculated, select the weight register corresponding to the next weight matrix to be calculated, read the weight data in the weight register and send it to the calculation circuit.

[0025] In a possible implementation, the input storage circuit includes a first input register and a second input register, and the calculation circuit includes a first calculation unit and a second calculation unit, wherein:

[0026] The first input register is configured to be connected to the first computing unit, store each first input data from the first input matrix, and send the first input data currently loaded to the first computing unit;

[0027] The second input register is configured to be connected to the second computing unit, store each second input data from the second input matrix, and send the currently loaded second input data to the second computing unit;

[0028] The first calculation unit is configured to perform a first calculation operation on the received corresponding first input data and first weight data;

[0029] The second calculation unit is configured to perform a second calculation operation on the received corresponding second input data and second weight data;

[0030] The first weight data and the second weight data come from different weight registers.

[0031] In one possible implementation, the weight storage circuit further includes:

[0032] a weight allocator, connected to each weight register, and configured to allocate weight data of a weight matrix loaded next from the systolic array to the weight register corresponding to the calculation operation in response to the first calculation unit or the second calculation unit completing the corresponding calculation operation, so as to replace the original weight data in the weight register;

[0033] Wherein, when both the first input register and the second input register store input data, the first calculation unit and the second calculation unit can both be in a working state to complete different calculation operations.

[0034] A second aspect of the present application provides a control method for a systolic array, wherein the systolic array includes a plurality of processing units, and the plurality of processing units are regularly arranged and connected in a matrix form, the method comprising:

[0035] In response to a weight loading instruction, starting from two boundary processing units at the head and tail of each column in the column direction of the systolic array, respectively, different weight data segments contained in the corresponding column vector data in the weight matrix are loaded simultaneously;

[0036] In response to an input load instruction, starting from two boundary processing units at the head and tail of each row in the row direction of the systolic array, simultaneously load different input data segments corresponding to column vectors in the input matrix; wherein, for a weight matrix and an input matrix that implement a calculation operation, the input matrix is loaded after the weight matrix has been loaded;

[0037] The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

[0038] A third aspect of the present application provides an accelerator, comprising: at least one pulsation array as described in the first aspect of the present application.

[0039] A fourth aspect of the present application provides an electronic device, comprising: at least one memory, at least one processor, and at least one systolic array according to the first aspect of the present application, wherein:

[0040] The memory is used to store a plurality of computer instructions for computing tasks;

[0041] The processor is configured to load and execute the computer instructions in response to a request to execute the computing task, and implement the following steps:

[0042] In response to a weight loading instruction, starting from two boundary processing units at the head and tail of each column in the column direction of the systolic array, respectively, different weight data segments contained in the corresponding column vector data in the weight matrix are loaded simultaneously;

[0043] In response to an input load instruction, starting from two boundary processing units at the head and tail of each row in the row direction of the systolic array, respectively, different input data segments corresponding to column vectors in the input matrix are loaded simultaneously;

[0044] Wherein, for a weight matrix and an input matrix for implementing a calculation operation, the input matrix is loaded after the weight matrix has been loaded;

[0045] The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0047] Figure 1 A schematic diagram of unidirectional data loading for a systolic array;

[0048] Figure 2 A schematic diagram of a matrix acquisition method for systolic array loading;

[0049] Figure 3 This is a schematic diagram of the structure of a systolic array proposed in Example 1 of the present application;

[0050] Figure 4 This is a schematic diagram of the structure of a systolic array proposed in Example 2 of this application;

[0051] Figure 5 A schematic structural diagram of an optional embodiment 1 of each processing unit in the systolic array proposed in this application;

[0052] Figure 6 A schematic structural diagram of an optional second embodiment of each processing unit in the systolic array proposed in this application;

[0053] Figure 7 This is a schematic diagram of the structure of each processing unit in the systolic array proposed in this application under the application scenario of the optional embodiment 2;

[0054] Figure 8A schematic diagram of the timing of data loading and calculation processes of the systolic array in the application scenario of the optional embodiment 2 of each processing unit in the systolic array proposed in this application;

[0055] Figure 9 A schematic structural diagram of an optional third embodiment of each processing unit in the systolic array proposed in this application;

[0056] Figure 10 A schematic structural diagram of an optional fourth embodiment of each processing unit in the systolic array proposed in this application;

[0057] Figure 11 This is a timing diagram of the data loading and calculation process of a single boundary PE in the application scenario of the optional embodiment 4 of each processing unit in the systolic array proposed in this application;

[0058] Figure 12 A schematic structural diagram of an optional fifth embodiment of each processing unit in the systolic array proposed in this application;

[0059] Figure 13 This is a schematic diagram of the structure of each processing unit in the systolic array proposed in this application under the application scenario of the optional embodiment 5;

[0060] Figure 14 A schematic diagram of the timing of data loading and calculation processes of the systolic array in the application scenario of the fifth optional embodiment of each processing unit in the systolic array proposed in this application;

[0061] Figure 15 A schematic flow chart of a control method for a systolic array provided in Example 1 of the present application;

[0062] Figure 16 A schematic flow chart of a control method for a systolic array provided in Example 2 of the present application;

[0063] Figure 17 A control device for a pulsating array proposed in an embodiment of the present application;

[0064] Figure 18 A schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation methods section of this application are only used to explain the specific embodiments of the present application and are not intended to limit this application. The embodiments of the present application are described below in conjunction with the drawings. It is known to those skilled in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0066] The terms "first", "second", etc. in the context of the present application and in the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same properties when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0067] This application can be applied to application fields such as matrix operations, digital signal processing, neural network acceleration, and image processing. The following uses the matrix multiplication or convolution of the convolutional layer in the neural network as an example to introduce the application scenarios of the product, but it is not limited to the application scenarios described in the following embodiments. In order to solve the problems in the background technology section, the embodiments of the present application provide a systolic array and its control method, accelerator, and electronic device. The following is a detailed introduction to the systolic array of the embodiment of the present application in conjunction with the accompanying drawings.

[0068] For ease of understanding, this application first describes the basic computing unit (Cube) loaded by the systolic array and its acquisition process. When a neural network performs a computational task (such as matrix multiplication or convolution), the two pieces of data involved are the global input data to be calculated (which refers to all input data A required to implement the computational task, such as image data to be processed in the field of image processing, or intermediate data generated during the image processing process) and the global weight data to be calculated (which refers to all weight data B required to implement the computational task). The global input data and global weight data to be calculated for the computational task can be referred to as global target data.

[0069] Typically, global target data is large, and on-chip SRAM (Static Random-Access Memory) is often unable to store all of the target data. Furthermore, the computational effort required to implement the computational task is significant, potentially consuming 90% of the entire neural network computation time. To improve computational speed, this application divides each global target data into layers and blocks based on the number of rows and columns in the systolic array. This results in basic computational units (i.e., the smallest data unit) that the systolic array can directly load to perform computational operations, significantly reducing the latency caused by frequent data block switching.

[0070] For example, refer to Figure 2The data layered and block processing method shown is explained using the example of an R×R structure of the systolic array. For global input data with a large amount of data (a large input data block), it can be layered and segmented according to the number of rows R of the systolic array. That is, starting from the first row, the layered segmentation is performed every R consecutive rows, and each layer of data is used as a block Tile (i.e., a small input data block), so that each Tile contains R rows of data. Afterwards, the column data of each Tile is divided into blocks according to the number of columns R of the systolic array. For example, starting from the first column, the block segmentation is performed every R consecutive columns, so that each cube contains R rows of data and R columns of data, so that the size of each cube is consistent with the size of the systolic array, that is, an R×R matrix, which is called the weight matrix and input matrix. This application can use the Cube as the basic computing unit of the systolic array, which is the basic unit of data transmission in the systolic array.

[0071] It should be noted that the hierarchical segmentation method for global target data includes but is not limited to the implementation method described in the previous paragraph. For example, the number of rows in each tile is not limited to R, but can also be a multiple of R (such as 2R or 3R, etc.). In this case, each tile is hierarchically divided into blocks so that the size of each resulting cube is R×R, etc. Therefore, the size of the tile data input to the systolic array of this application is proportional to the maximum computing bit width of the computing unit in the PE (such as MAC calculation). As the maximum computing bit width of the computing unit in the PE increases, larger tile data blocks can be divided into m cubes to input the systolic array. This application does not impose any restrictions on the value of m, which can be determined according to the circumstances.

[0072] In addition, the physical size of the systolic array, i.e., the number of rows and columns, and the on-chip storage capacity are limited by hardware resources. The number of rows and columns of the systolic array can be the same or different, and this application does not impose any restrictions on this. Depending on the circumstances, the embodiments of this application are described using a systolic array with a physical size of R×R=4×4 as an example. Systolic arrays of other physical sizes and their control methods are similar, and this application does not provide detailed examples. The aforementioned hierarchical and block processing can be performed off-chip. This fine-grained block partitioning of global target data leverages the parallel computing capabilities of the systolic array to maximize parallelism. Within the systolic array, pipelined data flow transmission allows for overlap of computation and communication, hiding latency and improving computational efficiency.

[0073] Reference Figure 3 , is a schematic diagram of the structure of a systolic array proposed in Example 1 of the present application, such as Figure 3 As shown, the systolic array may include a plurality of processing elements PE (which are composed of Figure 3 Each small square labeled Bij in the , is denoted as PE ij, i.e., the PE located in the i-th row and j-th column of the systolic array), multiple processing units PEs are regularly arranged and connected in a matrix form to efficiently perform specific types of computing tasks in the application fields listed above, such as matrix multiplication or convolution operations.

[0074] In such Figure 1 During the data loading process of the systolic array shown in the figure, when the systolic array loads Cube B (weight matrix), due to the following Figure 1 As shown, each column vector of the weight matrix is loaded from the head (or tail) boundary processing unit of each column in the systolic array (i.e., inputting the systolic array from the top to the bottom). This requires R (the number of rows in the systolic array) cycles to complete the loading of this weight matrix (which can be denoted as Bi, i.e., the i-th cube in each tile B after the global weight data is partitioned). Only after the loading of the weight matrix Bi is completed will the loading of the input matrix Ai (i.e., the i-th cube in each tile A after the global input data is partitioned) begin. Because this is also loaded unilaterally from the head or tail boundary processing unit of each row of the systolic array (e.g., loading Cube A from the left to the right of the systolic array), this also takes R cycles. As a result, the loading of the input matrix Ai is blocked by the weight matrix Bi. Especially for large tile data blocks, this leads to a large computational delay in the systolic array.

[0075] In order to shorten the loading time of each weight matrix and each input matrix of the systolic array, the present application proposes to load the data of each column vector of each matrix from two directions of the corresponding column of the systolic array at the same time. Figure 1 The method shown in FIG1 is to load the entire column vector data starting from only one direction of the corresponding column of the systolic array, which greatly shortens the loading delay of each matrix.

[0076] Based on this, in this application, the two boundary processing units (such as Figure 3 In the systolic array shown, the PEs in the first row (i.e., head) and the PEs in the last row (i.e., tail) in each column of PEs are, for example, Figure 3 The boundary PEs of the four headers marked with B30, B31, B32, and B33 can be recorded as PE in sequence. 00 PE 01 PE 02 PE 03 , and the four boundary PEs marked with B00, B01, B02, and B03 can be recorded as PE 30 PE 31 PE 32 PE 33 ), used to load weight matrices simultaneously (such as Figure 3 The weight data in the corresponding column vector in Cube B) is sequentially transmitted to the corresponding processing unit PE in the column along the column direction. The weight data transmission process between each PE in the systolic array is not described in detail in this application.

[0077] Take the physical size of the systolic array as R×R, R=2 as an example, the 0th to (R / 2-1)th row of each column vector in Cube B (such as Figure 3 The weight data of the first two rows shown in FIG1 (which can be recorded as the first weight data segment in the column vector, with a size of (R / 2) × R, as shown in FIG1 ( Figure 3 The weight data of the white background part of Cube B shown in the figure is input from the boundary PE at the end of the corresponding column (input from the bottom edge of the systolic array upwards). The R / 2~(R-1)th row of each column vector in Cube B (such as Figure 3 The weight data of the last two rows shown in FIG5 (which can be recorded as the second weight data segment in the column vector, with a size of (R / 2) × R, as shown in FIG5 (2)) Figure 3 The weight data of the gray background part of Cube B shown in FIG. 1 is input into the systolic array starting from the boundary PE of the corresponding column head (ie, input from the upper edge of the systolic array downward).

[0078] It can be seen that in the process of loading a weight matrix Cube B in the systolic array of this application, according to the method described above, at the same time, the PE 00 and PE 03 At the same time, load the corresponding weight data segment in the first column vector of the weight matrix, PE 10 and PE 13 At the same time, load the corresponding weight data segment in the second column vector of the weight matrix, PE 20 and PE 23 At the same time, the corresponding weight data segment in the third column vector of the weight matrix is loaded, PE 30 and PE 33 At the same time, the corresponding weight data segment in the fourth column vector of the weight matrix is loaded. A total of R / 2 cycles are required to complete the loading of the weight matrix. Figure 1 The loading method shown here reduces the loading time by half.

[0079] It should be understood that in Figure 3 In the implementation process of the systolic array loading weight data shown in FIG, each column vector in the weight matrix Cube B is divided into two parts, upper and lower, as shown in FIG. Figure 3Each column vector shown is evenly divided into two weight data segments (referred to as the first weight data segment and the second weight data segment, respectively) at the top and bottom. These two weight data segments are simultaneously input into the corresponding boundary processing unit from both the head and tail of the corresponding column. However, the number of rows R and columns R of each cube data (i.e., the physical size R of the systolic array) is not necessarily an even number, and this application does not restrict the shape of the systolic array. Equal column vector division is not possible when the number of rows or columns is an odd number.

[0080] For example, the size of the systolic array is R×C, where R and C are arbitrary integers. When R is an odd number, when the systolic array loads the weight matrix Cube B, each column vector is divided into two weight data segments of different sizes in the manner of (R-1) / 2 and (R+1) / 2. That is, at this time, the number of weight data contained in each of the two weight data segments is not equal. The time for the weight matrix to be completely loaded into the systolic array is based on the loading time of the larger weight data segment, which determines the loading time of the weight matrix.

[0081] The process of loading the input matrix by the systolic array is similar. Combined with the above analysis, the two boundary processing units (such as Figure 3 In the systolic array shown, the PEs in the first column (i.e., the head) and the PEs in the last column (i.e., the tail) in each row of PEs are, for example, Figure 3 The boundary PEs of the four headers marked with B30, B20, B10, and B00 can be recorded as PE 00 PE 10 PE 20 PE 30 , and the boundary PEs marked with the four tails B33, B23, B13, and B03 can be recorded as PE 03 PE 13 PE 23 PE 33 ), which is used to respond to the loaded weight data to be calculated and simultaneously load the different input data segments contained in the corresponding column vectors of the input matrix (Cube A, which corresponds to the weight matrix Cube B to which the loaded weight data belongs, for performing a calculation operation, such as Ai and Bi), and transmit the input data in the loaded input data segments in sequence to each processing element PE in the row along the row direction. The pipelined transmission process of input data between internal PEs in the systolic array is not described in detail in this application.

[0082] Based on this, each processing unit PE in the systolic array is used to perform calculation operations on the input data and weight data loaded into it to achieve the calculation task, that is, to complete the loading and calculation of the global target data according to the loading method described above to obtain the target calculation result of the calculation task.

[0083] Still taking the physical size of the systolic array as R×R, R=2 as an example, after the weight matrix Cube Bi is loaded, the corresponding input matrix Cube Ai begins to be loaded. During the loading process of the input matrix, Cube Ai can still divide each column vector into two input data segments by row, which are recorded as the first input data segment (such as Figure 3 The input data of the white background part in the middle) and the second input data segment (such as Figure 3 The input data in the gray background is shown in the figure. The size of these two input data segments is (R / 2) × R. The input data of rows 0 to (R / 2-1) of each column vector in Cube Ai (i.e., the first input data segment) is input from the boundary PE at the head of the corresponding row (from the left edge of the systolic array to the right). The input data of rows R / 2 to (R-1) of each column vector (i.e., the second input data segment) is input from the boundary PE at the end of the corresponding row (from the right edge of the systolic array to the left). A total of R / 2 cycles are required to complete the loading of the input matrix. Figure 1 The loading method shown above also reduces the loading time of the input matrix by half.

[0084] Among them, such as Figure 3 As shown, the first column vector of the input matrix Cube A corresponds to the fourth row of the systolic array, the second column vector corresponds to the third row of the systolic array, the third column vector corresponds to the second row of the systolic array, and the fourth column vector corresponds to the first row of the systolic array. Therefore, in the process of the systolic array of this application loading an input matrix Cube A, according to the method described above, at the same time, the PEs in the systolic array 30 and PE 33 At the same time, load the different input data segments corresponding to the first column vector of the input matrix, PE 20 and PE 23 At the same time, load the different input data segments corresponding to the second column vector of the input matrix, PE 10 and PE 13 At the same time, the corresponding different input data segments in the third column vector of the input matrix are loaded, and PE 00 and PE 03 At the same time, the different input data segments corresponding to the fourth column vector of the input matrix are loaded to increase the loading speed of the input array.

[0085] Similarly, it should be noted that Figure 3In the implementation process of the systolic array loading input data shown, each column vector in the input matrix CubeA is evenly divided into two parts, namely the first input data segment and the second input data segment, so as to be loaded simultaneously from the head and tail directions of the corresponding row. However, the present application is not limited to dividing each column vector in the input matrix CubeA into two input data segments evenly from the top and bottom, and an unequal division method can also be used. In particular, when the size of the systolic array is R×C, R and C are arbitrary integers, and when C is an odd number, each column vector in the input matrix CubeA can be divided into two input data segments of different sizes by using the methods of (C-1) / 2 and (C+1) / 2. At this time, the time when the input matrix is completely loaded into the systolic array is based on the loading time of the input data segment with the larger size, and the loading time of the input matrix is determined.

[0086] In summary, the systolic array proposed in the present embodiment no longer uses the method of loading the weight matrix Cube Bi and the input matrix Cube Ai from the head boundary PE of the corresponding column or row of the systolic array (i.e., a single direction). Instead, the matrix is divided into two data segments, and a column vector is simultaneously loaded from the two boundary PEs at the head and tail of the corresponding column / row of the systolic array (i.e., from two directions of the corresponding column / row). This significantly reduces the loading delay of the weight matrix and the input matrix. Moreover, it can be seen that this loading method is applicable to systolic arrays of any shape, and the acceleration effect is even more significant as the maximum computational bit width of the computational units in the PE is increased. This application does not provide detailed examples here.

[0087] It should be noted that the systolic array includes but is not limited to the implementation method described above of loading the input matrix from its row direction and loading the weight matrix from its column direction. The weight matrix can also be loaded from the row direction of the systolic array and the input matrix can be loaded from its column direction. This loading implementation method is similar to the loading implementation method described in the above embodiment. It can be based on the data operation relationship between the two matrices, that is, it is necessary to consider the operation relationship between the input data and the loaded weight data in the process of first loading the weight matrix and then loading the corresponding input matrix in the systolic array to ensure that the calculation operation between the input matrix and the weight matrix is correct. The implementation process is not described in detail in this application.

[0088] In the actual application of some embodiments, according to the loading method described above in this application, during the implementation process of the systolic array loading each matrix Cube, each boundary processing unit located at the edge of the systolic array can execute the loading logic instructions representing the matrix loading method described above to realize the loading of each Cube Bi and each Cube Ai, and it is not required to first split each column vector in the Cube Bi and Cube Ai to be loaded into two data segments before executing the data loading process. It should be noted that for the weight matrix and input matrix that implement a calculation operation, such as Cube Bi and each Cube Ai, the loading of the input matrix is after the weight matrix has been loaded, that is, the systolic array will start loading Cube Ai only after completing the loading of Cube Bi according to the loading method described above, so that during the loading process of Cube Ai, each PE uses the currently loaded input data and the loaded weight data to perform calculation operations.

[0089] Of course, this application can also first split each column vector in the Cube Bi and Cube Ai to be loaded into two data segments, and then execute the data loading process. Figure 4 As shown, the systolic array may further include a control unit 100 for communicating with at least a boundary processing unit (e.g., Figure 4 PE shown in Figure (a) 00 PE 01 PE 02 PE 03 PE 30 PE 31 PE 32 PE 33 PE 10 PE 20 PE 13 PE 23 ) are connected, and in response to the loading instructions for each column vector in the weight matrix or the input matrix, the two data segments into which the column vector to be loaded are divided are determined, and the two data segments are simultaneously transmitted to the two boundary processing units at the head and tail of the corresponding row or column in the corresponding row direction or column direction.

[0090] Among them, as to how each column vector in Cube Bi and Cube Ai is divided into two data segments, and how the two data segments into which each column vector is divided are input into the pulsation array at the same time, you can refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here. It should be noted that for Cube Bi and Cube Ai that perform the same calculation operation, the control unit 100 can first complete the division of each column vector, and then execute the data loading process. It is also possible to divide each column vector in the Cube into two data segments when determining the current Cube to be loaded; and when loading the next Cube, divide each column vector in the next Cube into two data segments, such as Figure 4 The split loading implementation shown is to load Cube B0 first, as shown in the following example. Figure 4 As shown in Figure (b), each PE in the matrix form in the systolic array stores a fixed weight data, such as PE 30 Storage B00, PE 32 Storage B02, PE 12 Storage B22, PE 23 B13 is stored in Cube A0, etc. Afterwards, the control unit divides each column vector in Cube A0 into two input data segments and controls the two boundary PEs of the corresponding row to load the two input data segments respectively.

[0091] In one possible implementation, different cache areas can be allocated within the control unit for both Cube A and Cube B, for independent caching of the two data segments after the splitting of each column vector, so that when the systolic array is controlled to load Cube Bi or Cube Ai, the data of the corresponding data segment can be read from the corresponding two cache areas at the same time (or the data selector can be controlled to select) and the two boundary PEs at the head and tail of the corresponding column / row can be input at the same time. Alternatively, the boundary PEs at the head and tail of the column / row direction of the systolic array can be controlled to simultaneously read the data in the allocated data segment from the corresponding cache area to shorten the data loading delay of the Cube. The detailed implementation process will not be described in detail in this application. This application does not limit how the control unit controls the boundary PE to realize the loading of each cube according to the data loading method proposed in this application.

[0092] In the systolic array described in the above embodiment, once each PE begins loading input data, it can perform a computation on the currently loaded input data (which belongs to Cube Ai) and the already loaded weight data (which belongs to Cube Bi). After completing the computation, it can transfer the input data to the next adjacent PE, which will then load the next input data and continue to perform computations on the next input data and the already loaded weight data (which is fixed before the computations of Cube Ai and Cube Bi are completed). After receiving the input data, the next PE performs computations on the input data and the already loaded weight data of the next PE (which is different from the weight data already loaded by the current PE).

[0093] For example, Figure 4 As shown, after the systolic array completes the loading of Cube B0, each PE in the first column and each PE in the last column of the systolic array load the corresponding input data at the same time. 00 PE while loading A03 03 Load A33. At this time, in the same cycle (recorded as T1), PE 00 Perform calculation operations (such as multiplication, etc.) on the loaded A03 and the loaded B30, PE 03 Perform calculations on the loaded A33 and the loaded B33, and enter the next cycle (recorded as T2), PE 00 Transfer A03 to PE 01 And load A13, PE 03 Transfer A33 to PE 02 And load A23, then in the next cycle, PE 00 Perform calculations on the loaded A13 and the loaded B30, PE 03 Perform calculations on the loaded A23 and the loaded B33. At the same time, PE 01 Perform calculations on the loaded A03 and the loaded B31, PE 02 Perform calculations on loaded A33 and loaded B32.

[0094] After two cycles, all input data in Cube A0 have been loaded into at least one PE in the systolic array, but the calculation of A0 and B0 has not been completed. Entering the next cycle (recorded as T3), PE 01 Transfer A03 to PE 02 and receive PE 02 A33 transmission makes PE 01 Perform calculations on the loaded A33 and the loaded B31. Similarly, PE 02 Transfer A33 to PE 01 While receiving PE 01The transmission of A03 makes PE 02 The loaded A03 and the loaded B32 are calculated. Entering the next cycle (denoted as T4), A03 will be transferred to PE 03 , A33 will be transferred to PE 00 , then PE 03 Perform calculations on the loaded A03 and the loaded B33, PE 00 The calculation operation is performed on the loaded A33 and the loaded B30. The input data flows to each PE in the row direction for calculation until all PEs in the row have completed the calculation operation, indicating that the calculation of Cube A0 is completed.

[0095] In one possible implementation, the control unit 100 may be a processor or logic circuit having the above-mentioned control function, such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA), or one or more combinations thereof, which may be determined in combination with actual application scenarios. This application does not impose any restrictions on the form or hardware structure of the control unit 100.

[0096] Preferably, in order to further reduce the loading delay and shorten the time that Cube Ai waits for Cube Bi to be loaded, each PE of the present application can independently store the weight data of the successively loaded Cube Bi and Cube B(i+1), so that when the PE completes the loading and data calculation of one data segment and waits for the data transmission of another data segment, the next Cube B(i+1) can be loaded. In this way, after completing the calculation of Cube Ai, there is no need to wait for R / 2 cycles to load Cube B(i+1), and Cube A(i+1) can be loaded directly.

[0097] Based on the above analysis, refer to Figure 5 FIG. 1 is a schematic diagram of an alternative embodiment of a first embodiment of each processing unit in a systolic array. Each processing unit PE may include a weight storage circuit 210, an input storage circuit 220, and a calculation circuit 230, wherein:

[0098] The weight storage circuit 210 can be used to connect with the calculation circuit 230 to independently store the loaded weight data from different weight matrices and send the weight data from the weight matrix currently to be calculated to the calculation circuit 230.

[0099] In the embodiment of the present application, combined with the above analysis, the weight storage circuit 210 can be configured with at least two independent storage areas. The weight data stored in different storage areas come from different weight matrices, thereby achieving safe isolation of weight data between different weight matrices, avoiding errors when reading weight data during subsequent calculations. For example, when calculating CubeA1, the weight data in CubeB1 is required to participate in the calculation (i.e., the calculation with CubeA1), rather than the weight data in CubeB0 or B2. Therefore, the present application proposes to independently store the weight data of CubeB0, CubeB1, CubeB2, etc. in the same cycle. When calculating CubeA1, the weight data is read from the storage space corresponding to CubeB1 to participate in the calculation, ensuring that the calculation operations are performed on CubeA1 and CubeB1.

[0100] Optionally, each of the above-mentioned storage areas can be composed of one or more storage media such as registers or buffers. The present application does not limit the number of independent storage areas contained in the weight storage circuit 210 in each PE, and can be determined comprehensively based on the maximum computing bit width of the PE's computing circuit 230 and the hardware resources of the device where the systolic array is located to avoid waste of resources.

[0101] It can be understood that, as analyzed above, when the weight data of a weight matrix stored in the weight storage circuit is used up, the systolic array can load the weight data of the next weight matrix and store the weight data in the storage area corresponding to the weight matrix that has been used up. Therefore, in combination with the above description of the control unit 100, that is, when the systolic array also includes a control unit for controlling the loading of the weight matrix and the input matrix, the control unit 100 can also be used to: in response to the weight data from the same weight matrix in the storage circuit being used up, control the systolic array to load the weight data of the next weight matrix, thereby reducing the time waiting for the weight matrix to be loaded and shortening the matrix loading delay.

[0102] The input storage circuit 220 can be connected to the calculation circuit 230 to store the input data loaded from the same input matrix (Cube A) and send the currently loaded input data to the calculation circuit 230. The calculation circuit 230 can be used to perform calculation operations on the corresponding weight data and input data received, such as performing calculation operations on the input data from CubeAi and the weight data from CubeBi. This application does not limit the type of calculation operation, which can be determined based on the task content of the calculation task.

[0103] In an embodiment of the present application, the input storage circuit 220 can be configured with a storage area to store the input data in the currently loaded input matrix, or it can be configured with multiple storage areas to respectively store the input data in different adjacently loaded input matrices, thereby realizing independent storage of input data from different input matrices. The implementation process is similar to the above-mentioned weight storage circuit 210's implementation method of independently storing weight data of different weight matrices, and the present application does not need to describe it in detail here.

[0104] It can be seen from this that when the input storage circuit 220 only stores input data from one input matrix in the same cycle, the PE will load the input data of the next input matrix and write it into the storage area for storage only after all the input data of the input matrix have been used up or the calculation operation has been completed.

[0105] Preferably, the input storage circuit 220 of the present application can be configured with two or more storage areas, and the number of the storage areas can be determined in combination with the computing power of the computing circuit 230 and the number of weight matrices from which the weight data stored in the same cycle of the weight storage circuit 210 come, so as to avoid insufficient computing power of the computing circuit 230 (such as the same cycle can only support the input data of one input matrix and the weight data of one weight matrix for computing operations), and also store the input data of multiple input matrices, resulting in a waste of storage resources of the input storage circuit 220. In this case, when the computing power of the computing circuit 230 is sufficient (such as the same cycle can support the input data of two or more input matrices and the weight data of the corresponding weight matrices for computing operations), the waiting time for loading the input matrix can be reduced to shorten the loading delay.

[0106] Based on this, when the input storage circuit 220 is configured with two or more storage areas, the computing circuit 230 needs to be configured with multiple computing units. The number of computing units can be the same as the number of independent storage areas in the input storage circuit 220, so as to achieve rational utilization of the storage resources and computing resources of the PE and reduce the computing delay of the systolic array.

[0107] Optional, combined Figure 4The connection relationship and functions of the control unit 100 shown in FIG. 1 are such that, in response to exhaustion of input data from the same input matrix in the input storage circuit 220, the control unit 100 can control the systolic array to load the input data of the next input matrix and write the input data of the next input matrix into the storage area corresponding to the exhausted input matrix. The storage area in the input storage circuit 220 can be constructed in the same manner as the storage area in the weight storage circuit 210, including but not limited to registers or buffers.

[0108] According to the above analysis, in some embodiments, the weight storage circuit 210 in each PE may include two storage areas, namely, two weight registers Reg B, referring to Figure 6 The structure diagram of the optional embodiment 2 of each processing unit in the systolic array shown in FIG. 1 is a block diagram of the weight storage circuit 210, which may include: a first weight register 211, a second weight register 212, and a first weight selector 213, wherein:

[0109] In this embodiment of the present application, first weight register 211 can store first weight data from a first weight matrix; second weight register 212 can be used to store second weight data from a second weight matrix. The first weight matrix and the second weight matrix are two weight matrices loaded sequentially by the systolic array, such as Cube Bi and CubeB (i+1), where i = 0, 1, 2, ..., (m-1), and m represents the number of cubes into which the tile is divided.

[0110] Based on this, the first weight selector 213 is used to be connected to the first weight register 211, the second weight register 212 and the calculation circuit 230 respectively, and in response to the control signal of the weight matrix to be calculated currently, select the weight register corresponding to the weight matrix to be calculated currently, read the weight data in the weight register and send it to the calculation circuit 230, so that the calculation circuit 230 performs calculation operations on the received weight data and input data.

[0111] In an optional embodiment, when the weight storage circuit 210 includes two weight registers, the input storage circuit 220 may include an input register Reg A, and the calculation circuit 230 may include a calculation unit. In this case, the structure of each processing unit PE is the same as Figure 6 The structure shown is similar to Figure 6 The calculation circuit 230 in FIG. 2 can be regarded as a calculation unit, and the input storage circuit 220 can be regarded as an input register Reg A.

[0112] In conjunction with the above description of the implementation process of loading the weight matrix and input matrix data into the systolic array, after each PE completes loading the first weight matrix Cube Bi, each PE can write the first weight data from the first weight matrix Cube Bi into the first weight register 212, completing the loading of Cube Bi. Then, the first input matrix Cube Ai can be loaded and written into the input register Reg A. At the same time, the second weight data from the second weight matrix Cube B (i+1) can also be loaded and written into the second weight register 212. The loading time of each matrix Cube is R / 2.

[0113] Among them, each PE is loaded with a first input data corresponding to a column vector from the first input matrix Cube Ai, which can be sent to the computing unit. At this time, the first weight selector 213 can choose to read the first weight data in the first weight register 212 and send it to the computing unit. In this way, the computing unit can perform calculation operations on the received input data and the first weight data. The PE sends the first input data after calculation to the next PE. The PE loads the next first input data from the first input matrix Cube Ai and still sends it to the input register Reg A for storage. When sending the next input data to the computing unit, since this application adopts a fixed-weight systolic array, the weight data received by the computing unit remains unchanged at this time, and the computing unit continues to perform calculation operations on the weight data and the next first input data until the calculation operations of all the first input data contained in a corresponding column vector in the first input matrix Cube Ai are completed. The PE will load the second input data corresponding to a column vector from the second input matrix Cube A(i+1), and repeat the above-mentioned loading and storage process of the first input data corresponding to a column vector from the first input matrix Cube Ai. This application will not elaborate on this.

[0114] It should be noted that, combined with Figure 4 Regarding the input data loading implementation process, since the present application divides a column vector into two input data segments and starts loading different input data segments simultaneously from the head and tail directions of the corresponding row of the systolic array, after R / 2 cycles, the input matrix has been loaded for the systolic array. However, at this time, one input data segment is only loaded by a portion of the PEs in the row, and the other portion of PEs has not yet loaded the input data in the input data segment. At this time, the calculation cycle of the input matrix has not yet ended. The input data in the input data segment continues to be transmitted to the other portion of PEs in the row direction, so that the other portion of PEs continues to perform calculation operations on the input data in the received input data segment and the loaded weight data, until the other portion of PEs completes the calculation of all input data in the input data segment, and the calculation cycle of the input matrix ends.

[0115] In some embodiments, in order to send weight data from different weight matrices to different weight registers for independent storage, the weight storage circuit of each PE may further include a weight distributor, which may be respectively connected to the first weight register 211 and the second weight register 212 to write the weight data loaded by the systolic array to a weight register in an idle state. In response to the computing unit completing a calculation operation, the weight data from the weight matrix loaded next by the systolic array may be distributed to the weight register corresponding to the calculation operation to replace the original weight data in the weight register.

[0116] Combined with the above analysis, we take the example of the systolic array (still taking the physical size of 4×4) where the first weight matrix loaded is B0, the second weight matrix is B1, and the calculation operation performed by each PE is Multiply Accumulate (MAC). Figure 7 A schematic diagram of the structure of an optional embodiment 2 of each processing unit in the systolic array is shown, in which the first weight register 211 can be represented as RegB0, the second weight register 212 can be represented as RegB1, the first weight selector 213 can be a MUX (Multiplexer), and the weight distributor can be a DMUX (De-multiplexer, a two-way distributor in this embodiment).

[0117] Based on this, in the implementation process of the systolic array loading weight matrix, for each PE, refer to Figure 7 The hardware structure of the PE shown, and Figure 8 The timing diagram of the data loading and calculation process of the systolic array shown in (b) is taken as an example to illustrate the scenario where TileA and TileB are both divided into 4 cubes and the physical size of the systolic array is 4×4. In this diagram, the DMUX responds to the control signal ctrl (which can come from the control unit 100 or the processor in the electronic device, etc.) and can send the weight data from the weight matrix B0 loaded by the current PE to the Reg B0 storage. After 2 cycles of loading the weight matrix B0, the weight data in the next loaded weight matrix B1 is sent to the Reg B1 storage. Figure 8 The four cycles in the traditional loading method shown in (a) shorten the loading delay.

[0118] Moreover, after the weight matrix B0 is loaded, as shown in Figure 8As shown in (b), the input data in the input matrix A0 will be loaded and sent to Reg A for storage. At the same time, during the calculation cycle of B0 and A0, MUX receives the selection signal for B0. In response to the selection signal, it selects Reg B0, which currently stores the weight data of B0, reads the weight data therein, and sends it to the input data stored in MAC and ReA for calculation until MAC completes the calculation of the four input data contained in a column vector of A0. It can be seen that Figure 8 As shown in (b), in this example, one calculation cycle of Cube A and Cube B is 6 cycles, which is relatively Figure 8 The 8 cycles of the traditional loading calculation method shown in (a) shorten the calculation delay of the matrix and improve the calculation efficiency.

[0119] Among them, such as Figure 8 As shown in Figure (b), combined with the above analysis, for the entire systolic array, after loading A0, the boundary PE that initially loaded the input data from A0 can begin loading the next input matrix, A1. At this point, this boundary PE has not yet completed loading and computing the other input data segment in the corresponding column vector of A0 (sent by the adjacent PE in the same row), and the computation cycle of A0 does not end until this boundary PE completes loading and computing all input data in this other input data segment. It should be noted that the weight data from A1 is loaded starting from the boundary PE. Since the weight data from B1 has already been loaded and stored in RegB1, there is no need to wait. This boundary PE can then compute the input data from A1 according to the method described above, and the computation cycle of A1 and B1 begins at this point.

[0120] Determine the calculation cycle of A0 and B0, such as Figure 8 As shown in Figure (b), from the second cycle to the eighth cycle, the weight data in RegB0 has been used up. The systolic array can then proceed to load the next weight matrix B2, sending the weight data from B2 to RegB0 (at this point, RegB0 can be called RegB2) for storage. The subsequent loading process is similar and will not be analyzed in detail in this application. This shows that each PE in the embodiment of this application uses an architecture with two weight registers and one computational unit. Due to the data loading method proposed in this application, which simultaneously loads different data segments of a column vector from the boundary PEs at both ends of each row / column of the systolic array, the data loading delay can be effectively shortened. This also shortens the computation cycle of one matrix for the entire systolic array, improving computational efficiency.

[0121] In some embodiments, in order to further improve the computational efficiency, each PE retains the weight storage circuit structure of the two weight registers, and proposes a computation circuit with dual computation units (such as a dual MAC structure). In this case, a dual cache structure is also used for the input matrix, i.e., two input registers. Figure 9 As shown in the structural diagram of the optional embodiment 3 of each processing unit in the systolic array, the computing circuit 230 may include a first computing unit 231 and a second computing unit 232, and the input storage circuit 220 may include a first input register 221 and a second input register 222, wherein the first computing unit 231 can be connected to the first input register 221, and the second computing unit 232 can be connected to the second input register 222.

[0122] In the embodiment of the present application, the first input register 221 is used to store each first input data from the first input matrix Cube Ai and send the currently loaded first input data to the first calculation unit 231. The second input register 222 is used to store each second input data from the second input matrix Cube A(i+1) and send the currently loaded second input data to the second calculation unit 232. The first calculation unit 231 can be used to perform a first calculation operation on the corresponding first input data and first weight data received; the second calculation unit 232 can be used to perform a second calculation operation on the corresponding second input data and second weight data received.

[0123] As analyzed above, the first weight data and the second weight data each come from different weight registers. For example, when the first weight data and the second weight data come from two weight matrices sequentially loaded by the systolic array, the first weight data comes from the first weight register, and the second weight data comes from the second weight register. It should be understood that when both the first input register 221 and the second input register 222 store input data, the first calculation unit 231 and the second calculation unit 232 can be in an active state to perform different calculation operations. The implementation process can be referred to in the example below.

[0124] Based on this, for a single PE, since it has two computing units, it has the ability to calculate the data of two sets of matrices at the same time. For example, the first computing unit is used to realize the data calculation from matrices Ai and Bi, and the second computing unit is used to realize the calculation of data from matrices A(i+1) and B(i+1) respectively. In this case, the two computing units can be in working state to complete different computing operations, thereby improving the computing resource utilization of a single PE.

[0125] For example, the first weight matrix loaded by the systolic array (still taking the physical size of 4×4 as an example) is B0, the second weight matrix is B1, and the calculation operation performed by each PE is Multiply Accumulate (MAC). Figure 10 The schematic diagram of the structure of the optional embodiment 4 of each processing unit in the systolic array shown in FIG. Figure 10 As shown, the first input register 221 can be represented as RegA0, the second input register 222 can be represented as RegA1, the first calculation unit 231 can be represented as MAC0, and the second calculation unit 232 can be represented as MAC1. The systolic array loads B0 and B1, and starts loading A0 after B0 is loaded. At the same time, MAC0 starts the data calculation of A0 and B0. This implementation process is the same as Figure 7 The loading calculation process of this part of the structure is similar. For the entire systolic array, the timing flow of this part is the same as Figure 8 The timing flow of the corresponding parts shown in Figure (a) (i.e., the data loading and calculation process of A0 and B0 in cycles 0-8, and the loading process of B1 and A1) is the same.

[0126] For the boundary PE that begins loading A1, the input data segment corresponding to the column vector in A0 it has loaded begins to be transferred to the PE on the other side of the row that has loaded another input data segment. After the input data from A0 is transferred from the boundary PE to the PE in the row, it does not receive the input data from A0 sent by the PE in the row, but instead receives the input data from A1. Therefore, for this boundary PE, MAC0 is idle during this period, and MAC1 begins to enter the working state. After two cycles, the input data from the other input data segment of A0 is transferred to the boundary PE along the row direction. MAC0 switches to the working state and continues to calculate until the calculation of each input data in the other input data segment is completed, ending the calculation cycle of A0 and B0.

[0127] Because a single PE has only two weight registers and the systolic array has fixed weights, the weight matrix must be loaded before loading its corresponding input matrix. Therefore, the systolic array can only begin loading B2's weight data after the first computational unit completes its computational operations, meaning the computation cycles for A0 and B0 have concluded, and the weight register RegB0 used to store B0's weight data is idle (i.e., sufficient storage resources exist, such as if the weight data from B0 has been cleared) or the original stored data can be replaced. During this period, the first computational unit MAC0 remains idle, waiting for B2 to be loaded and for A2's input data to begin loading. Only then will MAC0 enter an active state and begin performing computations on A2's input data and B2's weight data. Similarly, after the second computational unit MAC1 completes its computations on A1 and B1, it enters an idle state, but has not yet completed loading B3 and must continue waiting.

[0128] For example, in the scenario where both TileA and TileB are divided into 4 cubes and the systolic array adopts a 4×4 physical size, the boundary PE (such as PE 03 ) data loading and calculation process as an example. Figure 11 As shown in Figure (b), PE 03 After loading the data segment 1 of B0 (stored in RegB0), the data segment 1 of B1 (stored in RegB1) is loaded, and the data segment 1 of A0 (stored in RegA0) is loaded. MAC0 enters the working state and completes the loading and calculation of the data segment 1 of A0, but the data segment 2 of A0 has not yet been loaded from PE. 03 Other PEs in the same row are transferred to avoid waiting and will continue to load A1 data segment 1 (RegA1 storage). At this time, MAC1 enters the working state and MAC0 enters the idle state. After waiting for R / 2 cycles, A0 data segment 2 is transferred to PE 03 , PE 03 Continue to load A0 data segment 2 (RegA0 storage), and at the same time MAC0 enters the working state, and similarly waits for the transmission of A1 data segment 2. MAC1 enters the idle state and waits for R / 2 cycles before A1 data segment 2 is transmitted to PE 03 (RegA1 stores the input data for A1's data segment 2.) MAC1 enters the active state. Meanwhile, MAC0 completes the calculation for A0's data segment 2 and can begin loading B2's data segment 1 for storage in RegB0. After B2's data segment 1 is loaded, A1's data segment 2 is also loaded. MAC1 then enters the idle state. Since A2's data segment 1 begins loading, MAC0 enters the active state, and so on, until the calculation task is complete.

[0129] It should be understood that in Figure 7 In the case of configuring a MAC on a PE as shown in Figure 11 As shown in Figure (a), the MAC will alternately perform data calculations on A0 and A1. The MAC is always in working state, making full use of computing resources. When a single PE is configured with dual computing units, since there are only two weight registers in the PE, the timing flow of data transmission and calculation of the entire systolic array is the same as the timing flow of the PE with only one computing unit (e.g. Figure 8 As shown in Figure (b), for a single PE, the two MACs alternately enter the active state to perform calculations on the input data in RegA. Optionally, the two MACs in the PE can serve as each other's backup MACs. In this case, each RegA can switch between the two MACs. When MAC0 is idle and MAC1 is active, RegA1 connects to MAC1. If MAC1 experiences an abnormality, RegA1 can switch to MAC0, allowing MAC0 to take over the data calculation work for MAC1, avoiding interruption of MAC1's original data calculation operation and improving the efficiency and reliability of the calculation task.

[0130] In some embodiments, in order to further improve the computing efficiency, the present application can also configure three weight registers in the weight storage circuit of each PE. In order to make full use of the storage resources of the three registers, the computing circuit at this time still configures two computing units, and the input storage circuit configures two input registers accordingly. Based on this, refer to Figure 12 The schematic diagram of the structure of the optional embodiment 5 of each processing unit in the systolic array shown in FIG. Figure 9 Based on the structure of a single PE shown, the weight storage circuit 210 may further include: a third weight register 214 and a second weight selector 215, wherein:

[0131] The third weight register 214 is used to be connected to the first weight selector 213 and the second weight selector 215 respectively, and store the third weight data from the third weight matrix; the second weight selector 215 is used to be connected to the first weight register 211, the second weight register 212 and the third weight register 214 respectively, and in response to the control signal of the next weight matrix to be calculated, select the weight register corresponding to the next weight matrix to be calculated, read the weight data in the weight register and send it to the calculation circuit 230 (in this embodiment, the second calculation unit 232).

[0132] It can be seen that compared to Figure 9The PE structure and its operational performance are shown in the figure. Due to the addition of a weight register in this embodiment, this triple-cache mechanism allows for unrestricted and continuous preloading of the three cubes of Tile B (i.e., Bi, B(i+1), and B(i+2)). This ensures that the loading of the A(i+2) matrix is not constrained by the Ai matrix, accelerating data transmission and computation within the systolic array. Specifically, a single PE can simultaneously load the next Cube B while waiting for the next data segment of Cube A to be loaded. This reduces idle time within the compute unit, improves computational resource utilization, and shortens computational latency.

[0133] Among them, the selection and allocation of the weight matrices to be loaded to the three weight registers can be implemented by a weight allocator. In the embodiment of the present application, the weight allocator can be used to connect to each weight register respectively, and in response to the first calculation unit 231 or the second calculation unit 232 completing the corresponding calculation operation, the weight data of the weight matrix loaded next time from the systolic array is allocated to the weight register corresponding to the calculation operation to replace the original weight data in the weight register. In this way, when both the first input register 221 and the second input register 222 store input data, the first calculation unit 231 and the second calculation unit 232 can be in a working state to complete different calculation operations.

[0134] For example, Tile A and Tile B can be divided into 4 cubes, the systolic array adopts a physical size of 4×4, the first weight matrix loaded by the systolic array is B0, the second weight matrix is B1, the third weight matrix is B2, and the calculation operation performed by each PE is multiplication and accumulation MAC. Figure 10 On the basis of Figure 13 As shown, the added third weight register 214 can be represented as RegB2, the second weight selector 215 can be represented as MUX1, and the first weight selector 213 can be represented as MUX0. Figure 14 FIG. 1 is a timing diagram of the data loading and calculation process of the systolic array shown in FIG. 1 . In this embodiment, the data loading and calculation process of the systolic array is as follows:

[0135] According to the matrix data loading method described above (i.e., each column vector to be loaded is divided into two data segments, and different data segments are loaded simultaneously from both ends of the corresponding column / row, shortening the data loading cycle to R / 2 cycles), the systolic array first completes the loading of B0, which requires a total of R / 2 (R=4) cycles. From the next cycle, the matrix data loading method of the present application is still used to start loading the data of A0. At this time, the data of B1 can also be loaded, which requires R / 2 cycles. A0 and B1 are all input into the systolic array, and the MAC operation of A0 and B0 begins. However, at this time, not all PEs have loaded the data of the matrix, and still need to wait for data transmission between PEs within the systolic array.

[0136] Due to the dual-input buffer architecture, after input matrix A0 is fully loaded into the systolic array, the systolic array can begin loading A1. After R / 2 cycles, A1 is fully loaded into the systolic array. Meanwhile, the MAC operations for A0 and B0 are not yet complete. R cycles are required (i.e., R / 2 cycles are required to transfer the data segment loaded from the head boundary PE to the tail boundary PE, and then R / 2 cycles are required to remove this data segment from the systolic array) to complete the MAC operations for A0 and B0.

[0137] In the embodiment of the present application, since a three-weight cache structure is adopted and the loading of the weight matrix B2 is not constrained by the loading of the input matrix A1, B2 can be loaded after B1 is loaded, or B2 can be loaded after A1 is loaded. Preferably, considering the data transmission and calculation consistency of the entire systolic array, Figure 14 As shown, you can wait until A1 is loaded before starting to load B2. Then, continue using the data loading method of this application to load B3 and A2 simultaneously, and simultaneously start the MAC operation of A2 and B2. Similarly, load all the cubes in Tile A and Tile B to complete the MAC operation. After all of Cube A is loaded, it will take R cycles to complete the MAC result output.

[0138] It can be seen that Figure 14 As shown, this application starts loading A2 and calculating in the 8th cycle, relative to Figure 8 As shown in Figure (b), the method of loading and calculating A2 data starts at the 10th cycle. In this way, in the 8th to 10th cycles of this application, A1 and A2 use different MACs for simultaneous calculation, further shortening the calculation waiting time of the calculation circuit (i.e., calculation delay, reducing the idle time of MAC in a single PE), improving the utilization of computing resources, and shortening the time of the entire computing task (reduced from 18 cycles required by the above embodiment of this application to 16 cycles, and after testing, compared with Figure 1The control scheme shown in FIG. 28 requires 28 cycles. This embodiment reduces the delay of the computing task by 42.86%, thereby accelerating the computing task.

[0139] It should be noted that, as analyzed above, the number of matrix cubes that Tile A and Tile B can be split into is m. While the example above uses m=4, the number is not limited to 4. Furthermore, since the size of the tile data input to the systolic array is proportional to the maximum computational bit width of the MAC in a single PE, increasing the maximum computational bit width of the MAC in the PE allows for larger tile data blocks to be input (split into more cubes).

[0140] In actual situations, the above m may be an odd number or an even number. Figure 13 For the systolic array structure shown, the period T of the entire Tile A and Tile B calculation can be expressed by the following formula (1):

[0141] ; (1)

[0142] Relative to adopting Figure 1 The data loading method shown, and the PE uses dual weight cache and single MAC operation, the entire Tile A and Tile B calculation cycle It can be expressed as formula (2):

[0143] ; (2)

[0144] After comparison, it can be seen that the embodiment is relative to Figure 1 For the systolic array shown, the difference △T between the calculation cycles of Tile A and Tile B can be expressed as formula (3):

[0145] ; (3)

[0146] It can be seen that the systolic array proposed in the embodiment of the present application shortens the acceleration and computing delay, and divides the tiles with large amounts of data into cubes and then inputs them into the systolic array for calculation, which greatly saves the delay problem caused by the frequent switching of data blocks due to the participation of the entire tile data block in the calculation. The optimized delay is proportional to m and positively correlated with the physical size of the systolic array. As the physical size of the systolic array increases, the delay optimization effect becomes more significant. The present application does not limit the shape of the systolic array. It should be noted that the computing unit in a single PE includes but is not limited to MAC. When performing other types of computing operations, the computing unit can be a device or circuit that implements the computing operation. The computing control process based on this implementation is similar, and this application does not give detailed examples one by one.

[0147] In combination with the systolic array provided in the embodiment of the present application described above, a control method based on the systolic array will be described below.

[0148] Reference Figure 15 , is a flow chart of a control method of a systolic array provided in the first embodiment of the present application. The composition structure of the systolic array can refer to the relevant description of the systolic array embodiment above, and this embodiment will not be described in detail here. Based on this, Figure 5 As shown, the control method of the systolic array proposed in this embodiment may include but is not limited to:

[0149] Step S151, in response to the weight loading instruction, starting from the two boundary processing units at the head and tail of each column in the column direction of the systolic array, simultaneously load different weight data segments contained in the corresponding column vector data in the weight matrix;

[0150] Step S152 , in response to the input load instruction, starting from the two boundary processing units at the head and tail of each row in the row direction of the systolic array, different input data segments corresponding to the column vectors in the input matrix are loaded simultaneously; wherein, for a weight matrix and an input matrix that implement a calculation operation, the input matrix is loaded after the weight matrix is loaded.

[0151] In the embodiment of the present application, the weight matrix and input matrix are both basic computational units obtained by hierarchically partitioning the corresponding global target data to be calculated for the computational task based on the number of columns and rows of the systolic array. Each processing unit in the systolic array can perform computational operations on the received input data and weight data to achieve the computational task. Regarding the data loading implementation process of steps S151 and S152, as well as the computational process of the processing unit, please refer to the description of the corresponding parts of the systolic array embodiment above, and this embodiment will not be repeated here.

[0152] It can be seen that compared with the traditional control method of the systolic array, when loading the weight matrix and the input matrix, each column vector therein can only be loaded from one boundary PE of the corresponding column / row of the systolic array. In contrast, the present application simultaneously loads the two boundary PEs at the head and tail of the column / row, that is, loading different data segments of a column vector from both ends of the same row / column of the systolic array at the same time, thereby shortening the data loading delay and thus improving the computing efficiency.

[0153] Reference Figure 16 , which is a flow chart of a control method for a systolic array provided in the second embodiment of the present application. This embodiment can describe the calculation process of a single PE of the systolic array, such as Figure 16 As shown, based on the method for loading weight data and input data described in the above embodiment, each PE may further include but is not limited to performing the following steps:

[0154] Step S161, in response to a control signal for a weight matrix to be calculated, selecting a weight register corresponding to the weight matrix to be calculated as a target weight register; weight data stored in different weight registers in each processing unit come from different weight matrices;

[0155] Step S162, reading the weight data in the target weight register and sending it to the computing unit in the idle state;

[0156] Step S163: performing calculation operations on the received weight data and the input data from the corresponding input register through the calculation unit to achieve the calculation task.

[0157] Regarding the implementation process of step S161-step S162, reference can be made to the working process described based on the structure of a single PE in the above embodiment. Figure 5 、 Figure 6 、 Figure 9 、 Figure 12 The timing flow of steps S161 and S162 for PEs with different structures shown in the figures may differ. The implementation process can be referred to the description of the corresponding parts of the above embodiments, and this application does not elaborate on this. As analyzed above, as the number of weight registers and computing units in a PE increases, the computational latency optimization effect is improved, which further accelerates the computational tasks of the systolic array.

[0158] Reference Figure 17 , is a control device for a pulsating array proposed in an embodiment of the present application, such as Figure 17 As shown, the control device may include but is not limited to:

[0159] a weight data loading module 171 for simultaneously loading different weight data segments contained in the corresponding column vector data in the weight matrix, starting from two boundary processing units at the head and tail of each column in the column direction of the systolic array, in response to a weight loading instruction;

[0160] an input data loading module 172 for simultaneously loading different input data segments corresponding to column vectors in the input matrix, starting from two boundary processing units at the head and tail of each row in the row direction of the systolic array, in response to an input loading instruction; wherein, for a weight matrix and an input matrix implementing a calculation operation, the input matrix is loaded after the weight matrix has been loaded;

[0161] The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

[0162] Optionally, the control device of the systolic array may further include:

[0163] a selection module configured for each of the processing units to select, in response to a control signal for a weight matrix to be calculated, a weight register corresponding to the weight matrix to be calculated as a target weight register; wherein the weight data stored in different weight registers in each of the processing units come from different weight matrices;

[0164] A reading module, configured to read the weight data in the target weight register and send the data to a computing unit in an idle state;

[0165] A calculation module is used to perform calculation operations on the received weight data and the input data from the corresponding input register through the calculation unit to achieve the calculation task.

[0166] An embodiment of the present application further provides an accelerator, which may include at least one systolic array. The structure of the systolic array may refer to the relevant description of the systolic array embodiment above, and will not be described in detail in this embodiment.

[0167] In practical applications, accelerators can be dedicated hardware accelerators that implement specific computing tasks, such as matrix operation accelerators like TPUs (Tensor Processing Units), or signal processing accelerators like real-time signal processing chips used in radar and communication baseband processing. They can also be processing chips used in scientific computing and linear algebra applications. They can also be cryptography acceleration chips used to optimize password generation and accelerate steps. In the field of image / video processing, for example, the accelerators disclosed in this application can be convolution accelerators that implement image filtering, edge detection, feature extraction, and other tasks, as well as video codecs. In the field of AI applications, the accelerators disclosed in this application can also be AI accelerators that accelerate neural network inference / training. As can be seen, the accelerators disclosed in this application can take different product forms in different application fields, and this application does not impose any restrictions on this.

[0168] It should be noted that when an accelerator includes multiple systolic arrays, the structures of each systolic array and each PE thereof can be identical, such as the structures described in any of the systolic array embodiments above. Alternatively, the structures of different systolic arrays can be different, and combinations of the various systolic array structures provided in the systolic array embodiments above can be employed, but are not limited to these. This can be determined based on actual computing requirements and is not detailed in this application.

[0169] Reference Figure 18, is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application. The electronic device can be one or more terminal devices such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, a robot or a business terminal, or a server. The electronic device is described as a server as an example. Figure 18 As shown, the electronic device may include but is not limited to: at least one memory 181, at least one processor 182 and at least one of the above-mentioned systolic arrays 183. The composition structure and functional implementation of the systolic array can refer to the description of the corresponding parts of the above embodiments, and will not be repeated in this embodiment.

[0170] At least one memory 181, at least one processor 182, and at least one systolic array 183 can communicate with each other via a bus. As needed, an I / O interface can be connected to the bus to achieve control requirements for the systolic array. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 18 The bus is represented by only one bidirectional line, but this does not mean that there is only one bus or one type of bus.

[0171] The memory 181 can be used to store multiple computer instructions for computing tasks; the processor 182 can load and execute the multiple computer instructions stored in the memory 181 in response to an execution request for the computing task, thereby implementing the various steps of the control method of the systolic array proposed in the embodiment of the present application. The implementation process can refer to the description of the corresponding part of the method embodiment above.

[0172] In the embodiment of the present application, the memory 181 may include a storage medium such as a floppy disk, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. The processor 182 may include any one or more of a central processing unit (CPU), an image signal processor (ISP), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0173] It should be understood that Figure 18 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiment of the present application. In actual applications, the electronic device may include Figure 18 More or fewer components as shown, or a combination of certain components, such as when the electronic device is a terminal device, may also include image collectors (such as cameras), microphones, speakers, displays, various sensors, antennas, power modules, radio frequency components, external ports and other input / output components, etc., which can be determined according to the processing function requirements, and this application does not give detailed examples one by one.

[0174] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0175] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0176] Furthermore, the various embodiments in this specification are described in a progressive or parallel manner, with each embodiment focusing on its differences from the other embodiments. Reference will be made to the description of the devices, accelerators, and electronic devices disclosed in the embodiments for similarities and differences. Since these devices, accelerators, and electronic devices correspond to the systolic arrays disclosed in the embodiments, their descriptions are relatively simplified. For relevant details, refer to the description of the systolic array.

Claims

1. A systolic array comprising a plurality of processing units, wherein the plurality of processing units are regularly arranged and connected in a matrix form; Two boundary processing units located at the head and tail of each column in the column direction of the systolic array are used to simultaneously load different weight data segments contained in the corresponding column vectors in the weight matrix, and sequentially transmit the weight data in the loaded weight data segments to the corresponding processing units in the column along the column direction; two boundary processing units located at the head and tail of each row in the row direction of the systolic array, configured to, in response to the weight data to be calculated having been loaded, simultaneously load different input data segments contained in corresponding column vectors in the input matrix, and sequentially transmit the input data in the loaded input data segments to each processing unit in the row in the row direction; The processing unit is configured to perform a calculation operation on the input data and the weight data loaded therein to achieve a calculation task; The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

2. The systolic array of claim 1 , further comprising: a control unit, configured to be connected to at least a boundary processing unit located at an edge of the systolic array, and to determine, in response to a load instruction for each column vector in the weight matrix or the input matrix, two data segments into which the column vector to be loaded is divided, and to simultaneously transmit the two data segments to two boundary processing units located at the head and tail of a corresponding row or column in a corresponding row direction or column direction, respectively.

3. The systolic array of claim 1 , wherein each of the processing units comprises: A weight storage circuit, an input storage circuit, and a calculation circuit, wherein: The weight storage circuit is configured to be connected to the calculation circuit, independently store the loaded weight data from different weight matrices, and send the weight data from the weight matrix currently to be calculated to the calculation circuit; an input storage circuit, connected to the calculation circuit, for storing each input data loaded from the same input matrix, and sending the currently loaded input data to the calculation circuit; The calculation circuit is used to perform calculation operations on the received corresponding weight data and input data.

4. The systolic array according to claim 3 , wherein the systolic array further comprises a control unit for controlling loading of the weight matrix and the input matrix, the control unit is further configured to: In response to the data from the same matrix in the storage circuit being used up, the systolic array is controlled to load the data of the corresponding next matrix.

5. The systolic array of claim 3 , wherein the weight storage circuit comprises: a first weight register, a second weight register, and a first weight selector, wherein: The first weight register is used to store first weight data from the first weight matrix; The second weight register is used to store second weight data from a second weight matrix; The first weight selector is used to be connected to the first weight register, the second weight register and the calculation circuit respectively, and in response to the control signal of the weight matrix to be calculated currently, select the weight register corresponding to the weight matrix to be calculated currently, read the weight data in the weight register and send it to the calculation circuit.

6. The systolic array of claim 5 , wherein the weight storage circuit further comprises: a third weight register and a second weight selector; The third weight register is used to be connected to the first weight selector and the second weight selector respectively, and store third weight data from a third weight matrix; The second weight selector is used to be connected to the first weight register, the second weight register and the third weight register respectively, and in response to the control signal of the next weight matrix to be calculated, select the weight register corresponding to the next weight matrix to be calculated, read the weight data in the weight register and send it to the calculation circuit.

7. The systolic array according to claim 5 or 6, wherein the input storage circuit comprises a first input register and a second input register, and the calculation circuit comprises a first calculation unit and a second calculation unit, wherein: The first input register is configured to be connected to the first computing unit, store each first input data from the first input matrix, and send the first input data currently loaded to the first computing unit; The second input register is configured to be connected to the second computing unit, store each second input data from the second input matrix, and send the currently loaded second input data to the second computing unit; The first calculation unit is configured to perform a first calculation operation on the received corresponding first input data and first weight data; The second calculation unit is configured to perform a second calculation operation on the received corresponding second input data and second weight data; The first weight data and the second weight data come from different weight registers.

8. The systolic array of claim 7 , wherein the weight storage circuit further comprises: a weight allocator, connected to each weight register, and configured to allocate weight data of a weight matrix loaded next from the systolic array to the weight register corresponding to the calculation operation in response to the first calculation unit or the second calculation unit completing the corresponding calculation operation, so as to replace the original weight data in the weight register; Wherein, when both the first input register and the second input register store input data, the first calculation unit and the second calculation unit can both be in a working state to complete different calculation operations.

9. A method for controlling a systolic array, the systolic array comprising a plurality of processing units, the plurality of processing units being regularly arranged and connected in a matrix form, the method comprising: In response to a weight loading instruction, starting from two boundary processing units at the head and tail of each column in the column direction of the systolic array, respectively, different weight data segments contained in the corresponding column vector data in the weight matrix are loaded simultaneously; In response to an input load instruction, starting from two boundary processing units at the head and tail of each row in the row direction of the systolic array, simultaneously load different input data segments corresponding to column vectors in the input matrix; wherein, for a weight matrix and an input matrix that implement a calculation operation, the input matrix is loaded after the weight matrix has been loaded; The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

10. The method according to claim 9, further comprising: Each of the processing units selects a weight register corresponding to the weight matrix to be calculated currently as a target weight register in response to a control signal for the weight matrix to be calculated currently; the weight data stored in different weight registers in each of the processing units come from different weight matrices; Reading the weight data in the target weight register and sending it to the computing unit in an idle state; The calculation unit performs a calculation operation on the received weight data and the input data from the corresponding input register to achieve the calculation task.

11. An accelerator, comprising: At least one systolic array according to any one of claims 1 to 8.

12. An electronic device comprising: At least one memory, at least one processor, and at least one systolic array according to any one of claims 1 to 8, wherein: The memory is used to store a plurality of computer instructions for computing tasks; The processor is configured to load and execute the computer instructions in response to a request to execute the computing task, and implement the following steps: In response to a weight loading instruction, starting from two boundary processing units at the head and tail of each column in the column direction of the systolic array, respectively, different weight data segments contained in the corresponding column vector data in the weight matrix are loaded simultaneously; In response to an input load instruction, starting from two boundary processing units at the head and tail of each row in the row direction of the systolic array, respectively, different input data segments corresponding to column vectors in the input matrix are loaded simultaneously; Wherein, for a weight matrix and an input matrix for implementing a calculation operation, the input matrix is loaded after the weight matrix has been loaded; The weight matrix and the input matrix are both basic computing units obtained by layering and blocking the corresponding global target data to be calculated that participate in the computing task based on the number of columns and rows of the systolic array.

13. The electronic device according to claim 12, wherein the processor is further configured to: In response to a control signal for a weight matrix to be calculated currently, selecting a weight register corresponding to the weight matrix to be calculated currently as a target weight register; the weight data stored in different weight registers in each of the processing units come from different weight matrices; Reading the weight data in the target weight register and sending it to the computing unit in an idle state; The calculation unit performs a calculation operation on the received weight data and the input data from the corresponding input register to achieve the calculation task.

Images