Systolic array, systolic array control method, systolic array control equipment and storage medium

Abstract

The invention provides a systolic array, a systolic array control method, systolic array control equipment and a storage medium. The systolic array comprises a control module, a calculation module, an input buffer module and an output buffer module, the calculation module comprises at least N levels of processing unit array sub-modules; the output end of the control module is connected with the control ends of the processing unit array sub-modules, the input ends of the processing unit array sub-modules are connected with the input buffer module, the output ends of the processing unit array sub-modules are connected with the output buffer module, and except the last processing unit array sub-module, the output ends of the other processing unit array sub-modules are further connected with the next processing unit array sub-modules; and the control module is used for determining at least one target processing unit array sub-module from the N processing unit array sub-modules according to an operation requirement of to-be-calculated data input by the external control module, and controlling the target processing unit array sub-module to process the to-be-calculated data. According to the method, the computing resource utilization rate of the systolic array is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a systolic array, a control method and device for the systolic array, and a storage medium. Background Art

[0002] With the explosive growth of available data on the Internet, deep learning has been increasingly widely and importantly applied to complex problems in areas such as image recognition, speech processing, language translation, natural language processing, and self-driving cars. Deep learning often requires high computing power as support, and currently the computing module is mainly designed through the structure of systolic arrays. The systolic array is a pipelined high-speed computing structure with a large throughput, which is composed of multiple processing elements (PEs) arranged according to certain interconnection rules. However, the current computing modules with systolic array structures have the problems of low data throughput and low utilization of processing elements.

[0003] Therefore, how to improve the data throughput of computing modules and the utilization of processing units in the systolic array structure is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a systolic array, a control method, a device and a storage medium for the systolic array, so as to solve the problem in the prior art of how to improve the data throughput of a computing module and the utilization rate of a processing unit in a systolic array structure.

[0005] In a first aspect, the present application provides a systolic array, comprising: a control module, a computing module, an input buffer module, and an output buffer module; the computing module comprises at least N levels of processing unit array submodules;

[0006] The input end of the control module is used to connect with the external control module, the output end of the control module is connected with the control end of the processing unit array submodule, the input end of the processing unit array submodule is connected with the input buffer module, and the output end is connected with the output buffer module. Except for the last one, the output ends of the remaining processing unit array submodules are also connected with the processing unit array submodules of the next level; N is an integer greater than or equal to 2;

[0007] The input buffer module is used to store the data to be calculated;

[0008] The control module is used to determine at least one target processing unit array submodule from the N processing unit array submodules according to the computing requirements of the data to be calculated input by the external control module, and control the target processing unit array submodule to process the data to be calculated.

[0009] Optionally, the processing unit array submodules of levels 0 to N-1 in the computing module include: a processing unit array, a first multiplexer, and a second multiplexer; the processing unit array submodule of level N includes: a processing unit array, a first multiplexer;

[0010] The input end of the first multiplexer is connected to at least one storage unit in the input buffer module, the output end of the first multiplexer is connected to the input end of the processing unit array, and the output end of the processing unit array is connected to the processing unit array in the processing unit array submodule of the next level;

[0011] The input end of the second multiplexer is respectively connected to the output end of the processing unit array of the processing unit array submodule where it is located, and the output end of the processing unit array of the processing unit array submodule located after the processing unit array submodule where it is located;

[0012] The output end of the control module is connected to the control end of the first multiplexer and the control end of the second multiplexer respectively;

[0013] The control module is specifically used to control the first multiplexer of the target processing unit array submodule to conduct the target input path and the second multiplexer to conduct the target output path, so that the target processing unit array submodule processes the data to be calculated;

[0014] The first multiplexer is specifically configured to select at least one target storage unit from the storage units in the input buffer module to receive the data to be calculated according to the control of the control module;

[0015] The second multiplexer is specifically used to output the target operation result from the processing unit array of the processing unit array sub-module where the second multiplexer is located, or to output the target operation result from the processing unit array sub-module located after the processing unit array sub-module where the second multiplexer is located, according to the control of the control module.

[0016] Optionally, the processing unit array submodule further includes: a routing unit;

[0017] The output end of the processing unit array is connected to the processing unit array in the processing unit array submodule of the next level through the routing unit;

[0018] The input end of the second multiplexer is respectively connected to the output end of the routing unit of the processing unit array submodule where it is located, and is connected to the output end of the routing unit of the processing unit array submodule located after the processing unit array submodule where it is located.

[0019] Optionally, the processing unit array submodule further includes: a weight cache unit;

[0020] The output end of the weight cache unit is connected to the control end of the processing unit array;

[0021] The weight cache unit is used to configure the weight parameters used when the processing unit array calculates data.

[0022] Optionally, the processing unit array submodule further includes: a pipeline register; the output end of the processing unit array is connected to the input end of the routing unit through the pipeline register.

[0023] Optionally, the processing unit array comprises: M rows and Q columns of processing units; both M and Q are integers greater than or equal to 2;

[0024] The input end of the first processing unit in each row is connected to the output end of the first multiplexer, the first output end of the processing unit is connected to the input end of the next processing unit in its row, and the second output end of the processing unit is connected to the input end of the next processing unit in its column; the second output end of the processing unit in the last row is connected to the corresponding input end of the second multiplexer.

[0025] Optionally, the control module is used to determine at least one target processing unit array sub-module corresponding to the matrix structure from the N processing unit array sub-modules according to the matrix structure corresponding to the computing requirements of the data to be calculated input by the external control module, and control the target processing unit array sub-module to process the data to be calculated.

[0026] In a second aspect, the present application provides a control method for a systolic array, applied to the systolic array described in any one of the first aspects, comprising:

[0027] Determine at least one target processing unit array submodule from the N processing unit array submodules of the systolic array according to the computing requirements of the to-be-computed data input by the external control module;

[0028] Control the target processing unit array submodule to process the data to be calculated.

[0029] In a third aspect, the present application provides an accelerator, comprising a systolic array as described in any one of the first aspects, a data input interface, and a data output interface.

[0030] In a fourth aspect, the present application provides a system-level chip, wherein the system-level chip comprises a systolic array as described in any one of the first aspects.

[0031] In a fifth aspect, the present application provides an electronic device, comprising: a processor, a communication interface, and a memory; the processor is communicatively connected to the communication interface and the memory respectively;

[0032] The memory stores computer-executable instructions;

[0033] The communication interface communicates and interacts with external devices;

[0034] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the second aspect.

[0035] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the control method of the systolic array as described in the second aspect.

[0036] In a seventh aspect, the present application provides a computer program product, which, when executed by a processor, is used to implement the control method of the systolic array as described in the second aspect.

[0037] The systolic array, control method, device and storage medium provided in the present application realize the function of automatically adjusting the structure of the systolic array according to the computing requirements of the data to be calculated by determining the target processing unit array submodules required for the data to be calculated from the computing module and the arrangement of these target processing unit array submodules according to the computing requirements of the data to be calculated, thereby improving the computing resource utilization of the systolic array. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0039] Figure 1 A schematic diagram of the structure of a processing unit provided in this application;

[0040] Figure 2 A schematic diagram of the structure of a fixed systolic array computing module provided in this application;

[0041] Figure 3 A schematic diagram of the structure of a Mesh computing module provided in this application;

[0042] Figure 4 A schematic diagram of the structure of a systolic array provided in an embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application;

[0044] Figure 6 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application;

[0045] Figure 7 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application;

[0046] Figure 8 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application;

[0047] Fig. 9 A schematic diagram of the structure of a processing unit array submodule provided in an embodiment of the present application;

[0048] Fig.10 A schematic flow chart of a control method for a systolic array provided in an embodiment of the present application;

[0049] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0050] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0051] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0052] For ease of understanding, the processing unit (PE) involved in the present application is first described. The PE involved in the present application may be a multiplying accumulator (MAC). Figure 1 This is a schematic diagram of the structure of a processing unit provided in this application. Figure 1As shown, the PE includes a multiplier and an accumulator. The PE can receive data to be calculated, and input the data to be calculated into the multiplier and the next PE connected to the PE. When the data to be calculated is input into the next PE connected to the PE, the data to be calculated is also the calculation data of the next PE. Before receiving the data to be calculated, the multiplier needs to be configured with a weight parameter used when calculating the data to be calculated, and the multiplier can calculate the data to be calculated according to the weight parameter. After the multiplier completes the calculation, it outputs the calculation result to the accumulator. The accumulator generates the cumulative local sum of the PE according to the calculation result output by the multiplier and the cumulative local sum of the upstream PE connected to the PE (that is, the cumulative local sum of all PEs before the PE is received from the previous PE of the PE), and outputs the cumulative local sum to the downstream PE (that is, the cumulative local sum of the PE is output to the next PE of the PE).

[0053] At present, the main Figure 1 The PE shown in the figure forms a systolic array to form a computing module for deep learning. The prior art mainly implements the computing module through the following two solutions:

[0054] Solution 1: Fixed systolic array computing module.

[0055] Figure 2 This is a structural diagram of a fixed systolic array computing module provided in this application. Figure 2 As shown, the operation module includes: a fixed systolic array formed by n*n PEs, a weight buffer, an input buffer, and an output buffer.

[0056] The input buffer includes n memories (banks) for input, and the output buffer includes n memories for output. The number of PEs included in the fixed systolic array can be determined according to actual needs. For example, taking the architecture of the TPUv1 to TPUv4 series as an example, the n of a single computing module in the TPUv1 architecture is 256, that is, the fixed systolic array is a 256*256 PE array; the n of a single computing module in the TPUv2~TPUv4 architecture is 128, that is, the fixed systolic array is a 128*128 PE array.

[0057] Reference Figure 2In the operation module, the weight parameters required for calculating the calculation data are first pre-filled from the weight buffer for each PE participating in the calculation. The weight parameters can be, for example, the weight registers filled in the PE. The calculation data is input into the PE array from each memory in the input buffer, so that the accumulated local sum is obtained by the PE calculation, and the accumulated local sum is outputted downward along each column in the PE array until it is outputted to the memory in the output buffer for accumulation and storage.

[0058] Under this structure, the size of the systolic array in the operation module is fixed. If the number of rows and columns of the matrix of the calculation data to be calculated is smaller than the number of rows and columns of the systolic array, a large number of PEs will be idle, resulting in a low PE utilization problem. In addition, if the fixed systolic array is large and the sequence of calculation data input into the input buffer is short, the fixed systolic array needs to frequently replace the configured weights, resulting in a long time consumed for configuring weights and calculations, and a low data throughput problem.

[0059] Solution 2: Wireless mesh network (Mesh) computing module.

[0060] Figure 3 This is a schematic diagram of the structure of a Mesh computing module provided in this application. Figure 3 As shown, the operation module includes: Mesh network, multiple PEs. For each network node of the Mesh network, multiple PEs can be connected in an all-to-all network form, and the network node can distribute data with one or more PEs connected to the node in a unicast, multicast, or broadcast manner.

[0061] However, for this type of computing module, the network nodes and the PE clusters under the network nodes are all-to-all network architectures, which are not scalable. Therefore, the more PE mesh systolic array computing modules are included, the larger the mesh network is. The expansion of the mesh network will lead to the problem of low efficiency in data distribution. In addition, the mesh systolic array computing module is mainly optimized for convolution operations, and its computing efficiency for the currently common large-scale matrix multiplication operations is lower than that of the systolic array.

[0062] In view of this, the present application provides a systolic array, which controls the arrangement of the processing unit array according to the computing requirements of the computing data through a control module, and generates a processing unit array arrangement result that is consistent with the computing requirements of the computing data, thereby improving the utilization rate of the processing units in the systolic array and the data throughput of the systolic array.

[0063] Below, the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail through specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Figure 4 A schematic diagram of the structure of a systolic array provided in an embodiment of the present application. Figure 4 As shown, the systolic array includes: a control module, a calculation module, an input buffer module, and an output buffer module.

[0065] The computing module includes at least N levels of processing unit array submodules. The input end of the control module is used to connect to the external control module, the output end of the control module is connected to the control end of the processing unit array submodule, the input end of the processing unit array submodule is connected to the input buffer module, and the output end is connected to the output buffer module. Except for the last one, the output ends of the remaining processing unit array submodules are also connected to the processing unit array submodules of the next level, and N is an integer greater than or equal to 2.

[0066] The input buffer module is used to store data to be calculated. The data to be calculated can be directly transmitted to the input buffer module by external hardware, or the input buffer module can read from the memory or cache according to the instruction of the external hardware. The data to be calculated can be data to be convolutionally calculated, or data to be matrix multiplication calculated. Exemplarily, for example, the central processing unit (CPU) can control the data to be calculated to be input into the input buffer module.

[0067] The control module is used to determine at least one target processing unit array submodule from the N processing unit array submodules according to the computing requirements of the data to be calculated input by the external control module, and control the target processing unit array submodule to process the data to be calculated. The number of the target processing unit array submodules is less than N.

[0068] Among them, the operation requirement can indicate the processing unit array submodules required for the data to be calculated, and the arrangement of these processing unit array submodules. For example, the operation requirement can be determined according to the matrix structure of the data to be calculated, so as to determine the target processing unit array submodule for processing the data to be calculated and corresponding to the matrix structure from the idle processing unit array submodules according to the matrix structure. The arrangement of the processing unit array submodules can be, for example, parallel connection in the same row, or series connection in the same column, or connection in a matrix form of multiple rows and columns, etc. Alternatively, the operation requirement can also directly indicate the number of parallel connection and / or series connection of the processing unit array submodules required for the data to be calculated.

[0069] When the processing unit array submodules are arranged in parallel in the same row, taking 4 processing unit array submodules in parallel as an example, the input buffer module can input the data to be calculated into the 4 processing unit array submodules respectively, and after each processing unit array submodule completes the calculation, the calculation result is output to the output buffer module for storage.

[0070] When the arrangement of the processing unit array submodules is to be connected in series in the same column, taking 4 processing unit array submodules connected in series as an example, the input buffer module can input the data to be calculated into the 4 processing unit array submodules respectively, and after the first processing unit array submodule completes the calculation, the calculation result is input into the second processing unit array submodule, and the second processing unit array submodule performs calculation according to the calculation result of the first processing unit array submodule and the data to be calculated input by the input buffer module, and obtains the calculation result of the second processing unit array submodule. Similarly, when the last (i.e., the fourth) processing unit array submodule completes the calculation, the calculation result is output to the output buffer module for storage.

[0071] When the processing unit array submodules are arranged in a matrix form with multiple rows and columns, the above-mentioned serial connection method in the same column is used for calculation and output in each column, and the above-mentioned parallel connection method in the same row is used for calculation and output in each row.

[0072] Exemplarily, if the matrix structure of the data to be calculated is data of M rows (e.g., 256 rows) and Q columns (e.g., 256 columns), then the processing unit array submodules that can satisfy the 256*256 matrix structure can be selected from the idle processing unit array submodules according to the data of the M*Q (256*256) matrix structure. If the structure of each processing unit array submodule is M1 rows (e.g., 16 rows) and Q1 columns (e.g., 16 columns) (i.e., 16*16), then 16 columns of processing unit array submodules can be selected horizontally from the idle processing unit array submodules, and 16 rows of processing unit array submodules can be selected vertically to calculate the data to be calculated with a matrix structure of 256*256.

[0073] In a possible implementation, the computation requirement may indicate the number of processing unit array submodules required for the data to be calculated. The control module determines a target processing unit array submodule that meets the number from the idle processing unit array submodules according to the number of processing unit array submodules required for the data to be calculated.

[0074] In another possible implementation, the computation requirement may directly indicate the target processing unit array submodule required for the data to be calculated.

[0075] The systolic array provided in the embodiment of the present application realizes the function of automatically adjusting the structure of the systolic array according to the computing requirements of the data to be calculated, by determining the target processing unit array submodules required for the data to be calculated from the computing module and the arrangement of these target processing unit array submodules according to the computing requirements of the data to be calculated, thereby improving the computing resource utilization of the systolic array.

[0076] The specific structure of the processing unit array submodule in the embodiment of the present application is described in detail below. Figure 5 A schematic diagram of another systolic array structure provided in an embodiment of the present application. Figure 5 As shown, the processing unit array submodules of levels 0 to N-1 in the computing module include: a processing unit array, a first multiplexer, and a second multiplexer; the Nth level processing unit array submodule includes: a processing unit array and a first multiplexer.

[0077] For any level of processing unit array submodules in level 0 to N-1 of the computing module, at least a first multiplexer and a processing unit array are included; wherein the input end of the first multiplexer is connected to at least one storage unit in the input buffer module included in the systolic array, the output end of the first multiplexer is connected to the input end of the processing unit array, and the output end of the processing unit array is connected to the processing unit array in the processing unit array submodule of the next level.

[0078] For any level of processing unit array submodules among level 0 to level N-1 processing unit array submodules in the computing module, at least a second multiplexer and a processing unit array are included; the input end of the second multiplexer is respectively connected to the output end of the processing unit array of the processing unit array submodule where it is located, and the output end of the processing unit array of the processing unit array submodule located after the processing unit array submodule where it is located.

[0079] The output end of the control module is connected to the control end of the first multiplexer and the control end of the second multiplexer respectively.

[0080] The control module is specifically used to control the first multiplexer of the target processing unit array submodule to conduct the target input path and the second multiplexer to conduct the target output path, so that the target processing unit array submodule processes the data to be calculated.

[0081] The first multiplexer is specifically configured to select at least one target storage unit from the storage units in the input buffer module to receive the data to be calculated according to the control of the control module;

[0082] The second multiplexer is specifically used to output the target operation result from the processing unit array of the processing unit array submodule where the second multiplexer is located, or to output the target operation result from the processing unit array submodule located after the processing unit array submodule where the second multiplexer is located, according to the control of the control module.

[0083] The control module controls the first multiplexer of the target processing unit array submodule to determine and select the data to be calculated output by the storage unit that meets the calculation requirements from the data to be calculated output by at least one storage unit in the input buffer module, and outputs the data to be calculated to the target processing unit array submodule, thereby opening the target input path.

[0084] The control module controls the second multiplexer to determine and select a target operation result output by the second multiplexer from a plurality of operation results received from the output end of the processing unit array of the processing unit array submodule where the second multiplexer is located, and the output end of the processing unit array of the processing unit array submodule located after the processing unit array submodule where the second multiplexer is located, according to the operation requirements, and output the target operation result to the corresponding memory in the output buffer module, thereby opening the target output path.

[0085] Optionally, the processing unit array may include: processing units with M rows and Q columns, the processing units may be the aforementioned MAC, and M and Q are both integers greater than or equal to 2. The input end of the first processing unit in each row is connected to the output end of the first multiplexer, the first output end of the processing unit is connected to the input end of the next processing unit in the row, and the second output end of the processing unit is connected to the input end of the next processing unit in the column; the second output end of the processing unit in the last row is connected to the corresponding input end of the second multiplexer.

[0086] For example, in a possible implementation, when the processing unit array submodules are arranged in parallel in the same row, the structure of the systolic array can be as follows: Figure 6 shown. Figure 6 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application. Figure 6 As shown, the control module controls the first multiplexer of the target processing unit array submodule to obtain the data to be calculated from the memory corresponding to the target processing unit array submodule in the input buffer module.

[0087] Among them, the memory can output the data to be calculated to the first multiplexer of all target processing unit array sub-modules, and the processing unit array of each target processing unit array sub-module can process the received data to be calculated respectively, and directly output the calculation result of the target processing unit array sub-module processing the data to be calculated.

[0088] The control module controls the second multiplexer to receive the target operation result from the output end of the processing unit array of the processing unit array submodule where the second multiplexer is located, and outputs the target operation result to the corresponding memory in the output buffer module, thereby opening the target output path.

[0089] In this implementation, when the first memory inputs the data to be calculated into the target processing unit array submodule, any second memory other than the first memory can simultaneously obtain the data to be calculated that needs to be input into the target processing unit array submodule in the next round. The second memory and the first memory are both the above-mentioned memories that can output the data to be calculated to the first multiplexers of all target processing unit array submodules. For example Figure 6 In the example, bank0 is the first memory and bank1 is the second memory. Compared with the current method of waiting for the systolic array to complete the calculation before obtaining the next round of data to be calculated, this implementation method forms a double buffer mechanism through the first memory and the second memory, further improving the data throughput of the systolic array and shortening the data processing time.

[0090] In another possible implementation, when the processing unit array submodules are arranged in series in the same column, the structure of the systolic array can be as follows: Figure 7 shown. Figure 7 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application. Figure 7 As shown, the control module controls the first multiplexer of the target processing unit array submodule to obtain the data to be calculated from the memory corresponding to the target processing unit array submodule in the input buffer module.

[0091] Wherein, the memory only outputs the data to be calculated to the first multiplexer of the target processing unit array submodule corresponding to the memory, and after the processing unit array of each target processing unit array submodule processes the received data to be calculated, the processing unit array outputs the calculation result to the processing unit array of the next target processing unit array submodule and the second multiplexer of the target processing unit array submodule. The processing unit array of the next target processing unit array submodule calculates the calculation result of the next target processing unit array submodule according to the calculation result and the data to be calculated received from the corresponding memory, and outputs the calculation result to the next target processing unit array submodule, the second multiplexer of the next target processing unit array submodule, and the second multiplexer before the second multiplexer. By analogy, the last target processing unit array submodule directly outputs the calculation result to the second multiplexer output buffer module of the last target processing unit array submodule and all the previous target processing unit array submodules.

[0092] The control module controls the second multiplexer of any target processing unit array submodule to output the operation result of the last target processing unit array submodule, and outputs the target operation result to the corresponding memory in the output buffer module, thereby conducting the target output path.

[0093] In this implementation, when there are other target processing unit array submodules upstream of the target processing unit array submodule, the target processing unit array submodule needs to receive the calculation results of other target processing unit array submodules upstream before it can perform calculations. When the target processing unit array submodule is in the waiting process, if there are target data to be calculated in the subsequent round of data to be calculated that can be completed by the structure of only the target processing unit array submodule and the target processing unit array submodule downstream, the target data to be calculated can be first input into the target processing unit array submodule and the target processing unit array submodule downstream, and the target data to be calculated can be processed first. When the calculation results of other target processing unit array submodules upstream of the target processing unit array submodule are transmitted to the target processing unit array submodule, the data to be calculated in this round is processed, thereby realizing the time division multiplexing function of the systolic array, further improving the computing resource utilization of the systolic array, and improving the efficiency of data batch processing.

[0094] In another possible implementation, when the arrangement of the processing unit array submodules is in the form of a matrix connected in multiple rows and columns, the structure of the systolic array can be as follows: Figure 8 shown. Figure 8 A schematic diagram of the structure of another systolic array provided in an embodiment of the present application. Figure 8 As shown, the target processing unit array submodule includes target processing unit array submodule 1, target processing unit array submodule 2, target processing unit array submodule 3, and target processing unit array submodule 4. The corresponding relationship between the processing unit array, the first multiplexer, and the second multiplexer included in the target processing unit array submodule is shown in Table 1 below:

[0095] Table 1

[0096]

[0097] Among them, the control module controls the first multiplexer 1 and the first multiplexer 3 to control the output of the data to be calculated from the memory bank0 in the input buffer, and controls the first multiplexer 2 and the first multiplexer 4 to control the output of the data to be calculated from the memory bank1 in the input buffer, thereby completing the conduction of the target input path.

[0098] The control module controls the second multiplexer 1 to only output the calculation result from the processing unit array 2, and controls the second multiplexer 2 to only output the calculation result from the processing unit array 4, so as to complete the conduction of the target output path.

[0099] In this arrangement, the target processing unit array submodule 1 and the target processing unit array submodule 3 are arranged in series in the same column, the target processing unit array submodule 2 and the target processing unit array submodule 4 are arranged in series in the same column, the target processing unit array submodule 1 and the target processing unit array submodule 2 are arranged in parallel in the same row, and the target processing unit array submodule 3 and the target processing unit array submodule 4 are arranged in parallel in the same row, thereby connecting to form a 2-row and 2-column matrix of processing unit array submodules, so that the four target processing unit array submodules can process the data to be calculated through the processing unit arrays therein.

[0100] The systolic array provided in the embodiment of the present application controls the first multiplexer of the target processing unit array submodule to conduct the target input path and the second multiplexer to conduct the target output path through the control module, and changes the arrangement of the target processing unit array submodule to adapt to the computing requirements of the data to be calculated to process the data to be calculated, thereby improving the computing resource utilization of the systolic array. In addition, a double buffering mechanism can be constructed, or a time division multiplexing function can be realized to further improve the data throughput of the systolic array and improve the data processing efficiency of the systolic array.

[0101] In a possible implementation, the processing unit array submodule may further include: a routing unit. The output end of the processing unit array is connected to the processing unit array in the next level processing unit array submodule through the routing unit. The input end of the second multiplexer is respectively connected to the output end of the routing unit of the processing unit array submodule where it is located, and is connected to the output end of the routing unit of the processing unit array submodule located after the processing unit array submodule where it is located.

[0102] The routing unit is used to transmit the calculation result output by the processing unit array connected to the routing unit to the processing unit array in the processing unit array submodule of the next level, or to the second multiplexer. The control module can control the output path of the routing unit according to the calculation requirements of the data to be calculated, so as to determine whether the calculation result is output to the processing unit array in the processing unit array submodule of the next level or to the second multiplexer.

[0103] Since it is possible to set weight parameters for the multiplication operation when performing matrix multiplication and convolution calculations, when the weight parameters need to be set, it is also necessary to set weight parameters corresponding to the data to be calculated for the processing unit array.

[0104] In a possible implementation, the weight parameter is pre-set in the processing unit array, for example, in a memory in the processing unit array, and the weight parameter is obtained when processing the data to be calculated. Optionally, multiple sets of weight parameters can be stored in the processing unit array, and a set of weight parameters corresponding to the data to be calculated is determined from the multiple sets of weight parameters according to the computational requirements of the data to be calculated.

[0105] Another possible implementation is that the weight parameter is obtained from the computational requirements of the data to be calculated and configured into the processing unit array. In this implementation, the processing unit array submodule may also include: a weight cache unit, the output end of which is connected to the control end of the processing unit array. The weight cache unit is used to configure the weight parameters used when the processing unit array calculates the data. Before the processing unit array submodule processes the data to be calculated, the control module can input the weight parameter to the weight cache unit according to the computational requirements, and indicate the mapping relationship between the weight parameter and the processing unit array, so that the weight parameter is input to the processing unit array that conforms to the mapping relationship to configure the weight parameter for the processing unit array, so that when the subsequent processing unit array submodule starts to process the data to be calculated, the correct weight parameter can be used.

[0106] In any of the above implementations, the processing unit array submodule may further include: a pipeline register. Fig. 9 This is a schematic diagram of the structure of a processing unit array submodule provided in an embodiment of the present application. Fig. 9 As shown, the output end of the processing unit array is connected to the input end of the pipeline register, the output end of the pipeline register is connected to the input end of the routing unit, and the output end of the routing unit is connected to the processing unit array in the processing unit array submodule of the next level. The input end of the second multiplexer is respectively connected to the output end of the routing unit of the processing unit array submodule where it is located, and the output end of the routing unit of the processing unit array submodule located after the processing unit array submodule where it is located. The pipeline register is used to accumulate and store the operation results output by the processing unit array, so as to avoid that after the processing unit array completes data processing, its operation results cannot be temporarily transmitted to the processing unit array submodule of the next level, resulting in the processing unit array needing to wait for the completion of transmission before the next round of data processing, and there is a problem that the processing main frequency of the systolic array is low, so by temporarily storing the operation results, the data processing efficiency of the systolic array is improved.

[0107] Fig.10 A flow chart of a control method of a systolic array provided in an embodiment of the present application. Fig.10 As shown, the method is applied to the systolic array described in any of the above embodiments. The method includes:

[0108] S101 , determining at least one target processing unit array submodule from N processing unit array submodules of a systolic array according to a computational requirement of data to be calculated input by an external control module.

[0109] Among them, the external control module can be, for example, a CPU, and the computing requirements can include, for example, the number of processing unit array sub-modules required to process the data to be calculated or the identification of the target processing unit array sub-module, as well as the arrangement of the target processing unit array sub-modules (for example, the arrangement is related to the matrix structure of the data to be calculated).

[0110] In a possible implementation, the computation requirement may indicate the number of processing unit array submodules required for the data to be calculated. The control module determines a target processing unit array submodule that meets the number from the idle processing unit array submodules according to the number of processing unit array submodules required for the data to be calculated.

[0111] In another possible implementation, the computation requirement may directly indicate the target processing unit array submodule required for the data to be calculated.

[0112] S102, controlling the target processing unit array submodule to process the data to be calculated.

[0113] The target processing unit array submodule is controlled to determine an arrangement of the target processing unit array submodule according to the computing requirement, and the data to be calculated is processed in the arrangement.

[0114] Among them, the step of controlling the target processing unit array sub-module to process the data to be calculated in the control method of the systolic array provided in the embodiment of the present application can be implemented by the function of the systolic array in the above-mentioned structural embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0115] The present application also provides an accelerator, which includes the systolic array in the above embodiment, as well as a data input interface and a data output interface. The data input interface is used to receive data to be calculated, and the data output structure is used to output the calculation result of the systolic array processing the data to be calculated.

[0116] The present application also provides a system on chip (SoC), which includes the systolic array in the aforementioned embodiment.

[0117] Fig.11 The structure diagram of an electronic device provided in an embodiment of the present application. The electronic device is used to execute the control method of the systolic array mentioned above, for example, it can be the speed controller mentioned above, or a terminal device with the systolic array or SoC mentioned above. Fig.11As shown, the electronic device 1100 may include: at least one processor 1101 , a memory 1102 , and a systolic array 1103 .

[0118] The memory 1102 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0119] The memory 1102 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0120] The processor 1101 is used to execute the computer-executable instructions stored in the memory 1102 to implement the method described in the above method embodiment. The processor 1101 may be a CPU, or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0121] In a specific implementation, if the systolic array 1103, the memory 1102 and the processor 1101 are implemented independently, the systolic array 1103, the memory 1102 and the processor 1101 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0122] Optionally, in a specific implementation, if the systolic array 1103, the memory 1102 and the processor 1101 are integrated on a chip, the systolic array 1103, the memory 1102 and the processor 1101 can communicate through an internal interface.

[0123] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0124] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one processor of a computing device can read the execution instruction from the readable storage medium, and at least one processor executes the execution instruction so that the computing device implements the control method of the systolic array.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A systolic array, characterized in that: The systolic array includes: a control module, a calculation module, an input buffer module, and an output buffer module; the calculation module includes at least N levels of processing unit array submodules; The input end of the control module is used to connect with the external control module, the output end of the control module is connected with the control end of the processing unit array submodule, the input end of the processing unit array submodule is connected with the input buffer module, and the output end is connected with the output buffer module. Except for the last one, the output ends of the remaining processing unit array submodules are also connected with the processing unit array submodules of the next level; N is an integer greater than or equal to 2; The input buffer module is used to store the data to be calculated; The control module is used to determine at least one target processing unit array submodule from the N processing unit array submodules according to the computing requirements of the data to be calculated input by the external control module, and control the target processing unit array submodule to process the data to be calculated.

2. The systolic array according to claim 1, characterized in that: The processing unit array submodules of levels 0 to N-1 in the computing module include: a processing unit array, a first multiplexer, and a second multiplexer; the processing unit array submodule of level N includes: a processing unit array, a first multiplexer; The input end of the first multiplexer is connected to at least one storage unit in the input buffer module, the output end of the first multiplexer is connected to the input end of the processing unit array, and the output end of the processing unit array is connected to the processing unit array in the processing unit array submodule of the next level; The input end of the second multiplexer is respectively connected to the output end of the processing unit array of the processing unit array submodule where it is located, and the output end of the processing unit array of the processing unit array submodule located after the processing unit array submodule where it is located; The output end of the control module is connected to the control end of the first multiplexer and the control end of the second multiplexer respectively; The control module is specifically used to control the first multiplexer of the target processing unit array submodule to conduct the target input path and the second multiplexer to conduct the target output path, so that the target processing unit array submodule processes the data to be calculated; The first multiplexer is specifically configured to select at least one target storage unit from the storage units in the input buffer module to receive the data to be calculated according to the control of the control module; The second multiplexer is specifically used to output the target operation result from the processing unit array of the processing unit array sub-module where the second multiplexer is located, or to output the target operation result from the processing unit array sub-module located after the processing unit array sub-module where the second multiplexer is located, according to the control of the control module.

3. The systolic array according to claim 2, characterized in that: The processing unit array submodule further includes: a routing unit; The output end of the processing unit array is connected to the processing unit array in the processing unit array submodule of the next level through the routing unit; The input end of the second multiplexer is respectively connected to the output end of the routing unit of the processing unit array submodule where it is located, and is connected to the output end of the routing unit of the processing unit array submodule located after the processing unit array submodule where it is located.

4. The systolic array according to claim 2 or 3, characterized in that: The processing unit array submodule further includes: a weight cache unit; The output end of the weight cache unit is connected to the control end of the processing unit array; The weight cache unit is used to configure the weight parameters used when the processing unit array calculates data.

5. The systolic array according to claim 2 or 3, characterized in that: The processing unit array submodule further includes: a pipeline register; the output end of the processing unit array is connected to the input end of the routing unit through the pipeline register.

6. The systolic array according to claim 2 or 3, characterized in that: The processing unit array comprises: M rows and Q columns of processing units; both M and Q are integers greater than or equal to 2; The input end of the first processing unit in each row is connected to the output end of the first multiplexer, the first output end of the processing unit is connected to the input end of the next processing unit in its row, and the second output end of the processing unit is connected to the input end of the next processing unit in its column; the second output end of the processing unit in the last row is connected to the corresponding input end of the second multiplexer.

7. The systolic array according to claim 1, characterized in that: The control module is used to determine at least one target processing unit array submodule corresponding to the matrix structure from the N processing unit array submodules according to the matrix structure corresponding to the operation requirements of the data to be calculated input by the external control module, and control the target processing unit array submodule to process the data to be calculated.

8. A control method for a systolic array, characterized in that: The method is applied to the systolic array according to any one of claims 1 to 7, and the method comprises: Determine at least one target processing unit array submodule from the N processing unit array submodules of the systolic array according to the computing requirements of the to-be-computed data input by the external control module; Control the target processing unit array submodule to process the data to be calculated.

9. An accelerator, characterized in that: The accelerator comprises a systolic array as described in any one of claims 1 to 7, a data input interface, and a data output interface.

10. A system-on-chip, characterized in that: The system-on-chip comprises a systolic array as claimed in any one of claims 1-7.

11. An electronic device, characterized in that: The electronic device comprises a systolic array as claimed in any one of claims 1-7.

Images