Multiply-accumulate operation cluster and data processing method

Abstract

The present application discloses a multiplication and accumulation device operation cluster and a data processing method. The multiplication and accumulation device operation cluster includes a plurality of multiplication and accumulation device units, each of which is used to complete the multiplication and accumulation operation of at least one group of data to be processed. When the calculation dimension of the data to be processed is less than the total dimension of the multiplication and accumulation device unit, the multiplication and accumulation operation of each group of data to be processed is completed by at least one multiplication and accumulation device sub-unit in the multiplication and accumulation device unit, and the multiplication and accumulation operation result of the data to be processed is output; and / or, when the data to be processed includes a plurality of sequentially arranged data blocks, the multiplication and accumulation operation of a data block is completed by a plurality of multiplication and accumulation device sub-clusters in the multiplication and accumulation device operation cluster, and the multiplication and accumulation operation result of the data block is output to the next sequentially arranged multiplication and accumulation device sub-cluster, until the multiplication and accumulation operation of all data blocks is completed, and the multiplication and accumulation operation result of the data to be processed is output. The multiplication and accumulation device operation cluster improves the utilization rate of the multiplication and accumulation device operation unit.

Description

Technical Field

[0001] The present application belongs to the field of digital circuit technology, and specifically relates to a multiplier-accumulator operation cluster and a data processing method. Background Art

[0002] The Multiply and Accumulate (MAC) is used to perform multiplication and accumulation operations such as vector multiplication, matrix multiplication, and vector-matrix multiplication. It is an extremely important computing subsystem in processors such as coprocessors, digital signal processors, and central processing units (CPUs), especially the most basic computing unit in artificial intelligence (AI) accelerators.

[0003] The MAC in an AI accelerator can generally implement the dot product of X groups of M-dimensional vectors. For example, the multiplication and addition operations of two groups of M-dimensional vectors (vector A and vector B) are completed by first completing the operations of A1 •B1, A2 •B2 ...AX •BX, and then adding the products together. The processing dimension of each vector (A or B) is represented by M. Each data may be an integer or floating point number of different bit widths. Taking the input as an 8-bit integer as an example, the computing power of each clock cycle of the MAC is M times X times 2 (2 is understood as two operations of a multiplication and an addition) of 8-bit integer operations. However, in actual applications, the utilization rate of the MAC operation unit is low due to some reasons. For example, when the processing dimension of vector A or vector B is less than the dimension M of the multiplication accumulator (the dimension of the multiplication accumulator represents the number of multipliers contained), some multipliers will be idle when processing data. Only when this processing is completed can the idle multipliers be used for the next time, resulting in low utilization of the multipliers in the multiplication accumulator. For example, if the input data of the multiplier-accumulator is not prepared in time or the output data of the multiplier-accumulator is not output in time, the line will be blocked and the multiplier-accumulator cannot be used next time, which will also lead to low utilization of the multiplier-accumulator operation unit.

[0004] Currently, there is no good circuit structure that can solve the problem of low utilization of the multiplication and accumulation unit. Summary of the invention

[0005] In view of this, the present application provides a multiplier-accumulator operation cluster and a data processing method, the main purpose of which is to solve the problem of low utilization rate of the existing multiplier-accumulator operation units.

[0006] To solve the above problems, the present application provides a multiplication-accumulation operation cluster, the multiplication-accumulation operation cluster includes a plurality of multiplication-accumulation units, each of the multiplication-accumulation units is used to complete a multiplication-accumulation operation of at least one group of data to be processed, each group of the data to be processed includes two multi-dimensional data, each of the multiplication-accumulation units includes a plurality of multiplication-accumulation sub-units, each of the multiplication-accumulation sub-units is used to complete a multiplication-accumulation operation and output a multiplication-accumulation operation result, wherein,

[0007] When the calculation dimension of two of the multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulation unit, the multiplication-accumulation operation of each group of the data to be processed is completed by at least one multiplication-accumulation sub-unit in the multiplication-accumulation unit, and the multiplication-accumulation operation result of the data to be processed is output; and / or, when the data to be processed includes a plurality of sequentially arranged data blocks, the multiplication-accumulation operation of one data block is completed respectively by a plurality of multiplication-accumulation sub-clusters in the multiplication-accumulation operation cluster, and the multiplication-accumulation operation result of the data block is output to the next sequentially arranged multiplication-accumulation sub-cluster, until the multiplication-accumulation operation of all the data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

[0008] In one embodiment of the present application, optionally, the dimensions of the multiplication accumulator sub-units in the same multiplication accumulator unit are the same, the sum of the dimensions of all the multiplication accumulator sub-units in each of the multiplication accumulator units is the same as the total dimension of the multiplication accumulator unit, and the dimensions of the multiplication accumulator sub-units in multiple different multiplication accumulator units are the same as or different from each other.

[0009] In one embodiment of the present application, optionally, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication accumulator sub-unit in at least one multiplication accumulator unit, a target multiplication accumulator unit for operating the data to be processed is determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-unit in the multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the multiplication accumulator sub-unit in the target multiplication accumulator unit.

[0010] In one embodiment of the present application, optionally, when the calculation dimension of a group of data to be processed is greater than the dimension of the multiplication and accumulation sub-units in all the multiplication and accumulation units and less than the total dimension of each of the multiplication and accumulation units, the multiplication and accumulation operations of the data to be processed are completed by multiple multiplication and accumulation sub-units in the multiplication and accumulation units.

[0011] In one embodiment of the present application, optionally, when the computational dimension of the data to be processed is greater than the dimension of the multiplication accumulator subunits in all the multiplication accumulator units and less than the total dimension of each of the multiplication accumulator units, a target multiplication accumulator for operating the data to be processed and a target number of multiplication accumulator subunits in the target multiplication accumulator are determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator subunits in each multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the target number of multiplication accumulator subunits in the target multiplication accumulator.

[0012] In one embodiment of the present application, optionally, at least one multiplier accumulator sub-unit in the multiplier accumulator operation cluster serves as a multiplier accumulator sub-cluster.

[0013] In one embodiment of the present application, optionally, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimension of the multiplication and accumulation sub-unit in any multiplication and accumulation unit, each multiplication and accumulation sub-unit in the multiplication and accumulation unit is used as a multiplication and accumulation cluster, and the multiplication and accumulation operation of one data block is completed respectively by multiple multiplication and accumulation sub-units in the multiplication and accumulation unit, and the multiplication and accumulation operation result of the data block is output to the next multiplication and accumulation sub-unit arranged in sequence, until the multiplication and accumulation operation of all the data blocks is completed, and the multiplication and accumulation operation result of the data to be processed is output.

[0014] In one embodiment of the present application, optionally, the multiplication and accumulation unit includes a plurality of data input modules, a plurality of multiplication operation modules, a plurality of multi-stage addition operation modules and a plurality of data output modules, wherein one multiplication operation module, one multi-stage addition operation module and a plurality of data output modules constitute a multiplication and accumulation sub-unit, each of the multiplication operation modules includes a plurality of multipliers, and each of the multi-stage addition operation modules includes a plurality of adders;

[0015] The input end of each of the data input modules is electrically connected to the data providing module, the output end of each of the data input modules is electrically connected to the input end of a multiplication operation module, the output end of each of the multiplication operation modules is electrically connected to the input end of a multi-stage addition operation module, and the output end of each of the multi-stage addition operation modules is electrically connected to multiple data output modules.

[0016] The present application also provides a data processing method, which is applied to the multiplication and accumulation operation cluster described above, and the data processing method includes:

[0017] Acquire data to be processed, and determine a target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation subunit in each multiplication and accumulation unit;

[0018] The data to be processed are input into a multiplication-accumulation subunit in an idle state in the target multiplication-accumulation unit, and a multiplication-accumulation operation is performed on the data to be processed by the target multiplication-accumulation unit to obtain a multiplication-accumulation operation result of the data to be processed.

[0019] In one embodiment of the present application, optionally, the acquiring the data to be processed, and determining the target multiplication accumulator unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the sub-dimension of the multiplication accumulator sub-unit in each multiplication accumulator unit, includes:

[0020] A multiplication accumulator unit in which the dimension of the multiplication accumulator sub-unit is greater than or equal to the calculation dimension of the data to be processed is taken as a candidate multiplication accumulator unit; a candidate multiplication accumulator unit which is in an idle working state and has the largest dimension of the multiplication accumulator sub-unit is taken as a target multiplication accumulator unit corresponding to the data to be processed.

[0021] Beneficial effects of the present application: The present application provides a multiplication-accumulator operation cluster and a data processing method, wherein the multiplication-accumulator operation cluster includes a plurality of multiplication-accumulator units, each of which includes a plurality of multiplication-accumulator sub-units. When the calculation dimensions of two multi-dimensional data in the to-be-processed data are smaller than the total dimension of the multiplication-accumulator unit, the plurality of to-be-processed data are respectively input into a multiplication-accumulator sub-unit. Within the same clock cycle, the plurality of multiplication-accumulator sub-units in each multiplication-accumulator unit can simultaneously perform multiplication-accumulation operations on the to-be-processed data inputted by them, thereby improving the utilization rate of the multiplication-accumulator operation unit and increasing the The output bandwidth of the multiplier-accumulator is increased; when the data to be processed includes multiple sequentially arranged data blocks, one multiplier-accumulator sub-cluster completes a multiplication-accumulation operation of one data block, and outputs the multiplication-accumulation operation result of the data block to the next sequentially arranged multiplier-accumulator sub-cluster, thereby realizing the processing of the data to be processed in a pipeline working mode. In the same clock cycle, multiple multiplier-accumulator sub-clusters process their respective input data at the same time, and the input data of the multiplier-accumulator is prepared in time, and the output end outputs the operation result in time, thereby avoiding the problem of line blocking and improving the utilization rate of the multiplier-accumulator operation unit.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0024] Figure 1 is a schematic diagram of the structure of a traditional multiply-accumulate unit;

[0025] Figure 2 A structural block diagram of a multiplier-accumulator operation cluster of an exemplary embodiment of the present application;

[0026] Figure 3 A structural block diagram of another multiplication and accumulation operation cluster of an exemplary embodiment of the present application;

[0027] Figure 4 is a schematic diagram of the operation of a traditional multiply-accumulate unit;

[0028] Figure 5 A schematic diagram of the operation of a multiplication and accumulation unit of an exemplary embodiment of the present application;

[0029] Figure 6 A schematic diagram of using a conventional multiply-accumulate unit for processing data in a pipeline processing manner;

[0030] Figure 7 A schematic diagram of using a multiply-accumulate unit for processing data in a pipeline processing manner according to an exemplary embodiment of the present application;

[0031] Figure 8 A structural block diagram of another multiplier-accumulator operation cluster according to an exemplary embodiment of the present application;

[0032] Fig. 9 A schematic diagram of the structural connection of a multiplier-accumulator operation cluster of an exemplary embodiment of the present application;

[0033] Fig.10 A schematic diagram of the structural connection of a multiplier-accumulator subunit of an exemplary embodiment of the present application;

[0034] Fig.11 A flowchart of a data processing method of an exemplary embodiment of the present application;

[0035] in,

[0036] The numbers in the figure are as follows: 1-multiplication accumulator unit; 11-multiplication accumulator sub-unit; 2-multiplication accumulator sub-cluster; 12-data input module; 111-multiplication operation module; 112-multi-stage addition operation module; 113-data output module. DETAILED DESCRIPTION

[0037] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0038] In order to further explain the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present application are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0039] In the prior art, the commonly used multiplier-accumulator unit has only one data output terminal, such as Figure 1 As shown, D is the data output terminal in the figure. If there is an 8-bit integer MAC in an AI processor, where M=32, X=16, and M is the number of multipliers. When doing a normal convolution operation, such as a 3x3 depthwise convolution operation, only 9 multipliers are used for a 3x3 convolution operation. The MAC under the usual architecture has only one data output terminal. The multipliers used in the operation participate in the calculation, and the multipliers that are not used are in an idle waiting state. If this 3x3 convolution is calculated, the number of multipliers used is X'=9, and there are 32 multipliers in the multiplier-accumulator unit. There will be many multipliers in an idle state. Only after the convolution calculation is completed and the calculation result is output, the entire MAC will wait for the next operation. The operation utilization rate of the MAC is only 9 / 32 during the 3x3 convolution operation, and the operation utilization rate is relatively low.

[0040] This application proposes a multiply-accumulate operation cluster, such as Figure 2 As shown, the multiplication-accumulation operation cluster includes a plurality of multiplication-accumulation units 1, each of which is used to complete the multiplication-accumulation operation of at least one group of data to be processed, each group of data to be processed includes two multi-dimensional data, each multiplication-accumulation unit 1 includes a plurality of multiplication-accumulation sub-units 11, each multiplication-accumulation sub-unit 11 is used to complete a multiplication-accumulation operation and output a multiplication-accumulation operation result, wherein, when the calculation dimension of the two multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulation unit 1, the multiplication-accumulation operation of each group of data to be processed is completed by at least one multiplication-accumulation sub-unit 11 in the multiplication-accumulation unit 1, and the multiplication-accumulation operation result of the data to be processed is output; and / or, as Figure 3As shown, when the data to be processed includes multiple sequentially arranged data blocks, the multiplication and accumulation operations of one data block are completed respectively by multiple multiplication and accumulation sub-clusters 2 in the multiplication and accumulation operation cluster, and the multiplication and accumulation operation results of the data block are output to the next multiplication and accumulation sub-cluster 2 arranged in sequence, until the multiplication and accumulation operations of all data blocks are completed, and the multiplication and accumulation operation results of the data to be processed are output.

[0041] Specifically, the multiplication and accumulation operation cluster includes multiple multiplication and accumulation units, each of which includes at least one multiplication and accumulation sub-unit, and each multiplication and accumulation sub-unit can independently process the data to be processed. In the same clock cycle, when multiple groups of data to be processed are respectively sent to multiple multiplication and accumulation sub-units in the same multiplication and accumulation unit, the multiple multiplication and accumulation sub-units respectively perform multiplication and accumulation operations on their respective input data to be processed, and each outputs an operation result, so that in the same clock cycle, the same multiplication and accumulation unit can perform operation processing on multiple data to be processed, thereby improving the operation utilization rate of the multiplication and accumulation unit.

[0042] When the computational dimension of the data to be processed is smaller than the dimension of the multiplication and accumulation unit, only a small amount of MAC computing power can be effectively utilized when the traditional multiplication and accumulation unit is used to process the data, such as Figure 4 , when there is data to be processed with 1 / 4 dimension of the MACC input, only 1 / 4 of the multipliers in the traditional MACC unit are used, and the remaining multipliers are idle, and the operation utilization is very low. When the MACC unit is divided into multiple MACC sub-units, for example Figure 5 As shown, when the multiplier-accumulator unit is divided into four multiplier-accumulator sub-units, the total input dimension of the multiplier-accumulator unit remains unchanged, and the input dimension of each multiplier-accumulator sub-unit is 1 / 4 of the total dimension. In one clock cycle, four groups of 1 / 4-dimensional data to be processed are each input into a multiplier-accumulator sub-unit, and each multiplier-accumulator sub-unit performs data processing on its own. Each multiplier in the multiplier-accumulator unit is used, and all computing power is utilized, thereby improving computing utilization.

[0043] For example, a multiplication-accumulator unit includes four multiplication-accumulator sub-units, and the number of multipliers included in each multiplication-accumulator sub-unit is 9, that is, the dimension of the multiplication-accumulator sub-unit is 9, and the total dimension of the multiplication-accumulator unit is 36. If there are two groups of data to be input, the calculation dimensions of the two groups of data to be processed are 8 and 9 respectively. In the same clock cycle, the two multiplication-accumulator sub-units in the multiplication-accumulator unit perform multiplication-accumulation operations on the two groups of data to be processed respectively, and each obtains an output result. At this time, the operation utilization rate of the multiplication-accumulator unit is 8+9 / 36. If the two data to be processed are processed by the traditional multiplication-accumulator unit, two clock cycles are required, and each clock cycle processes a group of data to be processed. The operation utilization rates of the multiplication-accumulator unit in the two clock cycles are 8 / 36 and 9 / 36 respectively. However, the multiplication-accumulator operation cluster of the present application can be completed in the same clock cycle, and the operation utilization rate of the multiplication-accumulator unit is high.

[0044] When the input data of the multiplier-accumulator operation cluster is not prepared in time or the output data of the multiplier-accumulator operation cluster is not output in time, the line is blocked, resulting in the multiplier-accumulator operation cluster being unable to enter the next use, which also leads to low utilization of the multiplier-accumulator operation unit.

[0045] When the data to be processed is data to be processed in an assembly line processing manner, if a traditional multiplier-accumulator is used, after the data to be processed passes through the multiplier-accumulator for the first multiplication-accumulation operation, the operation result is input into the multiplication-accumulator again for the second multiplication-accumulation operation, and the operation result is input into the multiplication-accumulator unit again until the operation of the data to be processed is completed. The operation utilization rate of the multiplication-accumulator is relatively low.

[0046] Therefore, for data processed in a pipeline processing manner, the multiplication and accumulation unit operation cluster is divided into a plurality of multiplication and accumulation unit sub-clusters, and the plurality of multiplication and accumulation unit sub-clusters are arranged in sequence. The data to be processed includes a plurality of data blocks arranged in sequence. In each clock cycle, one data block is input to the accumulator sub-clusters in the first arrangement order. In the first clock cycle, the multiplication and accumulation unit sub-clusters in the first arrangement order process the input data, and the processing result is transmitted to the data input end of the multiplication and accumulation unit sub-clusters in the second arrangement order. In the second clock cycle, a new data block enters the multiplication and accumulation unit sub-clusters in the first arrangement order. The multiplication and accumulation unit sub-clusters in the first arrangement order and the multiplication and accumulation unit sub-clusters in the second arrangement order respectively process the input data, and the multiplication and accumulation unit sub-clusters in the second arrangement order are transmitted to the data input end of the multiplication and accumulation unit sub-clusters in the second arrangement order. The processing results of the clusters are transmitted to the data input end of the multiplier-accumulator sub-cluster of the third arrangement order, and the processing results of the multiplier-accumulator sub-cluster of the first arrangement order are transmitted to the data input end of the multiplier-accumulator sub-cluster of the second arrangement order. In the third clock cycle, a new data block is sent to the input end of the multiplier-accumulator sub-cluster of the first arrangement order, and the above process is repeated to input the output data of the multiplier-accumulator sub-cluster of the previous arrangement order to the data input end of the adjacent multiplier-accumulator sub-cluster of the next arrangement order, so that in the same clock cycle, multiple multiplier-accumulator sub-clusters process their respective input data at the same time, and the data input end of each multiplier-accumulator sub-cluster prepares data in time, and the data output end takes away data in time, thereby improving the operational utilization of the multiplier-accumulator.

[0047] For example, the processing of a neural network model is often defined as the operation process of several layers in sequence. Usually, the calculation process of several layers is that the layer 1 network layer calls the entire MAC to perform calculations, and its input and output bandwidths are both W. The MAC calculation result is the input data of the layer 2 network layer. After the layer 1 network layer completes all calculations, the layer 2 network layer continues to call the entire MAC to perform calculations until the entire calculation is completed. Figure 6 As shown, it can be considered as the time-division multiplexing of MAC in the AI ​​accelerator system, that is, each layer network layer uses the entire MAC, and the computing utilization rate of MAC is relatively low. The bandwidth at this time can be considered as the bandwidth of each layer network layer.

[0048] The multiplier-accumulator operation cluster proposed in the present application includes multiple multiplier-accumulator sub-clusters, for example, the multiplier-accumulator operation cluster includes four multiplier-accumulator sub-clusters P1_MAC, P2_MAC, P3_MAC, and P4_MAC. A pipeline operation is used between multiple layer network layers, where the layer1 network layer uses the multiplier-accumulator sub-cluster P1_MAC for operation processing, the layer2 network layer uses the multiplier-accumulator sub-cluster P2_MAC for operation processing, the layer3 network layer uses the multiplier-accumulator sub-cluster P3_MAC for operation processing, and the layer4 network layer uses the multiplier-accumulator sub-cluster P4_MAC for operation processing. In the first clock cycle, the data to be processed is input to the input end of the multiplication and accumulator sub-cluster P1_MAC. After the operation of the multiplication and accumulator sub-cluster P1_MAC is completed, the operation result of the multiplication and accumulator sub-cluster P1_MAC is used as the input data of the multiplication and accumulator sub-cluster P2_MAC. In the second clock cycle, new data to be processed is input to the input end of the multiplication and accumulator sub-cluster P1_MAC. The multiplication and accumulator sub-cluster P1_MAC and the multiplication and accumulator sub-cluster P2_MAC simultaneously perform operation and processing on their respective input data. After the operation is completed, the operation result of the multiplication and accumulator sub-cluster P1_MAC is used as the input data of the multiplication and accumulator sub-cluster P2_MAC, and the operation result of the multiplication and accumulator sub-cluster P2_MAC is used as the input data of the multiplication and accumulator sub-cluster P3_MAC. Similarly, the processing result of the multiplication and accumulator sub-cluster of the previous arrangement order is used as the input data of the multiplication and accumulator sub-cluster of the next arrangement order, as shown in FIG. Figure 7 As shown in the figure, P1_MAC has data to be processed input in sequence, and the corresponding P2_MAC~P4_MAC also have data to be processed input in sequence. In the same clock cycle, P1_MAC~P4_MAC each perform operations on the input data, and the total computing power of the entire multiplier-accumulator sub-cluster is improved. The output bandwidth becomes the sum of the bandwidth of each layer network layer. It can be considered that the output bandwidth becomes 4 times the original, meeting the purpose of increasing the bandwidth, but the output bandwidth of each multiplier-accumulator sub-cluster remains unchanged.

[0049] The multiplication and accumulation unit operation cluster includes a plurality of multiplication and accumulation unit, each multiplication and accumulation unit includes at least one multiplication and accumulation unit sub-unit, each multiplication and accumulation unit sub-unit can independently process the data to be processed, and realize it in the same clock cycle, thereby improving the operation utilization rate of the multiplication and accumulation unit, but the output bandwidth of the multiplication and accumulation unit increases a lot, and the increased output bandwidth may cause the subsequent processing module to fail to process in time. If the data output by the multiplication and accumulation unit cannot be taken away in time, the multiplication and accumulation unit cannot perform the next operation, causing line congestion, and affecting the operation efficiency of the multiplication and accumulation unit. Therefore, the multiplication and accumulation unit sub-unit is combined with the multiplication and accumulation unit sub-cluster, and at least one multiplication and accumulation unit sub-unit constitutes the multiplication and accumulation unit sub-cluster, such as Figure 8 As shown, a multiplication-accumulation subunit is used as a multiplication-accumulation sub-cluster, and the output data of the previous multiplication-accumulation subunit is sent to the data input end of the next multiplication-accumulation subunit in sequence, so that the output data of the multiplication-accumulation subunit is processed in time without causing line congestion, thereby improving the operation utilization rate of the multiplication-accumulation operation cluster.

[0050] Compared with the prior art, the present application provides a multiplication-accumulator operation cluster, which includes multiple multiplication-accumulator units, each of which includes multiple multiplication-accumulator sub-units. When the calculation dimension of two multi-dimensional data in the data to be processed is smaller than the total dimension of the multiplication-accumulator unit, within the same clock cycle, the multiple multiplication-accumulator sub-units in each multiplication-accumulator unit can simultaneously perform multiplication-accumulation operations on the data to be processed that are input by each of them, thereby improving the utilization rate of the multiplication-accumulator operation unit and increasing the output bandwidth of the multiplication-accumulator. When the data to be processed includes multiple sequentially arranged data blocks, a multiplication-accumulator sub-cluster completes a multiplication-accumulation operation of a data block, and outputs the multiplication-accumulation operation result of the data block to the next sequentially arranged multiplication-accumulator sub-cluster, thereby realizing the processing of the data to be processed in a pipeline working mode. Within the same clock cycle, multiple multiplication-accumulator sub-clusters simultaneously process the data input by each of them, and the input data of the multiplication-accumulator is prepared in time, and the output end outputs the operation result in time, thereby avoiding the problem of line blockage and improving the utilization rate of the multiplication-accumulator operation unit.

[0051] In one embodiment, the dimensions of the multiplication accumulator sub-units in the same multiplication accumulator unit are the same, the sum of the dimensions of all the multiplication accumulator sub-units in each multiplication accumulator unit is the same as the total dimension of the multiplication accumulator unit, and the dimensions of the multiplication accumulator sub-units in multiple different multiplication accumulator units are the same as or different from each other.

[0052] Specifically, the number of multipliers included in the multiplication-accumulator unit is the dimension of the multiplication-accumulator unit. The number of multipliers in the multiplication-accumulator sub-units included in the same multiplication-accumulator unit is the same, and the total number of multipliers in all the multiplication-accumulation sub-units in each multiplication-accumulator is equal to the total dimension of each multiplication-accumulator unit. There are multiple multiplication-accumulator units in the multiplication-accumulator operation cluster, and there must be some multiplication-accumulator units with different numbers of multipliers in the multiple multiplication-accumulator units. The number of multiplication-accumulator units with the same number of multipliers can be multiple. For example, the multiplication-accumulator operation cluster includes multiple multiplication-accumulator units, wherein the number of multipliers included in the multiplication-accumulator sub-units in two multiplication-accumulator units is 9, the number of multipliers included in the multiplication-accumulator sub-unit in one multiplication-accumulator unit is 12, the number of multipliers included in the multiplication-accumulator sub-unit in one multiplication-accumulator unit is 16, and so on.

[0053] In one embodiment, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication-accumulator sub-unit in at least one multiplication-accumulator unit, a target multiplication-accumulator unit for operating the data to be processed is determined based on the computational dimension of the data to be processed and the dimension of the multiplication-accumulator sub-unit in the multiplication-accumulator unit, and multiplication-accumulation operations are performed on the data to be processed through the multiplication-accumulator sub-unit in the target multiplication-accumulator unit.

[0054] Specifically, when the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication accumulator sub-unit in at least one multiplication accumulator unit, the computational dimension of the group of processed data is compared with the dimensions of the multiplication accumulator sub-units in these multiplication accumulator units, and the multiplication accumulator unit with the smallest difference between the computational dimension and the dimension of the multiplication accumulator sub-unit is selected as the target multiplication accumulator unit, and multiplication and addition operations are performed using the multiplication accumulator sub-units in the target multiplication accumulator unit that are in an idle state. For example, when the dimensions of the multiplication and accumulation sub-units in multiple multiplication and accumulation units are 6, 9, 12 and 16, and the number of multipliers required for processing data is 8, the multiplication and accumulation units whose dimensions of the multiplication and accumulation sub-units are greater than 8 are taken as candidate multiplication and accumulation units, that is, the multiplication and accumulation units whose dimensions of the multiplication and accumulation sub-units are 9, 12 and 16 are taken as candidate multiplication and accumulation units, and the difference between the calculation dimension and the dimension of the multiplication and accumulation sub-unit in each candidate multiplication and accumulation unit is obtained. The candidate multiplication and accumulation unit with the smallest difference is the candidate multiplication and accumulation unit with the dimension of the multiplication and accumulation sub-unit of 9, and this candidate multiplication and accumulation unit is taken as the target multiplication and accumulation unit.

[0055] In one embodiment, when the computational dimension of a set of data to be processed is greater than the dimension of the multiplication and accumulation subunits in all the multiplication and accumulation units and less than the total dimension of each multiplication and accumulation unit, the multiplication and accumulation operation of the data to be processed is completed by multiple multiplication and accumulation subunits in the multiplication and accumulation unit.

[0056] Specifically, when the computational dimension of a group of data to be processed is larger than the dimension of the multiplication accumulator sub-units in all the multiplication accumulator units, it means that one multiplication accumulator sub-unit in the multiplication accumulator unit cannot be used for calculation and processing, and it is necessary to use multiple multiplication accumulator sub-units in the multiplication accumulator unit to combine and process the data to be processed, that is, the data to be processed is jointly processed by multiple multiplication accumulator sub-units in the multiplication accumulator unit.

[0057] In one embodiment, when the computational dimension of the data to be processed is greater than the dimension of the multiplication accumulator sub-units in all the multiplication accumulator units and less than the total dimension of each multiplication accumulator unit, a target multiplication accumulator for operating the data to be processed and a target number of multiplication accumulator sub-units in the target multiplication accumulator are determined according to the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-units in each multiplication accumulator unit, and multiplication and accumulation operations are performed on the data to be processed through the target number of multiplication accumulator sub-units in the target multiplication accumulator.

[0058] Specifically, when the computational dimension of a group of data to be processed is greater than the dimension of the multiplication accumulator subunits in all the multiplication accumulator units, it is necessary to use multiple multiplication accumulator subunits in the multiplication accumulator unit to communicate and process the data to be processed. The quotient of the computational dimension of the data to be processed and the dimension of the multiplication accumulator subunit in any multiplication accumulator unit plus one is used as the target number, and the data to be processed is processed by the target number of multiplication accumulator subunits in this multiplication accumulator unit.

[0059] In one embodiment, at least one MACC subunit in a MACC operation cluster serves as a MACC sub-cluster.

[0060] Specifically, when multiple MACC subunits in the same MACC unit perform calculation processing independently, the calculation utilization rate of the MACC unit is improved, but at the same time, the output bandwidth of the MACC unit becomes multiple times of the original, and the increase in output bandwidth will cause processing line blockage. Therefore, at least one MACC subunit is used as a MACC sub-cluster, and the MACC sub-cluster processes the data to be processed in a pipeline operation manner, that is, the data output by the MACC sub-cluster of the previous arrangement order is input to the data input end of the MACC sub-cluster of the next arrangement order. In this way, although the output bandwidth of the MACC unit is increased, the output data of the MACC subunit is processed in time, and no blockage occurs, thereby improving the calculation utilization rate of the MACC.

[0061] In one embodiment, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimension of the multiplication-accumulator sub-unit in any multiplication-accumulator unit, each multiplication-accumulator sub-unit in the multiplication-accumulator unit is used as a multiplication-accumulator sub-cluster, and the multiplication-accumulation operation of one data block is completed respectively by multiple multiplication-accumulator sub-units in the multiplication-accumulator unit, and the multiplication-accumulation operation result of the data block is output to the next multiplication-accumulation sub-unit arranged in sequence, until the multiplication-accumulation operation of all data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

[0062] Specifically, when the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimensions of the multiplication and accumulation sub-units in any multiplication and accumulation unit, and the first operation results of these data blocks need to be subjected to another multiplication and accumulation operation, that is, when processed in an assembly line processing manner, each multiplication and accumulation sub-unit in the multiplication and accumulation unit is used as a multiplication and accumulation sub-cluster, and the data to be processed is processed in an assembly line manner through the multiplication and accumulation sub-clusters, that is, the data output by the multiplication and accumulation sub-unit of the previous arrangement order is input into the multiplication and accumulation sub-unit of the next arrangement order. In this way, although the output bandwidth of the multiplication and accumulation unit is increased, the output bandwidth of the multiplication and accumulation sub-unit remains unchanged, and no congestion will occur, thereby improving the computational utilization rate of the multiplication and accumulation unit.

[0063] In one embodiment, Fig. 9 As shown, the multiplier-accumulator unit includes multiple data input modules 12, multiple multiplication operation modules 111, multiple multi-stage addition operation modules 112 and multiple data output modules 113, wherein one multiplication operation module 111, one multi-stage addition operation module 112 and multiple data output modules 113 constitute a multiplication-accumulator sub-unit 11, each multiplication operation module includes multiple multipliers, and each multi-stage addition operation module includes multiple adders; the input end of each data input module 12 is respectively electrically connected to the data providing module, the output end of each data input module 12 is respectively electrically connected to the input end of a multiplication operation module 111, the output end of each multiplication operation module 111 is electrically connected to the input end of a multi-stage addition operation module 112, and the output end of each multi-stage addition operation module 112 is respectively electrically connected to multiple data output modules 113.

[0064] Specifically, the multiplication and accumulation unit includes a plurality of data input modules, a plurality of multiplication operation modules, a plurality of multi-stage addition operation modules and a plurality of data output modules, and one multiplication operation module, one multi-stage addition operation module and a plurality of data output modules constitute a multiplication and accumulation sub-unit. For each multiplication and accumulation sub-unit, the data providing module inputs the data to be processed into the input end of the data input module, the output end of the data input module transmits the data to the input end of the multiplication operation module, the multiplication operation module performs multiplication operation on the input data, and then outputs the multiplication operation result to the input end of the multi-stage addition operation module, the multi-stage addition operation module performs accumulation operation on the input data to obtain a plurality of accumulation operation results, and outputs each accumulation operation result to a data output module.

[0065] like Fig.10 As shown, multiple multipliers form a multiplication operation module, multiple adders form a multi-stage addition operation module, multiple data output terminals C1, C2, C3, C4 and D, input data is A1...A16, and input data is B1...A16. The multiplication operation module, the multi-stage addition operation module and the multiple data output terminals serve as a multiplication accumulator subunit.

[0066] The present application also provides a data processing method, which is applied to the above-mentioned multiplication and accumulation operation cluster, and the data processing method is as follows: Fig.11 As shown, including:

[0067] 112: Obtain the data to be processed, and determine the target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation subunit in each multiplication and accumulation unit;

[0068] 114: Input the data to be processed into the multiplication and accumulation subunit in the target multiplication and accumulation unit that is in an idle state, and perform multiplication and accumulation operations on the data to be processed through the target multiplication and accumulation unit to obtain the multiplication and accumulation operation results of the data to be processed.

[0069] Specifically, according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation unit sub-unit in each multiplication and accumulation unit, the target multiplication and accumulation unit corresponding to the data to be processed is determined, and the multiplication and accumulation operation is performed on the data to be processed by the multiplication and accumulation sub-unit in the target multiplication and accumulation unit that is in an idle state to obtain the multiplication and accumulation operation result of the data to be processed.

[0070] For example, when the calculation dimension of the data to be processed is 8, and the dimensions of the multiplication and accumulation sub-units in multiple multiplication and accumulation units are 6, 9, 12, and 16 respectively, a multiplication and accumulation unit in which the dimension of each multiplication and accumulation sub-unit in the multiplication and accumulation unit is 9 is selected as the target multiplication and accumulation unit, and multiplication and accumulation operations are performed on the data to be processed through the multiplication and accumulation sub-units in the target multiplication and accumulation unit that are in an idle state.

[0071] When there are multiple groups of data to be processed, the target multiplier-accumulator unit corresponding to each group of data to be processed is determined respectively, and the group of data to be processed is input into its corresponding target multiplier-accumulator unit, so that multiple target multiplier-accumulator units can process their respective input data simultaneously, thereby improving the data processing efficiency of the multiplier-accumulator operation cluster.

[0072] In one embodiment, obtaining data to be processed, and determining a target multiplication and accumulation unit corresponding to the data to be processed according to a calculation dimension of the data to be processed and a sub-dimension of a multiplication and accumulation unit in each multiplication and accumulation unit, comprises:

[0073] A multiplication accumulator unit in which the dimension of the multiplication accumulator sub-unit is greater than or equal to the calculation dimension of the data to be processed is taken as a candidate multiplication accumulator unit; a candidate multiplication accumulator unit that is in an idle working state and has the largest dimension of the multiplication accumulator sub-unit is taken as a target multiplication accumulator unit corresponding to the data to be processed.

[0074] Specifically, the computational dimension of the data to be processed and the dimension of the multiplication accumulator sub-unit in each multiplication accumulator unit are compared, and the multiplication accumulator unit whose dimension of the multiplication accumulator sub-unit is greater than or equal to the computational dimension of the data to be processed is taken as a candidate multiplication accumulator unit. There are multiple candidate accumulator units, and a candidate multiplication accumulator unit that is in an idle state and has the largest dimension of the multiplication accumulator sub-unit is selected from the multiple candidate multiplication accumulator units as the target multiplication accumulator unit, and the target multiplication accumulator unit is used to perform multiplication and addition operations on the data to be processed.

[0075] Compared with the prior art, the present application provides a data processing method, which determines the target multiplication accumulator unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension included in the multiplication accumulator sub-unit in each multiplication accumulator unit, and inputs the data to be processed into the data input end of the multiplication accumulator sub-unit in the target multiplication accumulator unit. Since the multiple multiplication accumulator sub-units in the target multiplication accumulator unit can work independently, the multiple multiplication accumulator sub-units can process data simultaneously, thereby improving the computational utilization and total output bandwidth of the multiplication accumulator module.

[0076] It should be understood that various modifications may be made to the embodiments of the present application. Therefore, the above description should not be considered as limiting, but only as an example of the embodiments. Other modifications within the scope and spirit of the present application will occur to those skilled in the art.

[0077] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.

[0078] These and other characteristics of the present application will become apparent from the following description of a preferred form of embodiment given as a non-limiting example with reference to the accompanying drawings.

[0079] It should also be understood that although the present application has been described with reference to some specific examples, those skilled in the art will be able to readily implement many other equivalent forms of the present application.

[0080] The above and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.

[0081] Specific embodiments of the present application are described hereinafter with reference to the accompanying drawings; however, it should be understood that the embodiments applied for are merely examples of the present application, which may be implemented in a variety of ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that obscure the present application. Therefore, the specific structural and functional details applied for herein are not intended to be limiting, but merely serve as a basis and representative basis for the claims to teach those skilled in the art to use the present application in a variety of ways with substantially any suitable detailed structure.

[0082] This specification may use the phrases "in one embodiment," "in another embodiment," "in yet another embodiment," or "in other embodiments," all of which may refer to one or more of the same or different embodiments according to the present application.

[0083] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.

Claims

1. A multiplier-accumulator operation cluster, characterized in that: The multiplication and accumulation operation cluster includes a plurality of multiplication and accumulation units, each of which is used to complete a multiplication and accumulation operation of at least one group of data to be processed, each group of data to be processed includes two multi-dimensional data, each of which includes a plurality of multiplication and accumulation sub-units, the dimensions of the multiplication and accumulation sub-units in the same multiplication and accumulation unit are the same, and each multiplication and accumulation sub-unit is used to complete a multiplication and accumulation operation and output a multiplication and accumulation operation result, wherein, When the calculation dimensions of two of the multi-dimensional data in the data to be processed are smaller than the total dimension of the multiplication-accumulation unit, the multiplication-accumulation operation of each group of the data to be processed is completed by at least one multiplication-accumulation sub-unit in the multiplication-accumulation unit, and the multiplication-accumulation operation result of the data to be processed is output; At least one multiplication-accumulation sub-unit in the multiplication-accumulation operation cluster is used as a multiplication-accumulation sub-cluster. When the data to be processed includes a plurality of sequentially arranged data blocks, the multiplication-accumulation operation of one data block is completed respectively by the plurality of multiplication-accumulation sub-clusters in the multiplication-accumulation operation cluster, and the multiplication-accumulation operation result of the data block is output to the next multiplication-accumulation sub-cluster arranged in sequence, until the multiplication-accumulation operation of all the data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output; When the computational dimension of a group of data to be processed is less than or equal to the dimension of a multiplication-accumulator subunit in at least one multiplication-accumulator unit, a target multiplication-accumulator unit for computing the data to be processed is determined according to the computational dimension of the data to be processed and the dimension of the multiplication-accumulator subunit in the multiplication-accumulator unit, and a multiplication-accumulation operation is performed on the data to be processed by the multiplication-accumulator subunit in the target multiplication-accumulator unit; When the calculation dimension of the data to be processed is greater than the dimension of the multiplication accumulator subunits in all the multiplication accumulator units and less than the total dimension of each of the multiplication accumulator units, a target multiplication accumulator for operating the data to be processed and a target number of multiplication accumulator subunits in the target multiplication accumulator are determined according to the calculation dimension of the data to be processed and the dimension of the multiplication accumulator subunits in each of the multiplication accumulator units, and a multiplication accumulation operation is performed on the data to be processed through the target number of multiplication accumulator subunits in the target multiplication accumulator; In the same clock cycle, a plurality of multiplication-accumulation subunits in the target multiplication-accumulation unit respectively perform multiplication-accumulation operations on a plurality of groups of data to be processed; The multiplication and accumulation unit comprises a plurality of data input modules, a plurality of multiplication operation modules, a plurality of multi-stage addition operation modules and a plurality of data output modules, wherein one multiplication operation module, one multi-stage addition operation module and a plurality of data output modules constitute a multiplication and accumulation sub-unit, each of the multiplication operation modules comprises a plurality of multipliers, and each of the multi-stage addition operation modules comprises a plurality of adders; The input end of each of the data input modules is electrically connected to the data providing module, the output end of each of the data input modules is electrically connected to the input end of a multiplication operation module, the output end of each of the multiplication operation modules is electrically connected to the input end of a multi-stage addition operation module, and the output end of each of the multi-stage addition operation modules is electrically connected to multiple data output modules.

2. The multiplier-accumulator operation cluster according to claim 1, characterized in that: The sum of the dimensions of all the MACC sub-units in each MACC unit is the same as the total dimension of the MACC unit, and the dimensions of the MACC sub-units in a plurality of different MACC units are the same as or different from each other.

3. The multiplier-accumulator operation cluster according to claim 1, characterized in that: When the computational dimensions of all data blocks in the data to be processed are less than or equal to the dimension of the multiplication-accumulator sub-unit in any multiplication-accumulator unit, each multiplication-accumulator sub-unit in the multiplication-accumulator unit is used as a multiplication-accumulator sub-cluster, and the multiplication-accumulation operation of one data block is completed respectively by multiple multiplication-accumulator sub-units in the multiplication-accumulator unit, and the multiplication-accumulation operation result of the data block is output to the next multiplication-accumulation sub-unit arranged in sequence, until the multiplication-accumulation operation of all the data blocks is completed, and the multiplication-accumulation operation result of the data to be processed is output.

4. A data processing method, applied to the multiplier-accumulator operation cluster according to any one of claims 1 to 3, characterized in that: The data processing method comprises: Acquire data to be processed, and determine a target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the dimension of the multiplication and accumulation subunit in each multiplication and accumulation unit; The data to be processed are input into a multiplication-accumulation subunit in an idle state in the target multiplication-accumulation unit, and a multiplication-accumulation operation is performed on the data to be processed by the target multiplication-accumulation unit to obtain a multiplication-accumulation operation result of the data to be processed.

5. The data processing method according to claim 4, characterized in that: The obtaining of the data to be processed, and determining a target multiplication and accumulation unit corresponding to the data to be processed according to the calculation dimension of the data to be processed and the sub-dimension of the multiplication and accumulation sub-unit in each multiplication and accumulation unit, comprises: Taking a multiplication and accumulation unit whose dimension of a multiplication and accumulation subunit in the multiplication and accumulation unit is greater than or equal to the calculation dimension of the data to be processed as a candidate multiplication and accumulation unit; A candidate multiplication and accumulation unit that is in an idle working state and has the largest dimension of a multiplication and accumulation sub-unit is used as a target multiplication and accumulation unit corresponding to the data to be processed.

Images