Operation circuit, processing unit, integrated circuit system, electronic assembly, and electronic device

Abstract

Provided in the present disclosure are an operation circuit, a processing unit, an integrated circuit system, an electronic assembly, and an electronic device. The present disclosure aims to support a matrix operation and a convolution operation with relatively high operation efficiency. In a matrix operation, a dot product operation is performed on data in a single row of one matrix and data in a single column of another matrix by means of a single operation unit in an operation unit array. In a convolution operation, a dot product operation is performed, by means of a single operation unit, on a single group of convolution kernel data of a corresponding convolution kernel and a single group of data to be convolved of a convolution object in a single convolution sliding window, wherein operation units in the same column correspond to the same convolution kernel, operation units in different columns correspond to different convolution kernels, the single group of convolution kernel data comprises convolution kernel data of a plurality of channels at the same position, and the single group of data to be convolved comprises data to be convolved of the plurality of channels at the same position. The present disclosure has good universality and can be applicable to chips such as a GPU, an NPU and an ASIC; and the efficiency of matrix and convolution operations is relatively high.

Description

Computing circuits, processors, integrated circuit systems, electronic components and equipment

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the Chinese patent application with application number 202410257137.9 and application date March 7, 2024, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby introduced into this application as a reference. Technical Field

[0003] The present disclosure relates to the field of integrated circuit technology, and in particular to an arithmetic circuit, a processor, an integrated circuit system, an electronic component, and an electronic device. Background Art

[0004] Matrix multiplication and matrix multiplication-addition operations are widely used in artificial intelligence and high-performance computing. For example, the convolution operation of a convolutional neural network can be converted into a matrix multiplication or matrix multiplication-addition operation. For another example, matrix multiplication or matrix multiplication-addition operations are also widely used in transformer neural networks.

[0005] For systems that excel at convolutional neural network operations, such as ASIC (Application Specific Integrated Circuit) and NPU (Neural Network Processing Unit), dedicated convolution operation circuits are mainly used to improve the efficiency of convolution operations, but their matrix operation efficiency is relatively low. For systems that excel at matrix operations, GPU (Graphics Processing Unit) and GPGPU (General-Purpose Graphics Processing Unit) mainly use matrix multiplication circuits and matrix multiplication and addition circuits. Their matrix multiplication and matrix multiplication and addition operations are highly efficient, but the performance and efficiency of convolution operations need to be optimized, and a larger cache is required. This is because when such systems perform convolution operations, they need to first use img2col to expand the image and convolution kernel into matrix form, and then perform matrix operations. After the operation is completed, the feature map generated needs to be converted to the original format.

[0006] Summary of the Invention

[0007] The purpose of the present disclosure is to provide an arithmetic circuit, a processor, an integrated circuit system, an electronic component and an electronic device, which are designed to be compatible with matrix operations and convolution operations through the same set of arithmetic circuits with higher operation efficiency.

[0008] According to one aspect of the present disclosure, an operation circuit is provided, comprising an operation unit array. In a matrix operation mode, a single operation unit in the operation unit array is configured to perform a dot product operation on a single row of data in a first matrix and a single column of data in a second matrix. In a convolution operation mode, a single operation unit in the operation unit array is configured to perform a dot product operation on a single set of convolution kernel data corresponding to a convolution kernel and a single set of to-be-convolved data of a convolution object within a single convolution sliding window. Operation units in the same column correspond to the same convolution kernel, while operation units in different columns correspond to different convolution kernels. A single set of convolution kernel data includes convolution kernel data of multiple channels at the same position, and a single set of to-be-convolved data includes to-be-convolved data of multiple channels at the same position.

[0009] The arithmetic circuit provided in the embodiments of the present disclosure supports matrix multiplication operations. Furthermore, the arithmetic circuit provided in the embodiments of the present disclosure can also support matrix multiplication and addition operations. If the arithmetic circuit supports matrix multiplication and addition operations, then the arithmetic unit is further configured to perform an addition operation on the multiplication result of a single row of data in the first matrix and a single column of data in the second matrix, and on the element data in the third matrix.

[0010] In the embodiment of the present disclosure, the operation circuit may include multiple row broadcast buses, multiple column broadcast buses, and multiple operation result processing units, wherein the operation result processing units include a transparent transmission channel and an addition channel.

[0011] Based on the operation circuit structure of the broadcast bus and the operation result processing unit, in the matrix operation working mode, it is optional: the single row data of the first matrix is ​​transmitted through the corresponding row broadcast bus, the single column data of the second matrix is ​​transmitted through the corresponding column broadcast bus, and the operation results of the operation units in the same column are output through the transparent channel of the corresponding operation result processing unit; in the convolution operation working mode, each group of data to be convolved in a single convolution sliding window is transmitted through the corresponding row broadcast bus.

[0012] Based on the operation circuit structure of the broadcast bus and the operation result processing unit, optionally, multiple first data transmission modules are set on each row broadcast bus, and each operation unit outside the first column corresponds to a first data transmission module respectively, and the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel.

[0013] On this basis, in the matrix operation working mode, the first data transmission module opens the transparent transmission channel.

[0014] In the embodiment of the present disclosure, optionally, a second data transmission module is provided at the output end of the target operation unit, the second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel, and the target operation unit includes at least each operation unit except the last row.

[0015] On this basis, in the matrix operation working mode, the second data transmission module opens the transparent transmission channel.

[0016] In the embodiment of the present disclosure, in the convolution operation working mode:

[0017] In an optional implementation, the first data transmission module and the second data transmission module open a transparent transmission channel; the operation result processing unit adds the operation results of the corresponding same-column operation units for a single convolution sliding window through the addition channel and outputs the result.

[0018] In another optional implementation, the first data transmission module opens a pipeline temporary storage channel so that each group of convolution data within a single convolution window is delivered to each operation unit in the same row in a pipeline manner; the second data transmission module opens a transparent transmission channel; and the operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution window through the addition channel and outputs them.

[0019] In another optional implementation, the first data transmission module opens the pipeline temporary storage channel so that each group of convolution data in each convolution sliding window is delivered to each operation unit in the same row in a pulsating manner; the second data transmission module opens the pipeline temporary storage channel so that the operation result of this operation unit enters the addition input end of the next operation unit in the same column through the pipeline temporary storage channel; the operation result processing unit opens the transparent transmission channel.

[0020] In the embodiment of the present disclosure, the operation circuit may include an operation unit array, a decoding unit, a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of operation result processing units. Among them, the decoding unit is used to determine the working mode according to the operation indication and generate channel control information. The working mode includes a matrix operation working mode and a convolution operation working mode. In the matrix operation working mode, the row broadcast bus is used to transmit the row data of the first matrix, and the column broadcast bus is used to transmit the column data of the second matrix; in the convolution operation working mode, the row broadcast bus is used to transmit the data to be convolved. A plurality of first data transmission modules are provided on each row broadcast bus, and each operation unit outside the first column corresponds to a first data transmission module respectively. The first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel; the first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the indication of the above-mentioned channel control information. The output end of the target operation unit is provided with a second data transmission module, which includes a transparent transmission channel and a pipeline temporary storage channel. The second data transmission module activates the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next operation unit in the same column through the pipeline temporary storage channel. The target operation unit includes at least each operation unit outside the last row. The operation result processing unit includes a transparent transmission channel and an addition channel. The operation result processing unit activates the transparent transmission channel or the addition channel according to the instruction of the channel control information.

[0021] Based on any of the above-mentioned arithmetic circuit embodiments, the arithmetic circuit may further include an accumulation processing unit. In a matrix operation mode, the accumulation processing unit is configured to accumulate the operation results of a matrix operation in each arithmetic unit, where a matrix operation includes a matrix multiplication operation of a first matrix and a second matrix. In a convolution operation mode, the accumulation processing unit is configured to accumulate the operation results of a convolution operation on a convolution object in the arithmetic units in the same column.

[0022] According to another aspect of the present disclosure, a computing circuit is provided, comprising at least two computing unit arrays of the same size. In a matrix computing mode, a single computing unit in each computing unit array is configured to perform a dot product operation on a single row of data in a first submatrix and a single column of data in a second submatrix. In a convolution computing mode, a single computing unit in each computing unit array is configured to perform a dot product operation on a single set of convolution kernel data corresponding to a subconvolution kernel and a single set of to-be-convolved data of a subconvolution object within a single convolution sliding window. Computing units in the same column correspond to the same subconvolution kernel, while computing units in different columns correspond to different subconvolution kernels. The single set of convolution kernel data includes convolution kernel data for multiple channels at the same position, and the single set of to-be-convolved data includes to-be-convolved data for multiple channels at the same position. The at least two computing unit arrays are connected via a connection unit to implement matrix multiplication between the first matrix and the second matrix, or convolution operation of the convolution kernel on the convolution object. The first matrix is ​​divided into a plurality of first submatrices in the row direction, and the second matrix is ​​divided into a plurality of second submatrices in the column direction. The convolution kernel is divided into several sub-convolution kernels in the channel direction, and the convolution object is divided into several sub-convolution objects in the channel direction.

[0023] In the embodiment of the present disclosure, the operation circuit may include multiple row broadcast buses, multiple column broadcast buses, and multiple operation result processing units, wherein the operation result processing units include a transparent transmission channel and an addition channel.

[0024] Based on the operation circuit structure of the broadcast bus and the operation result processing unit, in the matrix operation working mode, optionally: the single row data of the first sub-matrix is ​​transmitted through the corresponding row broadcast bus, the single column data of the second sub-matrix is ​​transmitted through the corresponding column broadcast bus, and the operation results of the operation units in the same column in the single operation unit array are output through the transparent channel of the corresponding operation result processing unit; in the convolution operation working mode, each group of to-be-convolved data in a single convolution sliding window is respectively transmitted through the corresponding row broadcast bus; each operation unit array reuses the broadcast bus, or each operation unit array uses an independent broadcast bus, and the broadcast bus includes a row broadcast bus and / or a column broadcast bus.

[0025] Based on the operation circuit structure of the broadcast bus and the operation result processing unit, optionally, multiple first data transmission modules are set on each row broadcast bus, and each operation unit outside the first column corresponds to a first data transmission module, and the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel.

[0026] On this basis, in the matrix operation working mode, the first data transmission module opens the transparent transmission channel.

[0027] In the embodiment of the present disclosure, optionally, a second data transmission module is provided at the output end of the target operation unit, the second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel, and the target operation unit includes at least each operation unit except the last row.

[0028] On this basis, in the matrix operation working mode, the second data transmission module opens the transparent transmission channel.

[0029] In the embodiment of the present disclosure, in the convolution operation working mode:

[0030] In an optional implementation, the first data transmission module and the second data transmission module open a transparent transmission channel; the operation result processing unit adds the operation results of the corresponding same-column operation units for a single convolution sliding window through the addition channel and outputs the result.

[0031] In another optional implementation, the first data transmission module opens a pipeline temporary storage channel so that each group of convolution data within a single convolution window is delivered to each operation unit in the same row in a pipeline manner; the second data transmission module opens a transparent transmission channel; and the operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution window through the addition channel and outputs them.

[0032] In another optional implementation, the first data transmission module opens the pipeline temporary storage channel so that each group of convolution data in each convolution sliding window is delivered to each operation unit in the same row in a pulsating manner; the second data transmission module opens the pipeline temporary storage channel so that the operation result of this operation unit enters the addition input end of the next operation unit in the same column through the pipeline temporary storage channel; the operation result processing unit opens the transparent transmission channel.

[0033] In the embodiment of the present disclosure, the operation circuit may include an operation unit array, a decoding unit, a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of operation result processing units. Among them, the decoding unit is used to determine the working mode according to the operation indication and generate channel control information. The working mode includes a matrix operation working mode and a convolution operation working mode. In the matrix operation working mode, the row broadcast bus is used to transmit the row data of the corresponding first sub-matrix, and the column broadcast bus is used to transmit the column data of the corresponding second sub-matrix; in the convolution operation working mode, the row broadcast bus is used to transmit the corresponding data to be convolved. A plurality of first data transmission modules are provided on each row broadcast bus, and each operation unit outside the first column corresponds to a first data transmission module respectively. The first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel; the first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the indication of the channel control information. The output end of the target operation unit is provided with a second data transmission module, and the second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel; the second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next operation unit in the same column through the pipeline temporary storage channel. The operation result processing unit includes a transparent transmission channel and an addition channel, and the operation result processing unit opens the transparent transmission channel or the addition channel according to the instruction of the channel control information. Each operation unit array reuses the broadcast bus, or each operation unit array uses an independent broadcast bus, and the broadcast bus includes a row broadcast bus and / or a column broadcast bus.

[0034] Based on any of the arithmetic circuit embodiments of the second aspect described above, the arithmetic circuit may further include an accumulation processing unit. In a matrix operation mode, the accumulation processing unit is configured to accumulate the results of a matrix operation in each arithmetic unit, where a matrix operation includes a matrix multiplication operation of a first matrix and a second matrix. In a convolution operation mode, the accumulation processing unit is configured to accumulate the results of a convolution operation on a convolution object in the arithmetic units in the same column.

[0035] According to another aspect of the present disclosure, the present disclosure provides an arithmetic circuit, comprising an array of a plurality of arithmetic units, a plurality of row broadcast buses, and a plurality of column broadcast buses. The row broadcast bus is used to transmit data operated by the corresponding row arithmetic unit. A plurality of first data transmission modules are provided on each row broadcast bus. Each arithmetic unit outside the first column corresponds to a first data transmission module. The first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to an instruction. The column broadcast bus is used to transmit data operated by the corresponding column arithmetic unit. Each arithmetic unit is used to perform a point multiplication operation on a set of data pairs. The output end of the target arithmetic unit is provided with a second data transmission module. The second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to an instruction, and outputs the operation result to the corresponding arithmetic result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next arithmetic unit in the same column through the pipeline temporary storage channel. The target arithmetic unit includes at least each arithmetic unit outside the last row.

[0036] On this basis, the arithmetic circuit may further include multiple arithmetic result processing units. The arithmetic result processing units include a transparent transmission channel and an addition channel. The arithmetic result processing units activate the transparent transmission channel or the addition channel according to the instruction, and transparently transmit the arithmetic result of the corresponding column arithmetic unit through the transparent transmission channel, and add the arithmetic results of the corresponding column arithmetic unit through the addition channel and output them.

[0037] According to another aspect of the present disclosure, the present disclosure provides a processor, which includes the operation circuit described in any one of the above embodiments.

[0038] According to another aspect of the present disclosure, an integrated circuit system is provided, comprising the arithmetic circuit described in any one of the above embodiments.

[0039] According to another aspect of the present disclosure, an electronic component is provided, comprising the processor described in the foregoing embodiment and / or the integrated circuit system described in the foregoing embodiment.

[0040] According to another aspect of the present disclosure, an electronic device is provided, comprising the electronic component described in any one of the above embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] FIG1 is a schematic diagram of the structure of an arithmetic circuit provided by an embodiment of the present disclosure;

[0042] FIG2 is a schematic diagram of the structure of a three-dimensional multiply-add array provided by one embodiment of the present disclosure;

[0043] FIG3 is a schematic structural diagram of a DOT unit provided by an embodiment of the present disclosure;

[0044] FIG4 is a schematic diagram of the structure of PB and KB provided in one embodiment of the present disclosure;

[0045] FIG5 is a schematic diagram of the structure of an add mux provided by an embodiment of the present disclosure;

[0046] FIG6 is a schematic structural diagram of a connection unit provided in one embodiment of the present disclosure;

[0047] FIG7 is a schematic diagram of the structure of an accumulation processing unit provided by one embodiment of the present disclosure;

[0048] FIG8 is a schematic diagram of a convolution sliding window movement according to an embodiment of the present disclosure;

[0049] FIG9 is a schematic diagram of pipeline data transmission and calculation according to an embodiment of the present disclosure;

[0050] FIG10 is a schematic diagram of pulsatile data transmission and calculation according to an embodiment of the present disclosure;

[0051] FIG11 is a schematic diagram of data transmission and calculation in a broadcast mode according to an embodiment of the present disclosure;

[0052] FIG12 is a schematic diagram of a convolution operation process provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0053] Before introducing the embodiments of the present disclosure, it should be noted that:

[0054] Some embodiments of the present disclosure are described as processing flows. Although the various operation steps of the flow may be given sequential step numbers, the operation steps therein may be implemented in parallel, concurrently, or simultaneously.

[0055] In the embodiments of the present disclosure, the terms "first", "second", etc. may be used to describe various features, but these features should not be limited by these terms. These terms are used only to distinguish one feature from another.

[0056] The term “and / or” may be used in embodiments of the present disclosure. “And / or” includes any and all combinations of one or more of the listed associated features.

[0057] It should be understood that when describing the connection relationship or communication relationship between two components, unless it is explicitly stated that the two components are directly connected or directly communicating, the connection or communication between the two components can be understood as direct connection or communication, or as indirect connection or communication through an intermediate component.

[0058] In order to make the technical solutions and advantages of the embodiments of the present disclosure more clearly understood, the exemplary embodiments of the present disclosure are further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, and are not an exhaustive list of all the embodiments. It should be noted that the embodiments and features in the embodiments of the present disclosure can be combined with each other unless they conflict.

[0059] The purpose of the present disclosure is to provide an operation circuit that can be compatible with matrix operations and convolution operations with higher operation efficiency. Specifically, in the matrix operation working mode, a single operation unit in the operation unit array can be used to perform a point multiplication operation on a single row of data of a matrix and a single column of data of another matrix. In the convolution operation working mode, a single operation unit in the operation unit array can be used to perform a point multiplication operation on a single set of convolution kernel data corresponding to the convolution kernel and a single set of data to be convolved of the convolution object in a single convolution sliding window, wherein the operation units in the same column correspond to the same convolution kernel, and the operation units in different columns correspond to different convolution kernels, a single set of convolution kernel data includes convolution kernel data of multiple channels at the same position, and a single set of data to be convolved includes data to be convolved of multiple channels at the same position. The operation unit array provided in the embodiment of the present disclosure is a three-dimensional multiplication-addition circuit, which can realize matrix multiplication or multiplication-addition operations with higher operation efficiency. By reusing the three-dimensional multiplication-addition circuit in this manner, HWC three-dimensional convolution operations can be performed without expanding the convolution object and convolution kernel or converting the matrix format. Since no format conversion is required before the calculation, the calculated feature map also does not need to be converted, thereby improving the efficiency of the convolution operation. In addition, the arithmetic circuit provided by the embodiments of the present disclosure is highly versatile and can be applied to chips such as GPUs, GPGPUs, NPUs, and ASICs.

[0060] One embodiment of the present disclosure provides an arithmetic circuit comprising at least an array of arithmetic units. In the embodiments of the present disclosure, the arithmetic units are logically arranged to form an array, and their physical spatial arrangement is not limited. Of course, for considerations such as production process and space conservation, it is preferred that the arithmetic units be physically arranged in an array.

[0061] In matrix operation mode, a single arithmetic unit in the arithmetic unit array performs a dot product operation on a single row of data in the first matrix and a single column of data in the second matrix. For example, an M×N arithmetic unit array implements matrix multiplication of an M×K matrix A and a K×N matrix B. The arithmetic unit in row i and column j performs a dot product operation on the data in row i of matrix A and the data in column j of matrix B. The result of the operation becomes the element data in row i and column j of the result matrix D.

[0062] The operation circuit provided by the embodiment of the present disclosure supports matrix multiplication operations. Furthermore, the operation circuit provided by the embodiment of the present disclosure can also support matrix multiplication and addition operations. If the operation circuit supports matrix multiplication and addition operations, then, in the matrix operation working mode, the operation unit is also used to perform addition operations on the multiplication operation results of the single row data of the first matrix and the single column of the second matrix and the element data of the third matrix. Take the M×N operation unit array to implement the above-mentioned matrix multiplication and addition operation of A matrix × B matrix + M×N C matrix as an example. The operation unit of the i-th row and j-th column performs a dot multiplication operation on the i-th row data of the A matrix and the j-th column data of the B matrix, and adds its operation result to the element data cij of the i-th row and j-th column of the C matrix as the element data of the i-th row and j-th column of the result matrix D.

[0063] In the convolution operation working mode, a single operation unit in the operation unit array is used to perform a point multiplication operation on a single set of convolution kernel data corresponding to the convolution kernel and a single set of data to be convolved of the convolution object within a single convolution sliding window. Operation units in the same column correspond to the same convolution kernel, and operation units in different columns correspond to different convolution kernels. A single set of convolution kernel data includes convolution kernel data of multiple channels at the same position, and a single set of data to be convolved includes data to be convolved of multiple channels at the same position.

[0064] In the embodiment of the present disclosure, the convolution object may be an image or a signal in other formats, which is not limited in the present disclosure.

[0065] The convolution operation works as follows: a convolution kernel is moved over the convolution object at a specific step size. Each move performs a point-wise multiplication on the corresponding points, and the sum of all these products is used as the convolution result at that position. When the convolution object has multiple channels, the convolution kernels must have the same number of channels. Each convolution kernel channel is convolved with the corresponding channel of the convolution object, and the convolution results of all channels are added bitwise to produce a final channel feature map, known as multi-channel convolution. When the input image is convolved with multiple different convolution kernels, a feature map containing multiple channels is generated.

[0066] Based on the above operation principle, take the example of performing a convolution operation on an image of size Hi×Wi×C using multiple Hw×Ww×C convolution kernels. Each convolution kernel moves across the image in a predetermined order with a moving step of 1. The overlapping portion of the convolution kernel and the image is called a convolution window. The size of the convolution window is Hw×Ww×C, and the convolution window moves with the convolution kernel. If the convolution operation is implemented using the above-mentioned M×N operation unit array, assuming that the value of N is the same as the number of convolution kernels and the value of M is the same as the value of Hw×Ww, then: the operation unit in the i-th row and j-th column performs a dot product operation on the i-th group of convolution kernel data of the j-th convolution kernel and the i-th group of data to be convolved in the image within a single convolution window. As for how many cycles the convolution operation of a single sliding window is completed, as well as the execution timing of the convolution operation of a single sliding window on each operation unit, and the execution timing of the convolution operation of multiple sliding windows on each operation unit, this disclosure does not limit it.

[0067] In the embodiment of the present disclosure, the data to be calculated can be delivered to the calculation unit through a variety of transmission methods, and the present disclosure does not limit this. As an example and not a limitation, data transmission can be carried out by means of a data bus to improve data transmission efficiency. Furthermore, the calculation circuit provided by the embodiment of the present disclosure may include multiple row broadcast buses and multiple column broadcast buses. Accordingly, in the matrix operation working mode, the single row data of the first matrix is ​​transmitted through the corresponding row broadcast bus (for example, the i-th row data of the A matrix is ​​transmitted through the i-th row broadcast bus), and the single column data of the second matrix is ​​transmitted through the corresponding column broadcast bus (for example, the j-th column data of the B matrix is ​​transmitted through the j-th column broadcast bus). Accordingly, in the convolution operation working mode, each group of data to be convolved in a single convolution sliding window is respectively transmitted through the corresponding row broadcast bus, and each group of convolution kernel data of the convolution kernel can be transmitted through the corresponding column broadcast bus.

[0068] In the convolution operation mode, a weight station method can be used, that is, during the convolution operation of the convolution object, the convolution kernel data only needs to be read and loaded once. Of course, other data flow methods can also be used, and this disclosure does not limit this.

[0069] As described above, the present disclosure does not limit the transmission method of the data to be operated, nor does it limit the execution timing of the operation unit.

[0070] For matrix operations, the row data of the first matrix and the column data of the second matrix can be transmitted via broadcast. For example, the data in row i of the first matrix is ​​broadcast via the i-th row broadcast bus, and each operation unit in row i obtains the data in row i via the i-th row broadcast bus. Transmitting data via broadcast can complete data delivery within a single clock cycle, resulting in high data transmission efficiency. Accordingly, for matrix operations, the operation unit array can complete the operation within a single clock cycle, achieving high operational efficiency. Furthermore, the operation result of each operation unit is the data element of the result matrix D, requiring no additional processing.

[0071] For convolution operations, each operation unit uses different convolution kernel data. The convolution kernel data can be transmitted by broadcast, on-demand, or in other ways.

[0072] Taking the broadcast transmission of convolution kernel data as an example, the convolution kernel data of the jth convolution kernel is broadcast through the jth column broadcast bus, and the i-th row and j-th column operation unit identifies and obtains the i-th group of convolution kernel data of the j-th convolution kernel from the broadcast data. There are many ways for the operation unit to identify the target data. For example, the target data can be identified by data identification (such as coordinate information). In addition, the convolution kernel data can be pre-loaded to each operation unit or loaded into each operation unit in real time during the operation process.

[0073] If the convolution kernel data is preloaded into each computing unit, for example, in a fixed-weight mode, multiple third data transmission modules can be provided on each column broadcast bus, one for each computing unit. The third data transmission module includes a transparent transmission channel and a convolution kernel temporary storage channel. The third data transmission module opens the transparent transmission channel or the convolution kernel temporary storage channel according to the instructions. The convolution kernel temporary storage channel of the third data transmission module temporarily stores the convolution kernel data used by the corresponding computing unit.

[0074] For convolution operations, the operation units in the same row need to operate on the same data to be convolved in the same convolution sliding window, and the operation units in the same column need to operate on different groups of data to be convolved in the same convolution sliding window. Accordingly, broadcasting, pipelining, pulsation and other methods can be used to transmit the data to be convolved.

[0075] If the data to be convolved is transmitted in a broadcast manner, the data to be convolved within a convolution sliding window can be broadcast in each clock cycle. Specifically, within one clock cycle, the i-th group of data to be convolved within the convolution sliding window is broadcast via the i-th row broadcast bus; accordingly, the operation unit array can complete the dot multiplication operation of a group of convolution kernel data and a group of data to be convolved within one clock cycle, and the operation results corresponding to a convolution kernel are added as the convolution result of the convolution kernel at a single convolution sliding window position. In order to achieve the addition of the operation results, the operation results corresponding to a convolution kernel can be input to the adder for addition, or they can be accumulated between corresponding operation units (for example, operation units in the same column). For the broadcast method, it is more efficient to input the operation results corresponding to a convolution kernel to the adder for addition. Accordingly, as an operation circuit compatible with matrix operations and convolution operations, for the output of the operation unit, if it is a matrix operation result, it can be output directly. For the convolution operation result, it needs to be added with the operation result of the corresponding operation unit before output. The embodiment of the present disclosure does not limit the specific hardware form for achieving such compatibility. As an example and not a limitation, on the one hand, the embodiments of the present disclosure may further include a plurality of operation result processing units, and the operation result processing units include a transparent transmission channel and an addition channel. For matrix operations, the operation results of the operation units in the same column are output through the transparent transmission channel of the corresponding operation result processing unit. For convolution operations, if the operation results have been accumulated between the operation units in the same column, then the operation results of the last operation unit in the same column are output through the transparent transmission channel of the corresponding operation result processing unit; if the operation results have not been accumulated between the operation units in the same column, then the operation results of the operation units in the same column are added through the addition channel and then output. On the other hand, the output end of the target operation unit may further include a second data transmission module, and the second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. For matrix operations, the second data transmission module opens the transparent transmission channel and transparently transmits the matrix operation results. For convolution operations, if the operation results need to be accumulated between the operation units in the same column, then the second data transmission module opens the pipeline temporary storage channel to output the operation results to the addition input end of the next operation unit in the same column; if the operation results do not need to be accumulated between the operation units in the same column, then the second data transmission module opens the transparent transmission channel to transparently transmit the convolution operation results, and the convolution operation results of the operation units in the same column are uniformly added and processed.

[0076] The target operation unit includes at least each operation unit except the last row.

[0077] If pipeline transmission of the data to be convolved is used, the data to be convolved within a convolution window is delivered to each arithmetic unit over multiple clock cycles. During each clock cycle, the data to be convolved within a different convolution window can be sent to the arithmetic unit in the first column. Accordingly, to implement pipeline transmission of the data to be convolved and maintain compatibility with broadcast data transmission, multiple first data transmission modules can be provided on each row broadcast bus, with each arithmetic unit outside the first column corresponding to a first data transmission module. The first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. For matrix operations, the first data transmission module enables the transparent transmission channel. For convolution operations, the first data transmission module enables the pipeline temporary storage channel so that each set of convolution data within a single convolution window is delivered sequentially to each arithmetic unit in the same row in a pipelined manner. Since each arithmetic unit in the same column receives the data to be convolved within the same convolution window during the same clock cycle, the computation results corresponding to a convolution kernel can be input into an adder (e.g., the addition channel of the computation result processing unit) for addition, as described above. Of course, the above implementation method can also be referred to, and the second data transmission module can be used to open a pipeline temporary storage channel to accumulate convolution operation results between corresponding operation units.

[0078] If a pulsating transmission method is used for the data to be convolved, the data to be convolved within a convolution window needs to be delivered to each arithmetic unit over multiple clock cycles. In this case, the data to be convolved within different convolution windows can be sent to the arithmetic units in the first column during each clock cycle. Accordingly, to achieve pulsating transmission of the data to be convolved and compatible with the broadcast data transmission method, a flexible combination of the first data transmission module, the second data transmission module, and the calculation result processing unit can be used. For convolution operations, the first data transmission module enables a pipeline temporary storage channel so that each set of convolution data within each convolution window is delivered to each arithmetic unit in the same row in a pulsating manner. In the pulsating transmission mode, since each arithmetic unit in the same column cannot receive the data to be convolved within the same convolution window within the same clock cycle, the calculation results need to be accumulated between the arithmetic units in the same column. Therefore, the second data transmission module enables a pipeline temporary storage channel so that the calculation result of the current arithmetic unit can be passed through the pipeline temporary storage channel to the addition input of the next arithmetic unit in the same column. Accordingly, if there is a calculation result processing unit, the calculation result processing unit enables a transparent transmission channel.

[0079] Based on any of the above implementations, the computing circuit provided in the embodiment of the present disclosure may further include a decoding unit for determining an operating mode based on an operation instruction and generating channel control information, wherein the operating mode includes a matrix operation mode and a convolution operation mode. The above channel control information may instruct the first data transmission module to open a transparent transmission channel or a pipeline temporary storage channel. The channel control information may also instruct the second data transmission module to open a transparent transmission channel or a pipeline temporary storage channel. The channel control information may also instruct the operation result processing unit to open a transparent transmission channel or an addition channel.

[0080] Based on any of the above implementations, the operation circuit provided by the embodiment of the present disclosure may further include at least one of the following units: an input cache unit, a matrix data fetching and sending unit, and a convolution data fetching and sending unit.

[0081] The input buffer unit is used to cache the data of the matrix to be operated on, or the convolution object (such as image, intermediate layer activation data), convolution kernel or weight, bias and other data to be operated on.

[0082] The matrix data fetching and sending unit is used to read the matrix data to be operated from the input buffer unit and send it to the operation unit array in the matrix operation working mode.

[0083] The convolution data fetch and send unit is used to read data from the input buffer unit and send it to the operation unit array according to the convolution operation data format requirements in the convolution operation working mode.

[0084] Based on any of the above-mentioned operation circuit embodiments, the operation circuit may further include an accumulation processing unit. In the matrix operation working mode, the accumulation processing unit is used to accumulate the operation results of a matrix operation in each operation unit respectively, and a matrix operation includes a matrix multiplication operation of a first matrix and a second matrix. In the convolution operation working mode, the accumulation processing unit is used to accumulate the operation results of the convolution operation of the convolution object in the operation units in the same column respectively. Due to the size of the operation unit array, when the matrix operation or convolution operation cannot be completed in one go, the operation of a large-size operation object can be achieved by performing multiple operations and accumulating the operation results.

[0085] In the embodiment of the present disclosure, whether it is a matrix operation or a convolution operation, there may be 0-value data. Since 0-value data does not need to participate in the operation, the operation efficiency can be further improved by compressing and discarding the 0-value data pairs. Then, the operation circuit provided by the embodiment of the present disclosure may also include a plurality of compression cache units, and each operation unit corresponds to a compression cache unit. The compression cache unit is used to cache two sets of data that need to be calculated by the corresponding operation unit (such as the row data of the first matrix and the column data of the second matrix, or a set of convolution kernel data and a set of data to be convolved), discard the 0-value data pairs therein to achieve data compression, and output the compressed data to the corresponding operation unit. Accordingly, the data on the bus first enters the compression cache unit, and then enters the corresponding operation unit through the compression cache unit.

[0086] In the disclosed embodiment, there are multiple ways to implement the compression cache unit to discard the 0-value data pairs. The 0-value data pairs can be discarded by deleting the 0-value data pairs, or by not sending the 0-value data pairs to the operation unit. If the 0-value data pairs are not sent to the operation unit, more specifically, after the 0-value data pairs are found, they can be marked, and when the data pairs are sent to the operation unit, the data pairs marked as 0 are not sent. It is also possible to synchronously search for 0-value data pairs and send compressed data pairs to the operation unit, that is, traverse the data pairs in order, and determine whether it is a 0-value data pair each time a data pair is traversed. If it is not a 0 value, the data pair is sent to the operation unit, otherwise it is not sent.

[0087] Regardless of the method used to discard the 0-value data pairs, there are multiple ways to implement the compression cache unit to search for the 0-value data pairs. In one feasible implementation, the compression cache unit searches for the 0-value data pairs based on the first bitmap, wherein the first bitmap is used to mark whether the value of the data pair is 0. In another feasible implementation, the compression cache unit searches for the 0-value data based on the second bitmap, and the data pair corresponding to the 0-value data is the 0-value data pair, wherein the second bitmap is used to determine whether each data in a group of data participating in the operation (such as a row of data in the first matrix, a column of data in the second matrix, a group of convolution kernel data, or a group of data to be convolved) is 0. In yet another feasible implementation, the compression cache unit directly identifies whether there is 0-value data in a group of cached data, and the data pair corresponding to the 0-value data is the 0-value data pair.

[0088] In the disclosed embodiment, when an operation is completed in a certain operation unit, the clocks of the operation unit and the corresponding compression cache unit are turned off until the next operation begins, thereby further reducing power consumption. An operation is a multiplication operation of a first matrix by a second matrix, or a convolution operation of a convolution object.

[0089] Another embodiment of the present disclosure provides an arithmetic circuit comprising at least two arithmetic unit arrays of the same size. In a matrix operation mode, a single arithmetic unit in each arithmetic unit array is configured to perform a dot multiplication operation on a single row of data in a first submatrix and a single column of data in a second submatrix. In a convolution operation mode, a single arithmetic unit in each arithmetic unit array is configured to perform a dot multiplication operation on a single set of convolution kernel data corresponding to a subconvolution kernel and a single set of to-be-convolved data of a subconvolution object within a single convolution sliding window. Arithmetic units in the same column correspond to the same subconvolution kernel, while arithmetic units in different columns correspond to different subconvolution kernels. The single set of convolution kernel data includes convolution kernel data for multiple channels at the same position, and the single set of to-be-convolved data includes to-be-convolved data for multiple channels at the same position. The at least two arithmetic unit arrays are connected by a connection unit to implement matrix multiplication between the first matrix and the second matrix, or convolution operation of the convolution kernel on the convolution object. The first matrix is ​​divided into a plurality of first submatrices in the row direction, and the second matrix is ​​divided into a plurality of second submatrices in the column direction. The convolution kernel is divided into several sub-convolution kernels in the channel direction, and the convolution object is divided into several sub-convolution objects in the channel direction.

[0090] In the embodiments of the present disclosure, the same size of the arithmetic unit arrays means that the arithmetic unit arrays include the same number of arithmetic units (the same number of rows and columns). The computing capabilities of the arithmetic units in different arrays may be different. For example, the arithmetic units in one array may complete the dot product operation of 8 pairs of data in one clock cycle, while the arithmetic units in another array may complete the dot product operation of 16 pairs of data in one clock cycle.

[0091] The specific implementation of this embodiment can refer to the implementation of the arithmetic circuit of the above-mentioned single arithmetic unit array, and the repeated parts will be omitted.

[0092] In the disclosed embodiment, each arithmetic unit array can reuse a broadcast bus (a row broadcast bus and / or a column broadcast bus). For example, the i-th row broadcast bus is used to transmit data to the i-th row arithmetic unit of each arithmetic unit array. Each arithmetic unit array can also use an independent broadcast bus, for example, each arithmetic unit array has its own independent row broadcast bus and column broadcast bus.

[0093] Furthermore, the arithmetic circuit comprising at least two arithmetic unit arrays may further include an accumulation processing unit. In a matrix operation mode, the accumulation processing unit is configured to separately accumulate the operation results of a matrix operation in each arithmetic unit, where a matrix operation includes a matrix multiplication operation of a first matrix and a second matrix. In a convolution operation mode, the accumulation processing unit is configured to separately accumulate the operation results of the convolution operation of the convolution object in the arithmetic units in the same column. The accumulation processing unit can be used to accumulate the operation results of matrices of different arithmetic units.

[0094] Another embodiment of the present disclosure provides an arithmetic circuit, comprising an array of several arithmetic units, a plurality of row broadcast buses and a plurality of column broadcast buses. The row broadcast bus is used to transmit data operated by the corresponding row arithmetic unit, and a plurality of first data transmission modules are provided on each row broadcast bus. Each arithmetic unit outside the first column corresponds to a first data transmission module, and the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction. The column broadcast bus is used to transmit data operated by the corresponding column arithmetic unit. Each arithmetic unit is used to perform a point multiplication operation on a group of data pairs. The output end of the target arithmetic unit is provided with a second data transmission module, and the second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next arithmetic unit in the same column through the pipeline temporary storage channel.

[0095] On this basis, the operation circuit may optionally further include multiple operation result processing units. The operation result processing units include a transparent transmission channel and an addition channel. The operation result processing units activate the transparent transmission channel or the addition channel according to the instruction, transparently transmit the operation results of the corresponding column operation units through the transparent transmission channel, and add the operation results of the corresponding column operation units through the addition channel and output them.

[0096] In this embodiment, multiple third data transmission modules are optionally provided on each column broadcast bus, with one corresponding to each computing unit. The third data transmission module includes a transparent transmission channel and a convolution kernel temporary storage channel. The third data transmission module opens the transparent transmission channel or the convolution kernel temporary storage channel according to the instruction. The convolution kernel temporary storage channel of the third data transmission module temporarily stores the convolution kernel data used by the corresponding computing unit.

[0097] The computational circuit provided by the disclosed embodiments is compatible with both matrix and convolution operations. By selecting and coordinating various channels, it is possible to switch between matrix and convolution operations and flexibly implement multiple computation modes for the same operation.

[0098] Among them, for matrix operations, you can refer to the description of the aforementioned embodiments, transmit data by broadcasting, and then implement matrix operations, or implement matrix operations by pipeline transmission, pulsating transmission, etc. Based on the content disclosed in the embodiments of the present disclosure, those skilled in the art can know the specific implementation method without creative work, and will not go into details here.

[0099] For convolution operations, the description of the preceding embodiments can be referred to, and data can be transmitted through broadcasting, pipelining, pulsation, or other methods to implement convolution operations. Other methods can also be used to implement convolution operations. Based on the content disclosed in the embodiments of this disclosure, those skilled in the art can easily understand specific implementation methods without inventive effort, and will not be further described here.

[0100] Another embodiment of the present disclosure provides an arithmetic circuit, as shown in FIG1 , comprising an input buffer unit, a decoding unit, a matrix data fetch and feed unit, a convolution data fetch and feed unit, a three-dimensional multiply-add network, and an accumulation processing unit. The three-dimensional multiply-add network comprises a plurality of three-dimensional multiply-add arrays (i.e., an arithmetic unit array). The arithmetic circuit can support three data transmission and calculation modes: pipeline, systolic, and broadcast. Furthermore, by connecting multiple three-dimensional multiply-add arrays (in the spatial dimension) and / or accumulating caches (in the temporal dimension), it can flexibly support matrix multiplication / multiply-add operations of various data types and sizes, as well as convolution operations of various data types and convolution kernels of various sizes.

[0101] The input buffer unit is used to cache the data of matrices to be processed. Assuming that the operation circuit shown in Figure 1 is needed to complete the matrix multiplication operation of matrix A and matrix B, then matrix A and matrix B will be cached in the input buffer unit. The input buffer unit is also used to cache data such as convolution objects (such as images, intermediate layer activation data), convolution kernels or weights, biases, etc. to be processed. Assuming that the operation circuit shown in Figure 1 is needed to complete the convolution operation of several convolution kernels on the convolution object, then the convolution kernels, convolution objects, and bias data will be cached in the input buffer unit.

[0102] The decoding unit is used to determine the operating mode based on the received operation instruction and generate control information (including data fetch and send control information and channel control information). The operating modes include matrix operation mode and convolution operation mode. The operation instruction can be an operation command (such as a command used by an ASIC) or an operation instruction (such as a SIMD processor instruction).

[0103] If the decoding unit receives a matrix operation command / instruction, instructing the operation circuit to perform a matrix multiplication operation or a matrix multiplication-addition operation, the decoding unit generates data fetch and feed control information (for example, including information such as matrix data address and matrix size), operation control information (for example, including matrix data type, such as FP32, FP16, matrix operation type, such as matrix multiplication, matrix multiplication-addition), and accumulation control information based on the matrix operation command / instruction, and sends the data fetch and feed control information to the matrix fetch and feed unit to instruct the matrix fetch and feed unit to read the data corresponding to the matrix operation command / instruction and send it into the three-dimensional multiplication-addition network, sends the operation control information to the three-dimensional multiplication-addition network to instruct the three-dimensional multiplication-addition network to perform the matrix operation, and sends the accumulation control information to the accumulation processing unit. Among them, each control information can be, but is not limited to, sent to the corresponding register so that the corresponding unit can read the control information from the corresponding register and perform the corresponding operation. In this embodiment, the operation control information is the channel control information.

[0104] If the decoding unit receives a convolution operation command / instruction and instructs the operation circuit to perform a convolution operation, the decoding unit generates data fetch and data feed control information (for example, including information such as the convolution kernel data address, the convolution kernel size, the convolution object data address, the convolution object size, and the bias data address), operation control information (for example, including data types such as FP32 and FP16, and operation types such as convolution operations) and accumulation control information based on the convolution operation command / instruction, and sends the data fetch and data feed control information to the convolution fetch and data feed unit to instruct the convolution fetch and data feed unit to read the data corresponding to the convolution operation command / instruction and send it into the three-dimensional multiplication-addition network, sends the operation control information to the three-dimensional multiplication-addition network to instruct the three-dimensional multiplication-addition network to perform the convolution operation, and sends the accumulation control information to the accumulation processing unit. Among them, each control information can be, but is not limited to, sent to the corresponding register so that the corresponding unit can read the control information from the corresponding register and perform the corresponding operation. In this embodiment, the operation control information is the channel control information.

[0105] The matrix data fetching and sending unit is used to read the matrix data to be processed from the input buffer unit and send the data to the broadcast bus according to the above-mentioned data fetching and sending control information.

[0106] The convolution data fetching and sending unit is used to read the data to be convolutionally calculated from the input buffer unit according to the above-mentioned data fetching and sending control information and the convolution calculation data format requirements and send the data to the broadcast bus.

[0107] The three-dimensional multiply-add network controls each data transmission module (the first data transmission module and the second data transmission module) to open corresponding channels based on the operation control information, and controls the operation unit to perform operations on the input data. The three-dimensional multiply-add network includes several three-dimensional multiply-add arrays. If the three-dimensional multiply-add network includes two or more three-dimensional multiply-add arrays, each three-dimensional multiply-add array can operate independently. In addition, two or more three-dimensional multiply-add arrays can be spliced ​​into a larger three-dimensional multiply-add array via a connection unit, thereby enabling operations on large-scale operation objects.

[0108] A single three-dimensional multiply-add array consists of M×N DOT units (i.e., arithmetic units), M row broadcast buses, N column broadcast buses, M×N output data buses (the output data buses serve as the output terminals of the corresponding DOT units), and N Add Muxes (addition multiplexers, i.e., the units that process the arithmetic results). Each row broadcast bus is equipped with N-1 pipe buffers (pipe buffers, i.e., the first data transmission module). The output data buses of all DOT units except the last row are equipped with PBs (i.e., the second data transmission module). Each column broadcast bus is equipped with N kernel buffers (KBs, i.e., the third data transmission module). Taking a 4×4 three-dimensional multiply-add array as an example, as shown in Figure 2, a single three-dimensional multiply-add array includes 16 DOT units, 4 row broadcast buses, 4 column broadcast buses, 16 output data buses, and 4 Add Muxes.

[0109] In the disclosed embodiment, the three-dimensional multiply-add array supports operations on a variety of data types, including but not limited to FP64, FP32, TF32, FP16, BF16, FP8, INT8, and the like.

[0110] A DOT unit is used to perform a dot multiplication operation on K data pairs (such as K pairs of floating-point numbers, or K pairs of integers). The present disclosure does not limit the specific structure and working principle of the DOT unit. As an example and not a limitation, the DOT unit shown in Figure 3 can be used. The DOT unit shown in Figure 3 is DOT8, which performs a dot multiplication operation on a set of data (such as the i-th row data of the A matrix, or a set of convolution kernel data of the convolution kernel) and another set of data (such as the j-th column data of the B matrix or a set of data to be convolved of the convolution object), and can add the result of the dot multiplication operation to another element data (element data cij of the C matrix). In the embodiment of the present disclosure, the two data that need to be dot multiplied constitute a data pair. For example, ai0 and b0j are a data pair.

[0111] The three-dimensional multiply-accumulate array shown in FIG2 implemented using the DOT unit shown in FIG3 is equivalent to 128 MAC (multiply-accumulate) units, with 256 operands per clock cycle.

[0112] In this embodiment, the PB has a transparent transmission channel and a pipeline temporary storage channel. Its structure can be, but is not limited to, that shown in Figure 4. The multiplexer mux controls the opening of the transparent transmission channel or the pipeline temporary storage channel. If the transparent transmission channel is enabled, data is output from the input end through the mux. If the pipeline temporary storage channel is enabled, data enters the reg from the input end and is output through the mux.

[0113] In this embodiment, the KB has a transparent transmission channel and a kernel temporary storage channel. Its structure can be, but is not limited to, that shown in Figure 4. A multiplexer (mux) controls the activation of either the transparent transmission channel or the kernel temporary storage channel. If the transparent transmission channel is enabled, data is output from the input port through the mux. If the kernel temporary storage channel is enabled, data is input to the register (register) and then output through the mux.

[0114] In this embodiment, the add mux has a transparent transmission channel and an addition channel. Its structure can be, but is not limited to, as shown in Figure 5, where the opening of the transparent transmission channel or the addition channel is controlled by the multiplexer mux. If the transparent transmission channel is turned on, the data from each channel enters the adder add from the input end and passes through the mux and is output from the corresponding output end. If the addition channel is turned on, the data from each channel enters the adder add from the input end for addition, and the output result passes through the mux and is output from the designated output end. In actual applications, the output of add can still be multi-channel, but the addition result is output from the designated output end (such as the solid line output end of add in Figure 5).

[0115] In this embodiment, the structure of the connection unit can be, but is not limited to, that shown in FIG6 , and the connection unit also includes a transparent transmission channel and an addition channel. For arithmetic unit arrays that need to work independently, each input of the arithmetic unit array is directly output through the corresponding multiplexer mux through its corresponding multiplexer mux. For arithmetic unit arrays that need to be spliced, each output of these arithmetic unit arrays enters the adder add respectively, and the adder adds the operation results of the corresponding outputs in these arithmetic unit arrays (for example, the first output of these arithmetic unit arrays is the corresponding output) and outputs them respectively.

[0116] In this embodiment, the structure of the accumulation processing unit can be, but is not limited to, as shown in FIG7 , where each output of the connection unit enters the adder add of the accumulation processing unit, and the multiple corresponding intermediate operation results in the cache module are simultaneously input into the adder add, and the adder add performs addition operations on the corresponding inputs and outputs them to the cache module for temporary storage. Specifically, when multiple matrix multiplication results are to be accumulated, multiple (the number of ways is equal to the number of DOT units in a column) previous matrix multiplication results are read from the cache module at the same time and added to the current matrix multiplication result, and the accumulated result is written back to the cache module. When partial convolution sums are to be accumulated, one partial convolution sum is read from the cache module and added to the current convolution sum operation result, and the accumulated result is written back to the cache module. The cache module can be a buffer or a FIFO.

[0117] 1 to 7 , an example is given in which a three-dimensional multiply-add network includes only a 4×4×8 three-dimensional multiply-add array to implement matrix multiplication of a 4×8 A matrix and an 8×4 B matrix.

[0118] The decoding unit receives a matrix multiplication command, generates data fetch and send control information, calculation control information and accumulation control information based on the matrix multiplication command, sends the data fetch and send control information to the matrix fetch and send unit, sends the calculation control information to the three-dimensional multiplication and addition array, and sends the accumulation control information to the accumulation processing unit.

[0119] After receiving the data fetch and send control information, the matrix fetch and send unit reads the data of matrix A from the input buffer unit in rows and sends them to the DOT units in the three-dimensional multiply-add array via row broadcast buses. It also reads the data of matrix B from the input buffer unit in columns and sends them to the DOT units in the three-dimensional multiply-add array via column broadcast buses. The i-th row data of matrix A is sent to the i-th row DOT unit via the i-th row broadcast bus; the j-th column data of matrix B is sent to the j-th column DOT unit via the j-th column broadcast bus. To improve processing efficiency, the matrix fetch and send unit reads and sends the data of matrix A and matrix B simultaneously.

[0120] After receiving the above-mentioned operation control information, the three-dimensional multiplication-addition array switches PB to the transparent transmission channel and switches the operation result processing unit to the transparent transmission channel. Accordingly, the data of each row of matrix A is transmitted on each row broadcast bus in a broadcast manner, and the data of each column of matrix B is transmitted on each column broadcast bus in a broadcast manner. The DOT unit performs a dot multiplication operation on the received row data of matrix A and the column data of matrix B. The operation result is output to the operation result processing unit corresponding to the operation unit of this column through the data output bus, and is output through the transparent transmission channel of the operation result processing unit. The operation result output by each DOT unit is an element data of the result matrix D.

[0121] Since there is only one three-dimensional multiplication-addition array, a connection unit can be omitted, and the data output by the calculation result processing unit directly enters the accumulation processing unit. Of course, a connection unit can also be set up. In this case, the three-dimensional multiplication-addition array switches the connection unit to a transparent transmission channel, and the data output by the calculation result processing unit enters the accumulation processing unit through the transparent transmission channel of the connection unit. The connection unit can also work in the addition channel, and the data output by the calculation result processing unit enters the accumulation processing unit through the addition channel of the connection unit (via a designated mux).

[0122] After receiving the accumulation control information, the accumulation processing unit temporarily stores the input operation result in a designated location of the cache module.

[0123] Through the operation circuit provided in this embodiment, the multiplication operation of matrix A and matrix B can be implemented in one clock cycle.

[0124] If the size of the matrix in the K direction is greater than 8, the A matrix and the B matrix can be split along the K direction. Each time the multiplication operation of a pair of sub-matrices after the split is processed, the intermediate operation results are accumulated in the accumulation processing module to obtain the matrix multiplication result of the A matrix and the B matrix.

[0125] The above describes the specific working method of using a 4×4×8 three-dimensional multiply-add array to implement matrix multiplication of a 4×8 matrix A and an 8×4 matrix B. If you want to use it to implement the multiplication and addition operation of matrix A × matrix B + matrix C, the multiplication part can refer to the above implementation method and will not be repeated here. The addition part is implemented as follows:

[0126] The matrix load and send units can send C matrix element data to the DOT units in the three-dimensional multiply-add array via the row broadcast bus or the column broadcast bus. Each DOT unit only receives the target element data, for example, DOT_00 only receives c00. The matrix load and send units can also send the corresponding C matrix element data to each DOT unit via point-to-point communication.

[0127] The connection unit can be used to concatenate multiple 3D multiply-add arrays into a larger 3D multiply-add array, thereby implementing large-scale matrix multiplication and convolution of large-scale convolution kernels. In conjunction with Figures 1 to 7, we use a 3D multiply-add network containing two 4×4×8 3D multiply-add arrays to implement matrix multiplication of a 4×16 matrix A and a 16×4 matrix B as an example.

[0128] Two three-dimensional multiply-add arrays are connected via a connection unit to add the computational results of the two three-dimensional multiply-add arrays. Specifically, the connection unit includes four adders, each of which is used to add the computational results of corresponding DOT units in the two three-dimensional multiply-add arrays. For example, the data output buses of DOT_00 of the two three-dimensional multiply-add arrays are both connected to the input of adder_00. Adder_00 adds the computational results of DOT_00 of the two three-dimensional multiply-add arrays and outputs them to the accumulation processing unit via the corresponding multiplexer mux_00 as the computational result D matrix element data d00.

[0129] After receiving the accumulation control information, the accumulation processing unit temporarily stores the input operation result in a designated location of the cache module.

[0130] The two three-dimensional multiply-add arrays may share a row broadcast bus and a column broadcast bus, or may use independent row broadcast buses and column broadcast buses.

[0131] In this embodiment, the working mode of the decoding unit can refer to the description of the above embodiment and will not be repeated here. After receiving the data fetch and send control information, the matrix data fetch and send unit reads the data of the A matrix from the input buffer unit in units of rows and sends them to the DOT units in the three-dimensional multiply-add network through the row broadcast bus, and reads the data of the B matrix from the input buffer unit in units of columns and sends them to the DOT units in the three-dimensional multiply-add network through the column broadcast bus. If two three-dimensional multiply-add arrays share the row broadcast bus and the column broadcast bus, then each DOT unit only receives the corresponding data. For example, DOT_00 of the first three-dimensional multiply-add array only receives the first 8 elements of the 0th row of the A matrix and the first 8 elements of the 0th column of the B matrix. DOT_00 of the second three-dimensional multiply-add array only receives the last 8 elements of the 0th row of the A matrix and the last 8 elements of the 0th column of the B matrix. If the two three-dimensional multiply-add arrays use independent row broadcast buses and column broadcast buses, then the matrix data fetch and send units send the corresponding matrix element data through the row broadcast buses and column broadcast buses of the two three-dimensional multiply-add arrays respectively. For example, the first 8 element data of each row of the A matrix are sent to the three-dimensional multiply-add array through the row broadcast bus of the first three-dimensional multiply-add array, and the last 8 element data of each row of the A matrix are sent to the three-dimensional multiply-add array through the row broadcast bus of the second three-dimensional multiply-add array.

[0132] The specific working method of the two three-dimensional multiplication-addition arrays can refer to the description of the above embodiment and will not be repeated here.

[0133] It is worth noting that this embodiment only illustrates a method of implementing matrix multiplication of one size by splicing two three-dimensional multiplication-addition arrays through a connection unit. In fact, the arithmetic circuit provided by the embodiment of the present disclosure can support matrix multiplication and convolution calculations of various sizes.

[0134] In conjunction with Figures 1 to 7, a 3D multiply-add network including a 4×4×8 3D multiply-add array is used as an example to implement convolution operations. In this embodiment, there are four convolution kernels of size Hw×Ww×C, the size of the convolution object is Hi×Wi×C, the shift step is 1, Hw=2, Ww=2, and C=8. The result of multiplying a 2×2×8 block of the convolution object with the corresponding point of a convolution kernel is added together to form a single point in the convolution result.

[0135] The decoding unit receives the convolution command, generates data acquisition and sending control information, calculation control information and accumulation control information based on the convolution command, sends the data acquisition and sending control information to the convolution data acquisition and sending unit, sends the calculation control information to the three-dimensional multiplication and addition network, and sends the accumulation control information to the accumulation processing unit.

[0136] After the convolution data fetch and send unit receives the data fetch and send control information, it reads out four convolution kernels from the input cache unit in sequence and writes them into a three-dimensional multiplication-addition array in the three-dimensional multiplication-addition network. In this embodiment, the convolution operation adopts a weight-fixed mode, that is, the convolution kernel only needs to be read and loaded once. Specifically, each convolution kernel is stored in a column of DOT units (more specifically, the convolution kernel data used by the DOT unit can be temporarily stored through the KB), such as the convolution kernel kernel_0 is stored in DOT_i0 (or temporarily stored in the KB corresponding to DOT_i0). A column of DOT units respectively stores a group of convolution kernel data of a convolution kernel, such as the 8 data of the convolution kernel kernel_0 at the (0,0) position are stored in DOT_00. In this embodiment, 4 columns of DOT units can realize parallel calculation of 4 convolution kernels.

[0137] After completing the convolution kernel data acquisition and delivery, the convolution data acquisition and delivery unit reads the data to be convolved within each convolution window from the input buffer unit in the order of the convolution window and sends it to the above-mentioned three-dimensional multiplication and addition array. As shown in Figure 8, the first time (step 0) reads the four sets of data to be convolved within the convolution window (C0,0, C0,1, C1,0, C1,1), the second time (step 1) reads the four sets of data to be convolved within the convolution window (C0,1, C0,2, C1,1, C1,2), and so on. Each set of data to be convolved C includes 8 points of the convolution object in the C direction (i.e., the channel direction).

[0138] The three-dimensional multiplication-addition array supports data transmission and calculation in three ways: pipeline, pulsation, and broadcast.

[0139] If a pipeline approach is used for data transmission and computation, as shown in Figure 9, the convolution data fetch and send units feed the convolution window's data to be convolved onto the row broadcast bus of the 3D multiply-add array during each clock cycle. Specifically, the four sets of data to be convolved within a convolution window are fed into four row broadcast buses.

[0140] After receiving the above-mentioned operation control information, the three-dimensional multiplication-addition array switches the PB on the row broadcast bus to the pipeline temporary storage channel, switches the PD on the data output bus to the transparent transmission channel, and switches the operation result processing unit to the addition channel. Correspondingly, the data to be convolved on the row broadcast bus is first sent to the first column of DOT units, and then the data to be convolved is transmitted in sequence between each column of DOT units in the form of a pipeline. The DOT unit performs a point multiplication operation on a set of data to be convolved and a set of convolution kernel data received. The operation result is output to the operation result processing unit corresponding to the operation unit in this column through the data output bus, and is added and output through the addition channel of the operation result processing unit. In each clock cycle, the operation results of the four DOT units in each column are added at the corresponding operation result processing unit to obtain the convolution operation result within the corresponding convolution sliding window.

[0141] If a pulsating method is used for data transmission and computation, as shown in Figure 10, the convolution data fetch and send units pulse each clock cycle to feed the data to be convolved from several convolution windows to the row broadcast bus of the three-dimensional multiply-add array. The four sets of data to be convolved within a convolution window are fed to four row broadcast buses. Specifically, during the first clock cycle, C0,0 within the convolution window corresponding to step 0 is fed to the first row broadcast bus. During the second clock cycle, C0,1 within the convolution window corresponding to step 1 is fed to the first row broadcast bus, and C0,1 within the convolution window corresponding to step 0 is fed to the second row broadcast bus. During the third clock cycle, C0,2 within the convolution window corresponding to step 2 is fed to the first row broadcast bus, C0,2 within the convolution window corresponding to step 1 is fed to the second row broadcast bus, and C1,0 within the convolution window corresponding to step 0 is fed to the third row broadcast bus, and so on.

[0142] After receiving the above-mentioned operation control information, the three-dimensional multiplication-addition array switches the PB to the pipeline temporary storage channel and switches the operation result processing unit to the transparent transmission channel. Accordingly, the data to be convolved on the row broadcast bus is first sent to the first column of DOT units. After that, the data to be convolved is transmitted in sequence between the DOT units in each column in the form of a pipeline. The DOT unit performs a point multiplication operation on a set of data to be convolved and a set of convolution kernel data received. The operation result is output to the addition input end of the next operation unit in this column through the data output bus. The operation result of the last operation unit in each column is output to the corresponding operation result processing unit through the data output bus and output through the transparent transmission channel of the operation result processing unit. The operation results of the four DOT units in each column are accumulated to obtain the convolution operation result in the corresponding convolution sliding window. The convolution operation result in each convolution sliding window requires four clock cycles to complete the calculation.

[0143] If broadcasting is used for data transmission and computation, as shown in Figure 11, the convolution data fetch and send units feed the data to be convolved for a convolution window onto the row broadcast bus of the 3D multiply-add array during each clock cycle. Specifically, the four sets of data to be convolved within a convolution window are fed into four row broadcast buses.

[0144] After receiving the above-mentioned operation control information, the three-dimensional multiplication and addition array switches the PB to the transparent channel and switches the operation result processing unit to the addition channel. Correspondingly, the data to be convolved on the row broadcast bus is sent to each DOT unit in a broadcast manner. The DOT unit performs a point multiplication operation on a set of data to be convolved and a set of convolution kernel data received from the convolution kernel. The operation result is output to the operation result processing unit corresponding to the operation unit in this column through the data output bus, and is added and output through the addition channel of the operation result processing unit. In each clock cycle, the operation results of the four DOT units in each column are added at the corresponding operation result processing unit to obtain the convolution operation result within the corresponding convolution sliding window. One clock cycle can process the operation of four convolution kernels in parallel.

[0145] Regardless of whether a pipeline, pulse, or broadcast method is used, if the three-dimensional multiplication-addition network only has one three-dimensional multiplication-addition array, a connection unit may not be provided, and the data output by the calculation result processing unit directly enters the accumulation processing unit. Of course, a connection unit can also be provided, in which case the three-dimensional multiplication-addition array switches the connection unit to a transparent transmission channel, and the data output by the calculation result processing unit enters the accumulation processing unit through the transparent transmission channel of the connection unit. The connection unit can also work in the addition channel, and the data output by the calculation result processing unit enters the accumulation processing unit through the addition channel of the connection unit (via a designated mux).

[0146] If a 3D multiply-add network has multiple 3D multiply-add arrays, you can select a specific 3D multiply-add array for convolution operations.

[0147] After receiving the accumulation control information, the accumulation processing unit temporarily stores the input operation result in a designated location of the cache module.

[0148] If the size of the convolution object in the C direction is greater than 8, the convolution object can be split into multiple blocks along the C direction and processed in batches. The intermediate operation results are accumulated in the accumulation processing module to obtain the convolution operation result of the convolution object.

[0149] In conjunction with Figures 1 to 7, a 3D multiply-add network including a 4×4×8 3D multiply-add array is used as an example to implement convolution operations. In this embodiment, there are four convolution kernels of size Hw×Ww×C, the size of the convolution object is Hi×Wi×C, the moving step is 1, Hw=4, Ww=4, and C=8. The result of multiplying a 4×4×8 block of the convolution object with the corresponding point of a convolution kernel is added together to form a single point in the convolution result.

[0150] A convolution kernel has 4×4=16 sets of convolution kernel data (each set of convolution kernel data includes 8 points in the C direction). Since a 3D multiplication-add array has only 4 DOT units per column, each DOT unit can only store one set of convolution kernel data at a time. Therefore, a 3D multiplication-add array can only store 1 / 4 of the convolution kernel at a time and can only calculate 1 / 4 of a convolution kernel. Combined with the accumulation processing unit, see Figure 12, a complete convolution calculation is ultimately achieved through four rounds of accumulation calculations (each time selecting 1 / 4 of the convolution kernel for convolution with the corresponding convolution object).

[0151] The accumulation processing unit of each round is used as follows: when partial convolution sum accumulation is to be performed, one partial convolution result is read from its cache module and accumulated with the convolution result corresponding to this calculation, and the accumulated result is written back to the cache module.

[0152] In this embodiment, the working methods of the decoding unit, the convolution data fetching and sending unit, and the three-dimensional multiplication-addition array can refer to the description of the above embodiment and will not be repeated here.

[0153] In this embodiment, due to size limitations, when a three-dimensional multiplication-addition array in a three-dimensional multiplication-addition network cannot complete the convolution calculation of a convolution kernel at one time, an accumulation processing unit is used to accumulate multiple partial convolution sums.

[0154] It is worth noting that this embodiment only uses a three-dimensional multiplication-addition array of one size to implement a convolution calculation of one size by combining an accumulation processing unit as an example. In fact, the operation circuit provided by the embodiment of the present disclosure can support three-dimensional multiplication-addition arrays of various sizes to calculate convolutions and matrix multiplications of various sizes.

[0155] The embodiments of the present disclosure provide a high-density and high-speed computing circuit that efficiently and compatibly implements matrix multiplication and convolution calculations.

[0156] Based on the same inventive concept, an embodiment of the present disclosure further provides a processor, which includes the arithmetic circuit described in any of the above embodiments. The processor may be, but is not limited to, a GPU or an NPU.

[0157] Based on the same inventive concept, an embodiment of the present disclosure further provides an integrated circuit system, which includes the arithmetic circuit described in any of the above embodiments. The integrated circuit system can be, but is not limited to, a SOC (System on Chip) chip, an ASIC chip, etc.

[0158] Based on the same inventive concept, embodiments of the present disclosure further provide an electronic component comprising the processor and / or the integrated circuit system described in any of the aforementioned embodiments. In some use cases, the electronic component is implemented as a graphics card; in other use cases, the electronic component is implemented as a CPU motherboard.

[0159] Based on the same inventive concept, embodiments of the present disclosure further provide an electronic device comprising the aforementioned electronic components. In some usage scenarios, the electronic device is a portable electronic device, such as a smartphone, tablet computer, or VR device; in other usage scenarios, the electronic device is a personal computer, game console, or the like.

[0160] Although the preferred embodiments of the present disclosure have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present disclosure.

[0161] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. An arithmetic circuit comprising an arithmetic unit array; In the matrix operation working mode, a single operation unit in the operation unit array is used to perform a dot product operation on a single row of data of the first matrix and a single column of data of the second matrix; In the convolution operation working mode, a single operation unit in the operation unit array is used to perform a point multiplication operation on a single set of convolution kernel data corresponding to the convolution kernel and a single set of data to be convolved of the convolution object within a single convolution sliding window. Operation units in the same column correspond to the same convolution kernel, and operation units in different columns correspond to different convolution kernels. A single set of convolution kernel data includes convolution kernel data of multiple channels at the same position, and a single set of data to be convolved includes data to be convolved of multiple channels at the same position.

2. The arithmetic circuit according to claim 1, further comprising a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of arithmetic result processing units; The operation result processing unit includes a transparent transmission channel and an addition channel; In the matrix operation working mode, the single row data of the first matrix is ​​transmitted through the corresponding row broadcast bus, the single column data of the second matrix is ​​transmitted through the corresponding column broadcast bus, and the operation results of the operation units in the same column are output through the transparent transmission channel of the corresponding operation result processing unit; In the convolution operation mode, each group of to-be-convolved data within a single convolution sliding window is transmitted through the corresponding row broadcast bus.

3. The arithmetic circuit according to claim 2, wherein each row broadcast bus is provided with a plurality of first data transmission modules, each arithmetic unit outside the first column corresponds to a first data transmission module, the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel, and in the matrix operation mode, the first data transmission module enables the transparent transmission channel; The output end of the target operation unit is provided with a second data transmission module, which includes a transparent transmission channel and a pipeline temporary storage channel. In the matrix operation working mode, the second data transmission module opens the transparent transmission channel, and the target operation unit includes at least each operation unit except the last row.

4. The arithmetic circuit according to claim 3, wherein in a convolution operation mode: The first data transmission module and the second data transmission module open a transparent transmission channel; The operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution sliding window through the addition channel and outputs the result.

5. The arithmetic circuit according to claim 3, wherein in a convolution operation mode: The first data transmission module opens a pipeline temporary storage channel so that each group of convolution data in a single convolution sliding window is sequentially delivered to each operation unit in the same row in a pipeline manner; The second data transmission module opens a transparent transmission channel; The operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution sliding window through the addition channel and outputs the result.

6. The arithmetic circuit according to claim 3, wherein in a convolution operation mode: The first data transmission module opens a pipeline temporary storage channel so that each group of convolution data in each convolution sliding window is sequentially delivered to each computing unit in the same row in a pulsating manner; The second data transmission module opens the pipeline temporary storage channel so that the calculation result of the current operation unit enters the addition input terminal of the next operation unit in the same column through the pipeline temporary storage channel; The calculation result processing unit opens a transparent transmission channel.

7. The arithmetic circuit according to claim 1, further comprising a decoding unit, a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of arithmetic result processing units; The decoding unit is used to determine the working mode according to the operation instruction and generate channel control information, wherein the working mode includes a matrix operation mode and a convolution operation mode; In the matrix operation working mode, the row broadcast bus is used to transmit the row data of the first matrix, and the column broadcast bus is used to transmit the column data of the second matrix; in the convolution operation working mode, the row broadcast bus is used to transmit the data to be convolved; Each row broadcast bus is provided with a plurality of first data transmission modules, each computing unit outside the first column corresponds to a first data transmission module, and the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel; the first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information; The output end of each operation unit is provided with a second data transmission module, which includes a transparent transmission channel and a pipeline temporary storage channel; the second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next operation unit in the same column through the pipeline temporary storage channel; The operation result processing unit includes a transparent transmission channel and an addition channel, and the operation result processing unit opens the transparent transmission channel or the addition channel according to the instruction of the channel control information.

8. The arithmetic circuit according to any one of claims 1 to 7, further comprising an accumulation processing unit; In the matrix operation working mode, the accumulation processing unit is used to accumulate the operation results of a matrix operation in each operation unit respectively, and the matrix operation includes a matrix multiplication operation of the first matrix and the second matrix; In the convolution operation working mode, the accumulation processing unit is used to accumulate the operation results of the convolution operation of the convolution object in the operation units in the same column respectively.

9. An arithmetic circuit comprising at least two arithmetic unit arrays of the same size; In the matrix operation working mode, a single operation unit in each operation unit array is used to perform a dot product operation on a single row of data of the first sub-matrix and a single column of data of the second sub-matrix; In the convolution operation working mode, a single operation unit in each operation unit array is used to perform a point multiplication operation on a single set of convolution kernel data corresponding to the subconvolution kernel and a single set of to-be-convolved data of the subconvolution object within a single convolution sliding window. Operation units in the same column correspond to the same subconvolution kernel, and operation units in different columns correspond to different subconvolution kernels. A single set of convolution kernel data includes convolution kernel data of multiple channels at the same position, and a single set of to-be-convolved data includes to-be-convolved data of multiple channels at the same position. The at least two operation unit arrays are connected through a connection unit to realize matrix multiplication of the first matrix and the second matrix, or convolution operation of the convolution kernel on the convolution object; the first matrix is ​​divided into several first sub-matrices in the row direction, and the second matrix is ​​divided into several second sub-matrices in the column direction; the convolution kernel is divided into several sub-convolution kernels in the channel direction, and the convolution object is divided into several sub-convolution objects in the channel direction.

10. The arithmetic circuit according to claim 9, further comprising a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of arithmetic result processing units; The operation result processing unit includes a transparent transmission channel and an addition channel; In the matrix operation working mode, the single row data of the first sub-matrix is ​​transmitted through the corresponding row broadcast bus, the single column data of the second sub-matrix is ​​transmitted through the corresponding column broadcast bus, and the operation results of the operation units in the same column in the single operation unit array are output through the transparent transmission channel of the corresponding operation result processing unit; In the convolution operation mode, each group of to-be-convolved data in a single convolution sliding window is transmitted through the corresponding row broadcast bus. Each operation unit array reuses the broadcast bus, or each operation unit array uses an independent broadcast bus, where the broadcast bus includes a row broadcast bus and / or a column broadcast bus.

11. The arithmetic circuit according to claim 10, wherein each row broadcast bus is provided with a plurality of first data transmission modules, each arithmetic unit other than the first column corresponds to a first data transmission module, the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel, and in a matrix operation mode, the first data transmission module enables the transparent transmission channel; The output end of the target operation unit is provided with a second data transmission module, which includes a transparent transmission channel and a pipeline temporary storage channel. In the matrix operation working mode, the second data transmission module opens the transparent transmission channel, and the target operation unit includes at least each operation unit except the last row.

12. The operation circuit according to claim 11, wherein in a convolution operation mode: The first data transmission module and the second data transmission module open a transparent transmission channel; The operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution sliding window through the addition channel and outputs the result.

13. The operation circuit according to claim 11, wherein in a convolution operation mode: The first data transmission module opens a pipeline temporary storage channel so that each group of convolution data in a single convolution sliding window is sequentially delivered to each operation unit in the same row in a pipeline manner; The second data transmission module opens a transparent transmission channel; The operation result processing unit adds the operation results of the corresponding operation units in the same column for a single convolution sliding window through the addition channel and outputs the result.

14. The operation circuit according to claim 11, wherein in a convolution operation mode: The first data transmission module opens a pipeline temporary storage channel so that each group of convolution data in each convolution sliding window is sequentially delivered to each computing unit in the same row in a pulsating manner; The second data transmission module opens the pipeline temporary storage channel so that the calculation result of the current operation unit enters the addition input terminal of the next operation unit in the same column through the pipeline temporary storage channel; The calculation result processing unit opens a transparent transmission channel.

15. The arithmetic circuit according to claim 9, further comprising a decoding unit, a plurality of row broadcast buses, a plurality of column broadcast buses and a plurality of arithmetic result processing units; The decoding unit is used to determine the working mode according to the operation instruction and generate channel control information. The working mode includes matrix operation Working mode and convolution operation working mode; In the matrix operation mode, the row broadcast bus is used to transmit the row data of the corresponding first sub-matrix, and the column broadcast bus is used to transmit the column data of the corresponding second sub-matrix; in the convolution operation mode, the row broadcast bus is used to transmit the corresponding data to be convolved; Each row broadcast bus is provided with a plurality of first data transmission modules, each computing unit outside the first column corresponds to a first data transmission module, and the first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel; the first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information; The output end of each operation unit is provided with a second data transmission module, which includes a transparent transmission channel and a pipeline temporary storage channel; the second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction of the channel control information, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next operation unit in the same column through the pipeline temporary storage channel; The operation result processing unit includes a transparent transmission channel and an addition channel, and the operation result processing unit opens the transparent transmission channel or the addition channel according to the instruction of the channel control information; Each operation unit array reuses the broadcast bus, or each operation unit array uses an independent broadcast bus, where the broadcast bus includes a row broadcast bus and / or a column broadcast bus.

16. The arithmetic circuit according to any one of claims 9 to 15, further comprising an accumulation processing unit; In the matrix operation working mode, the accumulation processing unit is used to accumulate the operation results of a matrix operation in each operation unit respectively, and the matrix operation includes a matrix multiplication operation of the first matrix and the second matrix; In the convolution operation working mode, the accumulation processing unit is used to accumulate the operation results of the convolution operation of the convolution object in the operation units in the same column respectively.

17. An arithmetic circuit comprising a plurality of arithmetic unit arrays, a plurality of row broadcast buses, and a plurality of column broadcast buses; The row broadcast bus is used to transmit data calculated by the corresponding row operation unit. Each row broadcast bus is provided with multiple first data transmission modules. Each operation unit outside the first column corresponds to a first data transmission module. The first data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The first data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction. The column broadcast bus is used to transmit data calculated by the corresponding column operation unit; Each operation unit is used to perform a point multiplication operation on a group of data pairs. The output end of the target operation unit is provided with a second data transmission module. The second data transmission module includes a transparent transmission channel and a pipeline temporary storage channel. The second data transmission module opens the transparent transmission channel or the pipeline temporary storage channel according to the instruction, and outputs the operation result to the corresponding operation result processing unit through the transparent transmission channel, and outputs the operation result to the addition input end of the next operation unit in the same column through the pipeline temporary storage channel. The target operation unit includes at least each operation unit except the last row.

18. The operation circuit according to claim 17, further comprising a plurality of operation result processing units; The operation result processing unit includes a transparent transmission channel and an addition channel. The operation result processing unit opens the transparent transmission channel or the addition channel according to the instruction, and transmits the operation results of the corresponding column operation units through the transparent transmission channel, and adds the operation results of the corresponding column operation units through the addition channel and outputs them.

19. A processor comprising the arithmetic circuit according to any one of claims 1 to 18.

20. An integrated circuit system comprising the arithmetic circuit according to any one of claims 1 to 18.

21. An electronic component comprising the processor according to claim 19 and / or the integrated circuit system according to claim 20.

22. An electronic device comprising the electronic component according to claim 21.

Images