Integrated circuit device, computing apparatus, system and method for matrix multiplication operation

Abstract

The present disclosure discloses an integrated circuit device, an electronic device, a board, and a method for performing matrix multiplication using the aforementioned integrated circuit device. The integrated circuit device may be included in a computing processing device of a combined processing device, and the computing processing device may include one or more integrated circuit devices. The aforementioned combined processing device may also include an interface device and other processing devices. The computing processing device interacts with other processing devices to jointly complete the calculation operation specified by the user. The combined processing device may also include a storage device, which is respectively connected to the device and other processing devices for storing data of the device and other processing devices. The scheme disclosed herein can reduce the amount of data transmission between the internal device and the external storage device, thereby minimizing the I / O bottleneck problem caused by bandwidth limitations, thereby improving the overall performance of the integrated circuit device.

Description

Technical Field

[0001] The present disclosure generally relates to the field of data processing. More specifically, the present disclosure relates to an integrated circuit device, a board, a computing device, a computing system and a method for matrix multiplication operations. Background Art

[0002] The field of artificial intelligence usually involves a large amount of data processing and calculations, including matrix multiplication operations of various types of data. Taking machine learning in the current field of artificial intelligence as an example, many of the computing tasks involve large-scale matrix multiplication operations, especially multiplication operations of large matrices. Taking deep learning in machine learning as an example, it contains a large number of types and types of matrix multiplication operations, including, for example, the matrix multiplication operations of the weight matrix and the input vector in the fully connected layer and the matrix multiplication operations of the input vector and the convolution kernel in the convolution layer. It can be imagined that the larger the amount of data and the data scale involved in the matrix multiplication operation, the higher the storage requirements for the computing platform (especially for the system on chip).

[0003] In existing matrix multiplication operations, processors such as central processing units ("CPUs") or image processing units ("GPUs") are usually used for operations. However, since the processor is subject to the capacity limitations of internal register resources, the huge amount of data operations may cause a large amount of data interaction between the processor and the external storage device. Since the bandwidth of the input / output ("I / O") bus between the processor and the external memory is limited, a serious I / O bottleneck problem is likely to occur, causing delays in data transfer and greatly reducing the computing efficiency during parallel operations. Furthermore, not only will the bandwidth limitation of the I / O bus become a bottleneck for system performance, but the large amount of I / O memory access between the processor and the external storage device will also have an adverse effect on computing and power consumption overhead. Summary of the invention

[0004] In order to at least solve the technical problems mentioned above, the present disclosure provides a hardware architecture and operation method that can efficiently perform matrix multiplication operations, thereby reducing the amount of data transmission with external storage devices, minimizing the solution to the I / O bottleneck problem caused by bus bandwidth limitations, and improving the operation efficiency of matrix multiplication. Specifically, the present disclosure provides the aforementioned solutions in the following aspects.

[0005] In a first aspect, the present disclosure discloses an integrated circuit device for matrix multiplication operation, comprising: an interface unit configured to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data comprises a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2; N 2 The N main computing units 2 The main computing units are connected in sequence to form a data transmission loop, wherein each main computing unit is configured to execute N 2 The main computing unit is configured to perform matrix block exchange with an adjacent main computing unit.

[0006] In executing a corresponding matrix multiplication task mentioned above, each of the main computing units is configured to: obtain a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and store them in the first storage area and the second storage area respectively; perform a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result; perform N-1 matrix block exchanges with the adjacent main computing unit through the control unit and using the first storage area and the second storage area, and perform a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and perform a summation operation on the N intermediate results to complete the matrix multiplication task associated with it.

[0007] In a second aspect, the present disclosure discloses a board comprising the integrated circuit device as described above and in a number of embodiments to be described later.

[0008] In a third aspect, the present disclosure discloses a computing device, comprising the board described above and in a number of embodiments to be described later.

[0009] In a fourth aspect, the present disclosure discloses a computing system, including the computing device described above and in a number of embodiments later.

[0010] In a fifth aspect, the present disclosure discloses a method for performing a matrix multiplication operation using an integrated circuit device described above and in a plurality of embodiments later, comprising: using an interface unit of the integrated circuit device to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data comprises a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2The second matrix block N 2 matrix multiplication tasks, wherein N is a positive integer greater than or equal to 2; and using each of the main computing units to perform the following operations: obtaining a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and storing them in a first storage area and a second storage area respectively; performing a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result; performing N-1 matrix block exchanges with adjacent main computing units through the control unit and using the first storage area and the second storage area, and performing a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and performing a summation operation on the N intermediate results to complete the matrix multiplication task associated with it.

[0011] In a sixth aspect, the present disclosure provides a computer program product comprising program instructions for performing matrix multiplication operations, which, when executed by one or more processors, implement the methods described above and later in multiple embodiments.

[0012] By utilizing the integrated circuit device, computing equipment, computing system, board and method disclosed above, the on-chip resources of the system on chip can be fully utilized to realize data sharing and transmission between the main computing units, thereby significantly reducing the I / O data interaction with the external memory, thereby realizing efficient parallel execution of data transmission and multiplication operations. Further, by combining the hardware architecture to split the matrix at multiple levels, the scheme disclosed herein simplifies the complexity of the matrix multiplication operation and supports the matrix multiplication operation of very large matrices. In addition, by significantly reducing the data interaction with the external memory, the scheme disclosed herein also improves the execution efficiency of the matrix multiplication operation, reduces the computing performance bottleneck problem caused by the on-chip and off-chip I / O bandwidth limitations, and thus can improve the overall performance of the integrated circuit device, computing equipment, computing system or board. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] By reading the detailed description below with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0014] Figure 1 is a schematic architectural diagram showing an integrated circuit device according to an embodiment of the present disclosure;

[0015] Figure 2 is a schematic diagram showing the structure of a single main computing unit according to an embodiment of the present disclosure;

[0016] Figure 3is an architectural diagram showing a "2*2" main computing unit according to an embodiment of the present disclosure;

[0017] Figure 4a and Figure 4b is a block diagram showing a "2*2" main computing unit for a convolution matrix multiplication operation according to an embodiment of the present disclosure;

[0018] Figure 5a and Figure 5b is a structural block diagram showing a "2*2" computing subunit for convolution matrix multiplication operation according to an embodiment of the present disclosure;

[0019] Figure 6 is a schematic diagram showing a pipeline operation performed by an integrated circuit device according to an embodiment of the present disclosure;

[0020] Figure 7 is a structural architecture diagram showing a "3*3" main computing unit according to an embodiment of the present disclosure;

[0021] Figure 8 is a board showing a matrix multiplication operation according to an embodiment of the present disclosure;

[0022] Fig. 9 is a diagram showing a computing system for matrix multiplication operations according to an embodiment of the present disclosure;

[0023] Fig.10 is a flow chart illustrating a method for performing a matrix multiplication operation according to an embodiment of the present disclosure;

[0024] Fig.11 is a structural diagram showing a combined processing device according to an embodiment of the present disclosure; and

[0025] Fig.12 It is a schematic diagram showing the structure of a board card according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the solutions disclosed in the present disclosure.

[0027] The specific implementation of the present disclosure is described in detail below with reference to the accompanying drawings.

[0028] Figure 11 is a schematic architecture diagram of an integrated circuit device 102 for matrix multiplication operation according to an embodiment of the present disclosure. In order to facilitate understanding of the scheme of the present disclosure, the figure also shows an external memory 104 for information interaction with the integrated circuit device 102. In one implementation scenario, the external memory may be a dynamic random access memory (Dynamic Random Access Memory, "DRAM"), and matrix data related to the matrix multiplication operation of the present disclosure may be stored in the dynamic random access memory. As can be understood by those skilled in the art, the matrix multiplication operation may involve a first matrix and a second matrix, and the first matrix and the second matrix may be divided into N 2 The first matrix block and N 2 second matrix blocks, where N is a positive integer greater than or equal to 2. For example, when N=2, the first matrix and the second matrix can be divided into 4 matrix blocks, for example, for a "4*4" first matrix or second matrix, it can be divided into 4 "2*2" first matrix blocks and second matrix blocks. For another example, when N=3, the first matrix and the second matrix can be divided into 9 matrix blocks, for example, for a "6*6" first matrix or second matrix, it can be divided into 9 "2*2" first matrix blocks and second matrix blocks. Through the aforementioned block processing, the scheme disclosed in the present invention can divide a large matrix multiplication operation into N 2 A matrix multiplication task is performed so as to be executed by the main computing unit of the present disclosure which will be described in detail below.

[0029] Further Figure 1 As shown in FIG. 1 , the integrated circuit device 102 provided by the present disclosure may include an interface unit 106 and an N 2 In one application scenario, a direct memory access (DMA) interface can be used as the aforementioned interface unit to send matrix data from an external memory to multiple main computing units 108, such as the five main computing units shown in the figure and one or more main computing units omitted in the middle with black dots. 2 The main computing units can form an "N*N" computing array to perform matrix multiplication operations in parallel. In one embodiment, the N main computing units disclosed herein can form an "N*N" computing array to perform matrix multiplication operations in parallel. 2 The main computing units are connected in sequence to form a data transmission loop, so that data including part of the row blocks and column blocks in the first matrix block or the second matrix block can be transmitted to other main computing units on the continuous loop, thereby executing the N mentioned above. 2 The corresponding one in the matrix multiplication task. Figure 2 The main computing unit of the present disclosure is described in detail.

[0030] like Figure 2As shown in FIG. , the main computing unit of the present disclosure may include M 2 computing subunits, that is, a computing array of "M*M", where M is a positive integer greater than or equal to 2. According to different implementation scenarios, M may be equal to or not equal to the aforementioned N, for example, N=2 and M=2, or N=2 and M=3. Further, the main computing unit may include multiple storage areas, such as the shared storage area shown in the figure and the private storage area associated with each computing subunit. In one embodiment, the shared storage area may be a storage area different from the private storage area. In another embodiment, the private storage area may be a storage space specially divided out in the shared storage area for temporary storage of computing subunits. In one implementation scenario, multiple storage areas in the main computing unit can be configured to store matrix blocks and intermediate results for performing matrix multiplication tasks.

[0031] In order to realize data interaction with adjacent main computing units forming a data transmission loop, the main computing unit disclosed in the present invention also includes a control unit, which is configured to exchange matrix blocks with adjacent main computing units. Thus, by means of an interface unit between the integrated circuit device and the external memory and the control units of each main computing unit, the scheme disclosed in the present invention enables multiple main computing units in the integrated circuit device to obtain a part of the matrix block data of each matrix multiplication task from the external memory, and obtain another part (or multiple parts) of the matrix block data from one or more adjacent main computing units through data interaction, so that the matrix block data required to complete the corresponding matrix multiplication task can be obtained and the corresponding matrix multiplication task can be completed based on this.

[0032] Specifically, in executing a corresponding one of the above-mentioned matrix multiplication tasks, each main computing unit can be configured to obtain a first matrix block (which comes from the first matrix) and a second matrix block (which comes from the second matrix) related to its matrix multiplication task through the interface unit, and store them in the first storage area and the second storage area respectively. Here, the first storage area and the second storage area can be two independent storage spaces allocated from the shared storage area to be used as a buffer for storing intermediate data.

[0033] After obtaining the above-mentioned first matrix block and second matrix block, the main computing unit of the present disclosure can perform a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result. As mentioned above, the matrix multiplication operation of the first matrix block and the second matrix block here can be performed by the M in the main computing unit. 2The main computing unit can execute the matrix blocks in parallel and pipeline by using the first storage area and the second storage area with the adjacent main computing unit through the control unit, and perform matrix multiplication operations on the first matrix block and the second matrix block exchanged each time, thereby obtaining N-1 intermediate results. For example, when N=2, that is, 4 main computing units are connected in series, then one main computing unit can obtain another first matrix block and a second matrix block from two adjacently connected main computing units, thereby obtaining another intermediate result. After obtaining N intermediate results, the main computing unit disclosed in the present invention can sum these intermediate results to complete a matrix multiplication task associated with it.

[0034] As mentioned above, the main computing unit of the present disclosure utilizes its M 2 Based on such an arrangement, the matrix multiplication operation disclosed in the present invention may involve a situation where the first matrix block and the second matrix block can be further divided. Specifically, the first matrix block and the second matrix block can be further divided into M 2 The first matrix sub-block and M 2 Based on this, a matrix multiplication task of the aforementioned main computing unit may include a matrix multiplication task based on M 2 The first matrix sub-block and M 2 The second matrix sub-block M 2 matrix multiplication tasks. Further, M 2 Each of the computing subunits can be configured to execute M 2 The corresponding one in the matrix multiplication tasks.

[0035] Specifically, in executing a corresponding matrix multiplication task, each computing subunit can be configured to perform M matrix multiplication operations to obtain M intermediate sub-results. In particular, the computing subunit can obtain a first matrix sub-block and a second matrix sub-block associated with its matrix multiplication task from a shared storage area (e.g., a first storage area and a second storage area). Then, the computing subunit can perform a matrix multiplication operation on a first matrix sub-block and a corresponding second matrix sub-block to obtain an intermediate sub-result. Finally, by performing a summation operation on the M intermediate sub-results, the matrix multiplication task associated with it is completed.

[0036] Based on the internal architecture and matrix partitioning of the integrated circuit device disclosed above, the solution disclosed in this disclosure also achieves highly parallel operation. 2 The main computing units can be configured to execute matrix multiplication tasks associated with each in parallel, and the M 2The computing subunits can be configured to execute the matrix multiplication tasks associated with each in parallel. In addition, with respect to the matrix partitioning disclosed in the present invention, the partitioning method can be partitioned according to the Cannon algorithm rule. For example, the first matrix and the second matrix involved in the matrix multiplication operation disclosed in the present invention can be partitioned into N matrix multiplication tasks at the main computing unit level according to the Cannon algorithm rule. 2 The first matrix block and N 2 Then, at the level of the computing subunit, a first matrix block and a second matrix block can be further divided according to the Cannon algorithm rules to obtain M 2 The first matrix sub-block and M 2 The first matrix sub-block.

[0037] Through the above combination Figure 1 and Figure 2 According to the description, those skilled in the art can understand that the present disclosure realizes parallel pipeline operation of matrix multiplication operation by performing multiple (or multiple rounds) block processing on matrix multiplication operation between large (or super large) matrices, and executing it through corresponding main computing units and computing sub-units. Therefore, the scheme disclosed in the present disclosure realizes the significant advantages of simplifying the complexity of matrix multiplication operation and accelerating matrix multiplication operation in terms of matrix multiplication operation. Further, by obtaining all matrix data from external memory and exchanging them with each other through control unit in main computing unit, frequent data interaction with external memory is avoided, breaking through the bottleneck of existing I / O interaction. Further, the number of main computing units and computing sub-units disclosed in the present disclosure can be flexibly set according to the computing scenario, and matrix multiplication operation of any scale can be realized in a cascade manner, so that the architecture layout is flexible and supports various matrix multiplication operation scenarios.

[0038] Figure 3 2 is a diagram showing an embodiment of the present disclosure 2 (i.e. 4) main computing units. Figure 3 As shown in FIG. 1 , the four main computing units (including main computing unit 0 to main computing unit 3) are interconnected through the control unit to form a "2*2" computing array. Figure 1 and Figure 2 As described above, the four main computing units may be configured to perform matrix multiplication operations between the four first matrix blocks and the four second matrix blocks, and each main computing unit may perform one matrix multiplication task among the four matrix multiplication tasks. Further, Figure 3 The M included in each main computing unit is also shown in 2 By assigning a matrix multiplication task to M 2 The computational sub-units can be used to perform parallel pipeline operations, thereby accelerating matrix multiplication operations and meeting the needs of various application scenarios.

[0039] In one application scenario, the integrated circuit device disclosed herein can be applied in the field of artificial intelligence, in particular, in machine learning including deep neural networks. For example, the integrated circuit device disclosed herein can perform convolution operations involved in neural networks on the received first matrix and second matrix, which involves a large number of matrix multiplication operations. In order to better understand how the integrated circuit device disclosed herein can be applied to such an application scenario, the following will be combined with Figure 4a and Figure 4b The integrated circuit device of the present disclosure is used to exemplarily describe the matrix multiplication operation involved in the convolution operation performed according to the Cannon algorithm.

[0040] Figure 4a A schematic diagram of an integrated circuit device structure according to an embodiment of the present disclosure is shown, which includes four (i.e., "2*2") interconnected main computing units, i.e., main computing unit 0 to main computing unit 3. In addition, for the sake of simplicity of the diagram, the figure does not show the multiple computing sub-units included in the main computing unit. Further, Figure 4b The two input matrices to be operated and the matrix blocks of the calculated results are schematically shown. Specifically, the two matrices to be performed matrix multiplication operation are a first matrix including the gradient of the convolution result and a second matrix including the convolution input. Further, the result matrix obtained after the two perform the matrix multiplication operation is the convolution weight gradient.

[0041] like Figure 4a As shown, four main computing units (each is Figure 1 The main computing units 102 in the figure are numbered in clockwise order as main computing unit 0, main computing unit 1, main computing unit 2 and main computing unit 3, and have been connected in sequence to form a closed circuit (or loop). Specifically, there is a two-way communication connection between adjacent main computing units 0 and 1, for example, two-way communication can be performed between the main computing units through DMA. Similarly, there are two two-way communication connections between adjacent main computing units 1 and 2, between main computing units 2 and 3, and between main computing units 3 and 0, so as to transfer matrix blocks to each other under the control of the control unit. In addition, each main computing unit can also be communicated with an external memory (shown in a dotted box in the figure) via an interface unit to obtain the matrix block data required to perform each computing task (in this example, the convolution result gradient and convolution input).

[0042] As known to those skilled in the art, the convolution weight gradient as the result of the matrix multiplication in this example can be used to update the gradient of the convolution result in the forward propagation during the back propagation of the neural network. In an operation scenario, the convolution weight gradient calculation is equivalent to the multiplication and accumulation calculation of the convolution result gradient of the first matrix in this example (when it is a four-dimensional matrix, its dimension can be expressed as NiHiWiCi as shown in the figure) and the convolution input of the second matrix in this example (when it is a four-dimensional matrix, its dimension is expressed as NoHoW℃o as shown in the figure). Here, N represents the number of samples, H represents the matrix height, W represents the matrix width, and C represents the number of channels. Further, according to the operation rules of matrix multiplication, the input matrix "convolution result gradient" can be expressed as Ci*NiHiWi, and the input matrix "convolution input" can be expressed as NoHoWo*Co. The two perform convolution weight gradient calculations (such as multiplication and addition operations) in the NiHiWi and NoHoWo directions, and the final output matrix "convolution weight gradient" can be expressed as Kh*Kw*Ci*Co (where Kh represents the height of the output matrix, Kw represents the width of the output matrix, Ci represents the number of channels of the input matrix "convolution result gradient", and Co represents the number of channels of the input matrix "convolution input"). For the purpose of simplicity, the figure only shows the convolution weight gradient calculation in the Ci*Co direction, that is, the matrix multiplication operation disclosed in the present invention.

[0043] Based on the above exemplary data placement rules (including, for example, the matrix partitioning method according to the Cannon algorithm) and the architecture of the four main computing units closed into a loop, the first matrix "convolution result gradient" and the second matrix "convolution input" stored in the external memory can be divided into four matrix blocks respectively. For the purpose of simplicity, the four matrix blocks divided by the first matrix "convolution result gradient" are represented as follows Figure 4b A00, A01, A10 and A11 are shown. Similarly, the four matrix blocks into which the second matrix "convolution input" is divided are represented as B00, B01, B10 and B11. Correspondingly, the output matrix "convolution weight gradient" as the result matrix can also be divided into four matrix blocks C00, C01, C10 and C11.

[0044] Based on the above data blocks, each main computing unit can respectively execute the following equations (1) to (4) to calculate the corresponding convolution weight gradients C00, C01, C11 and C10:

[0045] C00=A00*B00+A01*B10 (1)

[0046] C01=A00*B01+A01*B11 (2)

[0047] C11=A10*B01+A11*B11 (3)

[0048] C10=A10*B00+A11*B10 (4)

[0049] Specifically, the solution disclosed herein can Figure 4a The four main computing units 0, 1, 2 and 3 are respectively used to perform the computing tasks corresponding to the above equations (1) to (4) to obtain C00, C01, C11 and C10 respectively. In the scenario of performing the above matrix block multiplication using the Cannon algorithm, the following can be used according to the rules of the Cannon algorithm: Figure 4b The input matrix "convolution result gradient" shown in FIG. 1 is swapped with A10 and A11, and the input matrix "convolution input" is swapped with B01 and B11, as shown in FIG. Figure 4b Indicated by the arrow.

[0050] As described above, each main computing unit can receive its corresponding first matrix block and second matrix block from the external memory, and perform the corresponding matrix multiplication calculation. For example, the main computing unit 0 can receive a first matrix block "A00" and a second matrix block "B00" of the first matrix "convolution result gradient" and the second matrix "convolution input" from the external memory via the interface unit, and perform the first matrix multiplication subtask (A00*B00) as part of its matrix multiplication task according to formula (1), where "*" represents the matrix multiplication operation. Similarly, the main computing unit 1 receives its corresponding first matrix block and second matrix block, i.e. (A01 and B11), via the interface unit, and performs its first matrix multiplication task (A01*B11) according to formula (2). Similarly, the main computing units 2 and 3 receive their own first matrix block and second matrix block (A10 and B01) and (A11 and B10) respectively via the data interface, and execute their own first matrix multiplication tasks (A10*B01) and (A11*B10) respectively according to equations (3) and (4).

[0051] During the period when each main computing unit completes receiving the data of the matrix block from the external memory and performs the matrix multiplication task, it can also receive another first matrix block and another second matrix block from the interconnected main computing units. As mentioned above, each main computing unit of the present disclosure can use a bidirectional communication connection to send part of the matrix block data received from the external memory to the adjacent main computing unit respectively, as the corresponding matrix block data of another (or second) matrix multiplication task of the adjacent main computing unit.

[0052] As mentioned above, obtaining "C00" can be regarded as the matrix multiplication task of the main computing unit 0, and according to formula (1), the other first matrix block and the second matrix block required to complete the second matrix multiplication task in the "C00" matrix multiplication task are "A01" and "B10" respectively. Figure 4aIt can be seen that the main computing unit 1 adjacent to the main computing unit 0 can send the first matrix block "A01" previously received from the external memory to the main computing unit 0. Correspondingly, the main computing unit 3 adjacent to the main computing unit 0 can send the first matrix block B10 previously received from the external memory to the main computing unit 0. Thus, the main computing unit 0 can complete its second matrix multiplication task by performing a matrix multiplication operation on the received matrix block data "A01" and "B10". Similarly, the main computing units 1, 2 and 3 can also use a two-way communication connection to receive the matrix block data sent by the adjacent main computing units, that is, a corresponding first matrix block and a second matrix block, as shown in the figure ("A00" and "B01"), ("A11" and "B11") and ("A10" and "B00"). Next, each main computing unit can perform its own second matrix multiplication task according to equations (1) to (4), and obtain the matrix multiplication result associated with each main computing unit by summing the intermediate results of the first and second matrix multiplication tasks, that is, the convolution weight gradients C00, C01, C11 and C10 in this example, thereby completing their respective matrix multiplication tasks.

[0053] By combining the above Figure 4a and Figure 4b It can be seen from the description that each main computing unit disclosed in the present invention only needs to receive part of the matrix block data from the external memory, while the reception of the other part of the matrix block data makes better use of the high-speed communication bus between the main computing units. Therefore, the scheme disclosed in the present invention significantly reduces the data interaction between the main computing unit and the external memory, thereby significantly reducing the data transmission volume between on-chip and off-chip I / O and overcoming the I / O bottleneck caused by bandwidth limitations. It should be noted that Figure 4a The four main computing units shown in the figure forming a closed loop are merely exemplary and non-restrictive. Those skilled in the art may also pre-arrange other appropriate numbers of main computing units to form a processing array and a data transmission loop according to specific application scenarios, for example: Figure 7 As shown in (described in detail later).

[0054] As mentioned above, the matrix multiplication operation disclosed in the present invention can be performed by multiple computing sub-units in each main computing unit to perform specific matrix multiplication operations. Based on such a plurality of computing sub-unit settings, the first matrix block and the second matrix block disclosed in the present invention can be further divided into a plurality of first matrix sub-blocks and second matrix sub-blocks, and thus each matrix multiplication task (such as the above formula (1), (2), (3) or (4)) can be divided into a plurality of matrix multiplication sub-tasks corresponding to each of the plurality of computing sub-units. Based on this, based on the matrix multiplication sub-task associated therewith, each computing sub-unit can read a corresponding first matrix sub-block and a second matrix sub-block from the shared storage area to perform matrix operations. For a better understanding, the following will be combined with Figure 5a and Figure 5b Discuss how each computing subunit completes its corresponding matrix multiplication task according to the rules of the Gannon algorithm.

[0055] Figure 5a and Figure 5b is a block diagram showing a structure of a "2*2" computing subunit for convolution matrix multiplication operation according to an embodiment of the present disclosure. Figure 5a and Figure 5b To describe the convolution weight gradient calculation mentioned above, the main computing unit 0 performs its first matrix multiplication task "A00*B00" according to the Cannon algorithm.

[0056] like Figure 5a As shown, the main computing unit 0 includes a shared storage area and four computing subunits numbered 0, 1, 2 and 3 (each of which is Figure 2 During the matrix multiplication operation, each computing subunit may receive (or load) the matrix data of the respective first matrix subblock and the second matrix subblock from the shared storage area. Specifically, according to the respective associated matrix multiplication subtasks, Figure 5a Each computing subunit receives a respective first matrix subblock and a second matrix subblock from the shared storage area, and performs corresponding operations to obtain an intermediate subresult. Repeating the above steps, each computing subunit can obtain another intermediate subresult. Finally, by summing the above two intermediate subresults, an intermediate result for its matrix multiplication task is obtained.

[0057] like Figure 5bAs shown in , the first matrix block "convolution result gradient" A00 (for example, a four-dimensional matrix, expressed as Ci*NiHiWi) and the second matrix block "convolution input" B00 (for example, a four-dimensional matrix, expressed as NoHoWo*Co) stored in the shared memory area are used as two input data to execute the first matrix multiplication task "convolution weight gradient" (A00*B00) of the main computing unit 0 (only the Ci*Co direction is shown in the figure for the purpose of simplification). To this end, according to the Cannon algorithm, A00 can be divided into four first matrix sub-blocks a00, a01, a10 and a11, and B00 can be divided into four second matrix sub-blocks b00, b01, b10 and b11, and the eight matrix sub-blocks are stored in the shared memory area. Further, according to the Cannon algorithm, the result C00 of the output matrix (A00*B00) can also be divided into four sub-blocks c00, c01, c10 and c11. Based on this, according to the matrix multiplication operation rules in the Gannon algorithm, c00, c01, c11 and c10 can be obtained by the following equations (5) to (8):

[0058] c00=a00*b00+a01*b10(5)

[0059] c01=a00*b01+a01*b11 (6)

[0060] c11=a10*b01+a11*b11 (7)

[0061] c10=a10*b00+a11*b10 (8)

[0062] According to the scheme disclosed in this disclosure, Figure 5a The four computing subunits 0, 1, 2 and 3 shown in the figure respectively perform the calculations in the above equations (5) to (8), that is, respectively perform their own matrix multiplication tasks to obtain the corresponding c00, c01, c11 and c10. Taking the matrix multiplication task of obtaining c00 as an example, the matrix subblocks of the computing subunit 0 that performs the matrix multiplication task include a00, b00, a01 and b10. Similarly, for the matrix multiplication task of obtaining c11, the matrix subblocks of the computing subunit 2 that performs the subtask are a10, b01, a11 and b11.

[0063] Combined with Figure 2 Similar to the description, when using the Cannon algorithm for calculation, Figure 5bThe positions of a10 and a11 of the "convolution result gradient" A00 shown on the left are swapped, and the positions of b01 and b11 of the "convolution input" B00 are swapped. Thus, the first and second matrix sub-blocks of the calculation sub-unit 1 that performs the matrix multiplication task of obtaining c01 are a00, b01, a01, and b11, while the first and second matrix sub-blocks of the calculation sub-unit 3 that performs the matrix multiplication task of obtaining c10 are a10, b00, a11, and b10.

[0064] like Figure 5a As shown in the figure above, each of the four computing subunits can receive its own first and second matrix sub-blocks from the shared memory area. Taking computing subunit 0 as an example, it can load (a00 and b00) from the shared memory area to perform the matrix multiplication calculation of (a00*b00). Then, as Figure 5a As shown in the figure below, computing subunit 0 can then load the (a01 and b10) part from the shared memory area to perform the matrix multiplication calculation of (a01*b10). Finally, by adding the calculation results of (a00*b00) and (a01*b10), computing subunit 0 completes the matrix multiplication task associated with it. For computing subunits 1, 2, and 3, they also perform operations similar to computing subunit 0 to complete their respective matrix multiplication tasks.

[0065] Based on the above description, those skilled in the art will understand that the calculation result obtained by each matrix multiplication subtask in the first matrix multiplication task (such as A00*B00) of the main computing unit 0 is only an intermediate sub-result. Therefore, it is necessary to further complete multiple matrix multiplication subtasks corresponding to the second matrix multiplication task (such as A01*B10) to obtain another intermediate result, so that the two intermediate results can be summed to obtain the following result: Figure 5b The final calculation result of the matrix multiplication task C00 associated with the main computing unit 0 shown. Specifically, the computing subunit 0 can, for example, execute the matrix multiplication subtask corresponding to the first matrix multiplication task (A00*B00) according to formula (5), and use the obtained c00 as the first sub-c001. Then, the computing subunit 0 is used to execute the corresponding matrix multiplication subtask in the second matrix multiplication task (A01*B10) of C00 to obtain the second sub-c002. Finally. The two sub-c001 and c002 can be summed to obtain the matrix block c00 in the output matrix block C00. Considering that the right side of formula (5) includes a two-part addition operation and thus c002 is obtained by adding two intermediate results, c001 can also be sequentially added to the first intermediate result and the second intermediate result of c002 to obtain the matrix sub-block c00. The specific operation will be referred to later. Figure 6 The computational operations for the 6th and 7th time slices in FIG.

[0066] By performing operations similar to those of computing subunit 0, computing subunits 1, 2, and 3 can also obtain matrix subblocks c01, c11, and c10 in C00, respectively, such that Figure 5b The four matrix sub-blocks c00, c01, c11 and c10 shown on the right constitute the output matrix block C00 obtained by the main computing unit 0 performing the matrix multiplication task. Since the intermediate calculation results (such as c00, c01, c11 and c10) of each computing sub-unit can also be stored in the shared storage area of ​​the corresponding main computing unit without being stored in the external memory. Therefore, the scheme disclosed in the present invention can reduce the data exchange with the external memory, thereby reducing the I / O bottleneck caused by the external bandwidth limitation.

[0067] Further, according to the above description, those skilled in the art can understand Figure 5a The main computing unit shown in the figure includes four computing subunits, which is only exemplary and non-restrictive. According to different application scenarios, those skilled in the art can pre-set different numbers of computing subunits, or enable or disable different numbers of computing subunits based on the teachings of the present disclosure, so as to perform matrix multiplication calculations such as the Cannon algorithm.

[0068] Figure 6 is a schematic diagram showing the pipeline operation performed by an integrated circuit device (including a main computing unit and its computing subunits) according to an embodiment of the present disclosure. In particular, Figure 6 by Figure 5a and Figure 5b Taking the main computing unit 0 and its computing sub-unit 0 performing a convolution operation as an example, the data transmission and specific operations (including, for example, data loading and matrix multiplication operations) between the main computing unit 0, the computing sub-unit 0, the external memory and the shared storage area are shown in chronological order.

[0069] Specifically, Figure 6The main computing unit 0 and its computing subunit 0 perform corresponding data reception, transmission, loading or matrix multiplication operations in each time slice from the 1st time slice to the end of the 8th time slice in the form of rows, so as to finally obtain the pipeline operation of the matrix subblock c00 in the output matrix block C00 of the convolution weight gradient. Further, the four types of operations performed in each time slice are shown in the form of columns. As shown in the figure, the first column represents the operation of loading data from an external memory (e.g., via DDR), such as receiving the first matrix block and the second matrix block discussed in the present disclosure from it; the second column represents the data transfer between the main computing units, such as the shared storage area of ​​the main computing unit 0 sends its first and second matrix blocks to the adjacent main computing units 1 and 3, and receives their first and second matrix blocks from the main computing units 1 and 3 as the operation data for the main computing unit 0 to perform the second matrix multiplication task; the third column represents the data loading of the computing subunit 0; the fourth column represents the matrix multiplication operation performed in the computing subunit 0. According to the aforementioned time slice and operation division, the main computing unit 0 performs the corresponding operation in the corresponding time slice. For example, in the first time slice, the shared memory area of ​​the main computing unit 0 only performs the operation of storing B00 received from the external memory (i.e., "off-chip"). For another example, in the second time slice, the shared memory area of ​​the main computing unit performs the operation of receiving A00 from the external memory and the computing subunit 0 performs the operation of loading b00 in B00 from the shared memory area.

[0070] In order to efficiently utilize the on-chip I / O and computing resources, the on-chip operation disclosed herein can be a ping-pong pipeline operation. Specifically, according to the scheme disclosed herein, the on-chip storage resources can be divided into two parts: ping and pong. In one embodiment, when the ping storage resources are used to load data, the pong storage resources are used to perform matrix multiplication calculations; conversely, when the ping storage resources are used for matrix multiplication calculations, the pong storage resources are used to load data. Based on such resource allocation, the main computing unit disclosed herein can perform parallel ping-pong pipeline operations.

[0071] As can be seen from the figure, in the first time slice, the main computing unit 0 loads B00 from the external memory and stores it in the ping part of the shared memory area. In the second time slice, the main computing unit 0 loads A00 from the external memory and stores it in the ping part of the shared memory area. At the same time, b00 of B00 can be loaded to the computing subunit 0 in parallel. In the third time slice, a00 of A00 can be loaded to the computing subunit 0. In addition, during the third and fourth time slices, the main computing unit 0 sends A00 to the interconnected main computing unit 1 and sends B00 to the interconnected main computing unit 3 through the control unit. At the same time, A01 from the main computing unit 1 and B10 from the main computing unit 3 are received via the control unit.

[0072] In the data loading column of the 4th time slice, b10 of B00 and a01 of A00 can be loaded to the calculation subunit 0; at the same time, in the matrix multiplication operation column of the 4th time slice, a00*b00 of A00 and B00 are calculated to obtain an intermediate sub-result. In the data loading column of the 5th time slice, b00 of B10 and a00 of A01 can be loaded to the calculation subunit 0; at the same time, in the calculation operation column of the 5th time slice, a01*b10 of A00 and B00 are calculated to obtain an intermediate sub-result, and the intermediate sub-result is accumulated with the intermediate sub-result of the previous time slice to obtain the intermediate result at the 5th time slice. In the data loading column of the 6th time slice, b10 of B10 and a01 of A01 can be loaded to the calculation subunit 0; at the same time, in the matrix multiplication operation column of the 6th time slice, a00*b00 of A01 and B10 is calculated to obtain an intermediate sub-result, and the intermediate sub-result is accumulated with the intermediate sub-result of the previous time slice to obtain the intermediate result of the 6th time slice. In the matrix multiplication operation column of the 7th time slice, a01*b10 of A01 and B10 is calculated to obtain an intermediate sub-result, and the intermediate sub-result is accumulated with the intermediate sub-result of the previous time slice to obtain the matrix sub-block c00 of the output matrix block C00.

[0073] During the execution of data loading and calculation in the above-mentioned 3rd to 7th time slices, the pong part of the above-mentioned on-chip storage resources is used to receive the next group of B00 (B00') and A00 (A00') from the external memory for the main computing unit 0 to perform its first matrix multiplication task. Then, starting from the 8th time slice, the computing subunit 0 stores c00 of C00 calculated and output in the previous slice into the shared storage area. At the same time, the next group of b00 of B00' and a00 of A00' are loaded to the computing subunit 0 for calculation in the next time slice (not shown).

[0074] Similarly, the computing subunits 1, 2, and 3 of the main computing unit 0, as well as different main computing units and their computing subunits, also perform similar operations of the above 8 time slices to obtain the corresponding matrix blocks of their respective output matrices. Since the input matrices "convolution result gradient" and "convolution input" can be multi-dimensional structures, the calculation results in the three directions of NHW can be calculated and accumulated first. Then, the above calculations are performed cyclically on the Ci and Co dimensions of the two input matrices to obtain the calculation results of the output matrix "convolution weight gradient".

[0075] Figure 7 is a diagram showing the structure of the "3*3" main computing unit according to the embodiment of the present disclosure. Figure 7 As can be seen from the content shown, the "3*3" main computing unit can execute by forming a computing array and a data transmission loop. Figure 7The matrix multiplication operation shown in the upper part. Unlike the operation of the aforementioned "2*2" main computing unit, the "3*3" main computing unit requires 2 data transfers between adjacent main computing units, rather than one data transfer in the "2*2" main computing unit. In other words, for the scheme disclosed in this disclosure, the "N*N" main computing units require (N-1) data transfers or exchanges between adjacent main computing units. For ease of understanding, Figure 7 The lower part shows the first matrix block and the second matrix block data obtained by each main computing unit after the first and second rounds of data transfer. Taking the main computing unit 5 as an example, after obtaining its first matrix block "A23" and the second matrix block "B32" from the external memory, in the first round of data transfer, it receives another first matrix block "A21" from the main computing unit 6 and the second matrix block "B12" from the main computing unit 8 to perform its corresponding matrix multiplication task "A21*B12". Thereafter, in the second round of data transfer, it receives another first matrix block "A22" from the main computing unit 6 and the second matrix block "B22" from the main computing unit 8 to perform its corresponding matrix multiplication task "A22*B22". It can be seen that through Figure 7 According to the architecture and matrix partitioning shown in , the “3*3” main computing unit can support partitioning two large matrices into two “3*3” matrix blocks to perform matrix multiplication operations.

[0076] Figure 8 FIG. 8 is a diagram showing a board 800 for matrix multiplication operations according to an embodiment of the present disclosure. Figure 8 As shown, the board includes four Figure 1-Figure 7 It is understood that although four are shown here, those skilled in the art can arrange the interconnected P 2 An integrated circuit device, wherein P is a positive integer greater than or equal to 2. 2 The scheme disclosed herein can be divided into “P 2 *N 2 *M 2 "A matrix multiplication operation is performed on the first matrix and the second matrix of the matrix block.

[0077] Fig. 9 FIG. 9 is a diagram showing a computing system 900 for matrix multiplication operations according to an embodiment of the present disclosure. Fig. 9 As shown, the computing system 900 includes four servers or hosts, each of which is equipped with one or more Figure 8 Specifically, when two very large matrices are multiplied, they can be multiplied according to Fig. 9The computing system is divided into four matrix blocks. Then, each matrix block is further divided according to the number of boards on each host. And so on, until the super-large matrix involved in the matrix multiplication calculation is divided into the matrix multiplication operation granularity supported by the computing sub-unit disclosed in the present invention.

[0078] Fig.10 1 is a flowchart showing a method 1000 for performing a matrix multiplication operation according to an embodiment of the present disclosure. In combination with the above description, it can be understood that the method 1000 can be performed by the integrated circuit device of the present disclosure, so the description of the integrated circuit device is also applicable to the following description of the method 1000.

[0079] like Fig.10 As shown in FIG. 1 , at step 1002, method 1000 uses an interface unit of an integrated circuit device to obtain matrix data for the matrix multiplication operation from an external memory. In one embodiment, the matrix data here includes a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2. Next, for each main computing unit, method 1000 executes steps 1004 to 1010 to complete the matrix multiplication task of the main computing unit.

[0080] Specifically, at step 1004, the method 1000 obtains a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and stores them in the first storage area and the second storage area respectively. Then, at step 1006, the method 1000 performs a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result. Subsequently, at step 1008, the method 1000 performs N-1 matrix block exchanges with the adjacent main computing unit through the control unit and using the first storage area and the second storage area, and performs a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively. Finally, at step 1010, the method 1000 performs a summation operation on the N intermediate results to complete the matrix multiplication task associated with it.

[0081] For the purpose of simplicity, the above only combines Fig.10 The method of the present disclosure is described. A person skilled in the art may also conceive, based on the disclosure of the present disclosure, that the method 1000 of the present disclosure may include more steps, and the execution of these steps may achieve the above-mentioned combination Figure 1-Figure 9The various operations described in this disclosure will not be repeated here.

[0082] Fig.11 1 is a structural diagram showing a combined processing device 1100 according to an embodiment of the present disclosure. Fig.11 As shown in FIG. 1 , the combined processing device 1100 includes a computing processing device 1102, an interface device 1104, other processing devices 1106, and a storage device 1108. According to different application scenarios, the computing processing device may include one or more integrated circuit devices 1110, which may be configured to execute the combined processing device 1102 of the present invention. Figure 1-Figure 10 Described matrix multiplication operation.

[0083] In different embodiments, the computing and processing device of the present disclosure may be configured to perform user-specified operations. In an exemplary application, the computing and processing device may be implemented as a multi-core artificial intelligence processor. Similarly, one or more computing devices included in the computing and processing device may be implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core. When multiple computing devices are implemented as an artificial intelligence processor core or a partial hardware structure of an artificial intelligence processor core, the computing and processing device of the present disclosure may be regarded as having a single-core structure or a homogeneous multi-core structure.

[0084] In an exemplary operation, the computing processing device of the present disclosure can interact with other processing devices through an interface device to jointly complete the operation specified by the user. Depending on the implementation, other processing devices of the present disclosure may include one or more types of processors in general and / or special processors such as a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence processor, etc. These processors may include but are not limited to a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., and their number can be determined according to actual needs. As mentioned above, only with respect to the computing processing device of the present disclosure, it can be regarded as having a single-core structure or a homogeneous multi-core structure. However, when the computing processing device and other processing devices are considered together, the two can be regarded as forming a heterogeneous multi-core structure.

[0085] In one or more embodiments, the other processing device can serve as an interface between the computing device disclosed herein (which can be embodied as an artificial intelligence such as a computing device related to neural network computing) and external data and control, and perform basic controls including but not limited to data handling, starting and / or stopping the computing device, etc. In other embodiments, the other processing device can also cooperate with the computing device to jointly complete computing tasks.

[0086] In one or more embodiments, the interface device can be used to transmit data and control instructions between the computing and processing device and other processing devices. For example, the computing and processing device can obtain input data from other processing devices via the interface device and write it into a storage device (or memory) on the computing and processing device chip. Further, the computing and processing device can obtain control instructions from other processing devices via the interface device and write them into a control cache on the computing and processing device chip. Alternatively or optionally, the interface device can also read data from the storage device of the computing and processing device and transmit it to other processing devices.

[0087] Additionally or optionally, the combined processing device of the present disclosure may further include a storage device. As shown in the figure, the storage device is connected to the computing processing device and the other processing device, respectively. In one or more embodiments, the storage device may be used to store data of the computing processing device and / or the other processing device. For example, the data may be data that cannot be fully stored in the internal or on-chip storage device of the computing processing device or other processing device.

[0088] In some embodiments, the present disclosure also discloses a chip (e.g. Fig.12 In one implementation, the chip is a system on chip (SoC) and integrates one or more Fig.11 The chip can be connected to an external interface device (such as Fig.12 The external interface device 1206 shown in the figure is connected to other related components. The related components may be, for example, a camera, a display, a mouse, a keyboard, a network card or a wifi interface. In some application scenarios, other processing units (such as a video codec) and / or interface modules (such as a DRAM interface) may be integrated on the chip. In some embodiments, the present disclosure also discloses a chip packaging structure, which includes the above-mentioned chip. In some embodiments, the present disclosure also discloses a board card, which includes the above-mentioned chip packaging structure. The following will be combined with Fig.12 The board is described in detail.

[0089] Fig.12 is a schematic diagram showing the structure of a board 1200 according to an embodiment of the present disclosure, wherein Figure 8 The board shown can be regarded as a specific form of the board 1200. Fig.12 As shown in , the board includes a storage device 1204 for storing data, which includes one or more storage units 1210. The storage device can be connected and data can be transmitted with the control device 1208 and the chip 1202 described above by means of, for example, a bus. Further, the board also includes an external interface device 1206, which is configured for data relay or switching between the chip (or the chip in the chip packaging structure) and an external device 1212 (such as a server or computer, etc.). For example, the data to be processed can be transmitted to the chip by the external device through the external interface device. For another example, the calculation result of the chip can be transmitted back to the external device via the external interface device. According to different application scenarios, the external interface device can have different interface forms, for example, it can adopt a standard PCIE interface, etc.

[0090] In one or more embodiments, the control device in the disclosed board may be configured to regulate the state of the chip. To this end, in an application scenario, the control device may include a microcontroller unit (MCU) to regulate the working state of the chip.

[0091] According to the above combination Fig.11 and Fig.12 Based on the description, those skilled in the art can understand that the present disclosure also discloses an electronic device or apparatus, which may include one or more of the above-mentioned boards, one or more of the above-mentioned chips and / or one or more of the above-mentioned combined processing devices.

[0092] According to different application scenarios, the electronic equipment or device disclosed herein may include servers, cloud servers, server clusters, data processing devices, robots, computers, printers, scanners, tablet computers, smart terminals, PC devices, IoT terminals, mobile terminals, mobile phones, driving recorders, navigators, sensors, cameras, cameras, video cameras, projectors, watches, headphones, mobile storage, wearable devices, visual terminals, automatic driving terminals, transportation, household appliances, and / or medical equipment. The transportation includes airplanes, ships and / or vehicles; the household appliances include televisions, air conditioners, microwave ovens, refrigerators, rice cookers, humidifiers, washing machines, electric lights, gas stoves, and range hoods; the medical equipment includes magnetic resonance imaging, ultrasound machines and / or electrocardiographs. The electronic equipment or device disclosed herein may also be applied to the Internet, IoT, data centers, energy, transportation, public administration, manufacturing, education, power grids, telecommunications, finance, retail, construction sites, and medical fields. Further, the electronic equipment or device disclosed herein may also be used in cloud, edge, and terminal applications related to artificial intelligence, big data, and / or cloud computing. In one or more embodiments, electronic devices or devices with high computing power according to the disclosed solution can be applied to cloud devices (such as cloud servers), while electronic devices or devices with low power consumption can be applied to terminal devices and / or edge devices (such as smart phones or cameras). In one or more embodiments, the hardware information of the cloud device and the hardware information of the terminal device and / or edge device are compatible with each other, so that according to the hardware information of the terminal device and / or edge device, appropriate hardware resources can be matched from the hardware resources of the cloud device to simulate the hardware resources of the terminal device and / or edge device, so as to complete the unified management, scheduling and collaborative work of end-to-end or cloud-edge-to-end.

[0093] It should be noted that, for the purpose of simplicity, the present disclosure describes some methods and embodiments thereof as a series of actions and combinations thereof, but those skilled in the art will appreciate that the scheme of the present disclosure is not limited by the order of the actions described. Therefore, based on the disclosure or teaching of the present disclosure, those skilled in the art will appreciate that some of the steps therein may be performed in other orders or simultaneously. Further, those skilled in the art will appreciate that the embodiments described in the present disclosure may be regarded as optional embodiments, i.e., the actions or modules involved therein may not necessarily be necessary for the implementation of one or some of the schemes of the present disclosure. In addition, depending on the different schemes, the present disclosure also has different focuses on the description of some embodiments. In view of this, those skilled in the art may appreciate the parts that are not described in detail in a certain embodiment of the present disclosure, and may also refer to the relevant descriptions of other embodiments.

[0094] In terms of specific implementation, based on the disclosure and teaching of this disclosure, those skilled in the art can understand that several embodiments disclosed in this disclosure can also be implemented by other methods not disclosed herein. For example, with respect to the various units in the electronic device or device embodiments described above, this article divides them on the basis of considering logical functions, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features or functions in a unit or component can be selectively disabled. In terms of the connection relationship between different units or components, the connection discussed in the above text in conjunction with the accompanying drawings can be a direct or indirect coupling between units or components. In some scenarios, the aforementioned direct or indirect coupling involves a communication connection using an interface, wherein the communication interface can support electrical, optical, acoustic, magnetic or other forms of signal transmission.

[0095] In the present disclosure, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units. The aforementioned components or units may be located in the same location or distributed on multiple network units. In addition, according to actual needs, some or all of the units may be selected to achieve the purpose of the scheme described in the embodiments of the present disclosure. In addition, in some scenarios, multiple units in the embodiments of the present disclosure may be integrated into one unit or each unit may exist physically separately.

[0096] In some implementation scenarios, the above-mentioned integrated unit can be implemented in the form of a software program module. If implemented in the form of a software program module and sold or used as an independent product, the integrated unit can be stored in a computer-readable memory. Based on this, when the scheme of the present disclosure is embodied in the form of a software product (such as a computer-readable storage medium), the software product can be stored in a memory, which may include several instructions to enable a computer device (such as a personal computer, a server or a network device, etc.) to perform some or all of the steps of the method described in the embodiment of the present disclosure. The aforementioned memory may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0097] In some other implementation scenarios, the above-mentioned integrated unit can also be implemented in the form of hardware, that is, a specific hardware circuit, which may include digital circuits and / or analog circuits, etc. The physical implementation of the hardware structure of the circuit may include but is not limited to physical devices, and the physical devices may include but are not limited to devices such as transistors or memristors. In view of this, the various devices described herein (such as computing devices or other processing devices) can be implemented by appropriate hardware processors, such as CPU, GPU, FPGA, DSP and ASIC, etc. Further, the aforementioned storage unit or storage device can be any appropriate storage medium (including magnetic storage medium or magneto-optical storage medium, etc.), which can be, for example, a variable resistive memory (Resistive Random Access Memory, RRAM), a dynamic random access memory (Dynamic Random Access Memory, DRAM), a static random access memory (Static Random Access Memory, SRAM), an enhanced dynamic random access memory (Enhanced Dynamic Random Access Memory, EDRAM), a high bandwidth memory (High Bandwidth Memory, HBM), a hybrid memory cube (Hybrid Memory Cube, HMC), ROM and RAM, etc.

[0098] Based on the above full disclosure of this disclosure, those skilled in the art can understand that this disclosure also discloses the technical solutions described in the following clauses:

[0099] Clause A1. An integrated circuit device for matrix multiplication operation, comprising:

[0100] The interface unit is configured to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data includes a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2;

[0101] N 2 The N main computing units 2 The main computing units are connected in sequence to form a data transmission loop, wherein each main computing unit is configured to execute N 2 The corresponding one of the matrix multiplication tasks includes:

[0102] a plurality of storage areas configured to store matrix blocks and intermediate results for performing matrix multiplication tasks; and

[0103] a control unit configured to perform matrix block exchange with an adjacent main computing unit;

[0104] Wherein, in executing a corresponding matrix multiplication task, each of the main computing units is configured to:

[0105] Acquire a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and store them in a first storage area and a second storage area respectively;

[0106] Performing a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result;

[0107] Performing N-1 matrix block exchanges with adjacent main computing units by using the control unit and the first storage area and the second storage area, and performing a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and

[0108] A sum operation is performed on the N intermediate results to complete the matrix multiplication task associated therewith.

[0109] Clause A2. An integrated circuit device according to clause A1, wherein each of the main computing units comprises M 2 computing subunits, and the first matrix block and the second matrix block are respectively divided into M 2 The first matrix sub-block and M 2 A second matrix sub-block, and a matrix multiplication task includes a matrix multiplication task based on M 2 The first matrix sub-block and M 2 The second matrix sub-block M 2 matrix multiplication tasks, where the M 2 Each of the computing subunits is configured to execute M 2 A corresponding one of the matrix multiplication tasks, and in executing the corresponding one matrix multiplication task, the computing subunit is configured as:

[0110] Perform the following operation M times to obtain M intermediate sub-results:

[0111] Retrieve a first matrix sub-block and a second matrix sub-block associated with the matrix multiplication task from the first storage area and the second storage area respectively;

[0112] Performing a matrix multiplication operation on the first matrix sub-block and a corresponding second matrix sub-block to obtain an intermediate sub-result;

[0113] A summation operation is performed on the M intermediate sub-results to complete the matrix multiplication tasks associated therewith.

[0114] Clause A3. An integrated circuit device according to clause A2, wherein the first storage area and the second storage area are formed by the N 2 A shared memory area shared by multiple computing sub-units.

[0115] Clause A4. An integrated circuit device according to clause A2, wherein each of the plurality of storage areas of the main computing unit further comprises M 2 private sub-storage areas, and each private sub-storage area is associated with a corresponding computing sub-unit and is configured to store intermediate sub-results.

[0116] Clause A5. An integrated circuit device as described in clause A2, wherein the N 2 The main computing units are configured to execute matrix multiplication tasks associated with each in parallel, and the M 2 The computing subunits are configured to execute matrix multiplication tasks associated with each of them in parallel.

[0117] Clause A6. An integrated circuit device according to any one of clauses A1-A5, wherein the first matrix and the second matrix are divided according to a Cannon algorithm rule to obtain N 2 The first matrix block and N 2 A second matrix block.

[0118] Clause A7. An integrated circuit device according to any one of clauses A2-A5, wherein the first matrix block and the second matrix block are divided according to a Cannon algorithm rule to obtain M 2 The first matrix sub-block and M 2 A second matrix sub-block.

[0119] Clause A8. A board comprising one or more integrated circuit devices according to any one of clauses A1-A7.

[0120] Clause A9. A card according to clause A8, wherein when the card comprises P 2 When the integrated circuit devices are divided into P, the integrated circuit devices are connected in sequence to form a data transmission loop so as to 2 *N 2 *M 2 A matrix multiplication operation is performed on a first matrix and a second matrix of a matrix block, where P is a positive integer greater than or equal to 2.

[0121] Clause A10. A computing device comprising one or more boards according to clause A8.

[0122] Item A11. A computing system comprising a plurality of computing devices according to Item A10, wherein the plurality of computing devices are interconnected and operate in coordination to implement distributed matrix multiplication operations.

[0123] Clause A12. A method of performing a matrix multiplication operation using an integrated circuit device according to any one of clauses A1 to A7, comprising:

[0124] The interface unit of the integrated circuit device is used to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data includes a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2; and

[0125] Each of the master computing units is used to perform the following operations:

[0126] Acquire a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and store them in a first storage area and a second storage area respectively;

[0127] Performing a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result;

[0128] Performing N-1 matrix block exchanges with adjacent main computing units by using the control unit and the first storage area and the second storage area, and performing a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and

[0129] A sum operation is performed on the N intermediate results to complete the matrix multiplication task associated therewith.

[0130] Clause A13. The method of clause A12, wherein the computing subunit is further used to perform the following operations:

[0131] Perform the following operation M times to obtain M intermediate sub-results:

[0132] Retrieve a first matrix sub-block and a second matrix sub-block associated with the matrix multiplication task from the first storage area and the second storage area respectively;

[0133] Performing a matrix multiplication operation on the first matrix sub-block and a corresponding second matrix sub-block to obtain an intermediate sub-result;

[0134] A summation operation is performed on the M intermediate sub-results to complete the matrix multiplication tasks associated therewith.

[0135] Clause A14. A method according to clause A13, wherein the first storage area and the second storage area are stored by the N 2 A shared memory area shared by multiple computing sub-units.

[0136] Clause A15. A method according to clause A13, wherein each of the plurality of storage areas of the main computing unit further comprises M 2 private sub-storage areas, and each private sub-storage area is associated with a corresponding computing sub-unit and is configured to store intermediate sub-results.

[0137] Clause A16. A method according to clause A13, wherein the N 2 The main computing units are used to execute the matrix multiplication tasks associated with each in parallel, and the M 2 The computing subunits are used to execute the matrix multiplication tasks associated with each of them in parallel.

[0138] Clause A17. A method according to any one of clauses A12-A16, comprising using a Cannon algorithm rule to divide the first matrix and the second matrix to obtain N 2 The first matrix block and N 2 The second matrix block.

[0139] Clause A18. A method according to any one of clauses A13-A16, wherein the first matrix block and the second matrix block are divided according to a Cannon algorithm rule to obtain M 2 The first matrix sub-block and M 2 A second matrix sub-block.

[0140] Clause A19. A computer program product comprising program instructions for performing a matrix multiplication operation, which, when executed by one or more processors, enables implementation of the method according to any one of clauses A12-A18.

[0141] It should be understood that the terms "first", "second", "third", "fourth", etc. in the claims, specifications and drawings of the present disclosure are used to distinguish different objects rather than to describe a specific order. The terms "include" and "comprise" used in the specifications and claims of the present disclosure indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0142] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0143] As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0144] Although multiple embodiments of the present disclosure have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art may think of many changes, modifications, and alternatives without departing from the thought and spirit of the present disclosure. It should be understood that in the process of practicing the present disclosure, various alternatives to the embodiments of the present disclosure described herein may be adopted. The attached claims are intended to define the scope of protection of the present disclosure, and therefore cover equivalents or alternatives within the scope of these claims.

Claims

1. An integrated circuit device for matrix multiplication operation, comprising: The interface unit is configured to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data includes a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2; N 2 The N main computing units 2 The main computing units are connected in sequence to form a data transmission loop, wherein each main computing unit is configured to execute N 2 The corresponding one of the matrix multiplication tasks includes: a plurality of storage areas configured to store matrix blocks and intermediate results for performing matrix multiplication tasks; and a control unit configured to perform matrix block exchange with an adjacent main computing unit; Wherein, in executing a corresponding matrix multiplication task, each of the main computing units is configured to: Acquire a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and store them in a first storage area and a second storage area respectively; Performing a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result; Performing N-1 matrix block exchanges with adjacent main computing units by using the control unit and the first storage area and the second storage area, and performing a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and A sum operation is performed on the N intermediate results to complete the matrix multiplication task associated therewith.

2. The integrated circuit device according to claim 1, wherein each of the main computing units comprises an M 2 computing subunits, and the first matrix block and the second matrix block are respectively divided into M 2 The first matrix sub-block and M 2 A second matrix sub-block, and a matrix multiplication task includes a matrix multiplication task based on M 2 The first matrix sub-block and M 2 The second matrix sub-block M 2 matrix multiplication tasks, where the M 2 Each of the computing subunits is configured to execute M 2 A corresponding one of the matrix multiplication tasks, and in executing the corresponding one matrix multiplication task, the computing subunit is configured as: Perform the following operation M times to obtain M intermediate sub-results: Retrieve a first matrix sub-block and a second matrix sub-block associated with the matrix multiplication task from the first storage area and the second storage area respectively; Performing a matrix multiplication operation on the first matrix sub-block and a corresponding second matrix sub-block to obtain an intermediate sub-result; A summation operation is performed on the M intermediate sub-results to complete the matrix multiplication tasks associated therewith.

3. The integrated circuit device according to claim 2, wherein the first storage area and the second storage area are formed by the N 2 A shared memory area shared by multiple computing sub-units.

4. The integrated circuit device according to claim 2, wherein the plurality of storage areas of each of the main computing units further comprises M 2 private sub-storage areas, and each private sub-storage area is associated with a corresponding computing sub-unit and is configured to store intermediate sub-results.

5. The integrated circuit device according to claim 2, wherein the N 2 The main computing units are configured to execute matrix multiplication tasks associated with each in parallel, and the M 2 The computing subunits are configured to execute matrix multiplication tasks associated with each of them in parallel.

6. The integrated circuit device according to any one of claims 1 to 5, wherein the first matrix and the second matrix are divided according to a Cannon algorithm rule to obtain N 2 The first matrix block and N 2 The second matrix block.

7. The integrated circuit device according to any one of claims 2 to 5, wherein the first matrix block and the second matrix block are divided according to a Cannon algorithm rule to obtain M 2 The first matrix sub-block and M 2 A second matrix sub-block.

8. A board comprising one or more integrated circuit devices according to any one of claims 1-7.

9. The board according to claim 8, wherein when the board comprises a P 2 The integrated circuit device and each of the main computing units includes M 2 When the computing sub-units are divided into P, the integrated circuit devices are connected in sequence to form a data transmission loop so as to 2 *N 2 *M 2 A matrix multiplication operation is performed on a first matrix and a second matrix of a matrix block, where P is a positive integer greater than or equal to 2.

10. A computing device comprising one or more boards according to claim 8.

11. A computing system comprising a plurality of computing devices according to claim 10, wherein the plurality of computing devices are interconnected and operate in coordination to implement a distributed matrix multiplication operation.

12. A method for performing a matrix multiplication operation using an integrated circuit device according to any one of claims 1 to 7, comprising: The interface unit of the integrated circuit device is used to obtain matrix data for the matrix multiplication operation from an external memory, wherein the matrix data includes a first matrix and a second matrix, wherein the first matrix and the second matrix are respectively divided into N 2 The first matrix block and N 2 The matrix multiplication operation of the first matrix and the second matrix includes a matrix multiplication operation based on N 2 The first matrix block and N 2 The second matrix block N 2 matrix multiplication tasks, where N is a positive integer greater than or equal to 2; as well as Each of the master computing units is used to perform the following operations: Acquire a first matrix block and a second matrix block associated with its matrix multiplication task through the interface unit, and store them in a first storage area and a second storage area respectively; Performing a matrix multiplication operation on the first matrix block and the second matrix block to obtain an intermediate result; Performing N-1 matrix block exchanges with adjacent main computing units by using the control unit and the first storage area and the second storage area, and performing a matrix multiplication operation on the first matrix block and the second matrix block exchanged each time to obtain N-1 intermediate results respectively; and A sum operation is performed on the N intermediate results to complete the matrix multiplication task associated therewith.

13. The method according to claim 12, wherein the computing subunit is further used to perform the following operations: Perform the following operation M times to obtain M intermediate sub-results: Retrieve a first matrix sub-block and a second matrix sub-block associated with the matrix multiplication task from the first storage area and the second storage area respectively; Performing a matrix multiplication operation on the first matrix sub-block and a corresponding second matrix sub-block to obtain an intermediate sub-result; A summation operation is performed on the M intermediate sub-results to complete the matrix multiplication tasks associated therewith.

14. The method according to claim 13, wherein the first storage area and the second storage area are formed by the N 2 A shared memory area shared by multiple computing sub-units.

15. The method according to claim 13, wherein the plurality of storage areas of each of the main computing units further comprises M 2 private sub-storage areas, and each private sub-storage area is associated with a corresponding computing sub-unit and is configured to store intermediate sub-results.

16. The method according to claim 13, wherein when each of the main computing units comprises M 2 When there are N computing subunits, use the N 2 The main computing units are used to execute the matrix multiplication tasks associated with each in parallel, and the M 2 The computing subunits are used to execute the matrix multiplication tasks associated with each of them in parallel.

17. The method according to any one of claims 12 to 16, comprising using a Cannon algorithm rule to divide the first matrix and the second matrix to obtain N 2 The first matrix block and N 2 The second matrix block.

18. The method according to any one of claims 13 to 16, wherein the first matrix block and the second matrix block are divided according to a Cannon algorithm rule to obtain M 2 The first matrix sub-block and M 2 A second matrix sub-block.

19. A computer program product comprising program instructions for executing a matrix multiplication operation, which, when executed by one or more processors, enables implementation of the method according to any one of claims 12 to 18.

Images