Computing device, neural network processing equipment, chip and method for processing data

Abstract

A computing device, a neural network processing device, a chip and a method for processing data are provided. The computing device includes: a data memory for storing matrix data of a first matrix and a second matrix, the first matrix is ​​an m×s matrix, the second matrix is ​​an s×n matrix, the matrix data includes s groups of data, wherein each group of data includes a column vector in the first matrix and a row vector in the second matrix with the same index as the column vector; a computing module connected to the data memory for performing multiplication operations of the first matrix and the second matrix; a scheduling module connected to the data memory for controlling the data memory to input s groups of data to the computing module, and the computing module performs s vector outer product operations based on the s groups of data to obtain the product result of the first matrix and the second matrix. The embodiment of the present application proposes a matrix multiplication operation based on the vector outer product, which has a higher data reuse rate, and can therefore reduce the amount of data transmission and transmission power consumption.

Description

Technical Field

[0001] The present application relates to the field of data processing, and more specifically, to a computing device, a neural network processing device, a chip, and a method for processing data. Background Art

[0002] The essence of many data processing tasks is matrix multiplication. For example, the essence of convolution and full connection tasks is matrix multiplication.

[0003] The traditional matrix multiplication operation method uses an operation method based on vector inner product. For the same element in the matrix data, the operation method based on vector inner product requires a large amount of repeated scheduling of the data corresponding to the element, resulting in a large amount of data to be transmitted during the matrix multiplication operation and high transmission power consumption. Summary of the invention

[0004] The present application provides a computing device, a neural network processing equipment, a chip and a method for processing data to reduce the amount of data required to be transmitted and the transmission power consumption for matrix multiplication operations.

[0005] In a first aspect, a computing device is provided, comprising: a data storage device for storing matrix data of a first matrix and a second matrix, wherein the first matrix is ​​an m×s matrix, and the second matrix is ​​an s×n matrix, and the matrix data comprises s groups of data, wherein each group of data comprises a column vector in the first matrix and a row vector in the second matrix having the same index as the column vector, wherein s, m, and n are all positive integers greater than or equal to 1; a computing module, connected to the data storage device, and configured to perform multiplication operations of the first matrix and the second matrix; a scheduling module, connected to the data storage device, and configured to control the data storage device to input the s groups of data into the computing module, the computing module performing a vector outer product operation based on the column vector and the row vector in each group of data in the s groups of data to obtain s m×n intermediate result matrices corresponding to the s groups of data, and adding corresponding elements in the s m×n intermediate result matrices to obtain the product result of the first matrix and the second matrix.

[0006] In a second aspect, a neural network processing device is provided, comprising the computing device as described in the first aspect.

[0007] According to a third aspect, a chip is provided, comprising the computing device as described in the first aspect or the second aspect.

[0008] In a fourth aspect, a method for processing data is provided, which is applied to a computing device, wherein the computing device comprises: a data storage device, for storing matrix data of a first matrix and a second matrix, wherein the first matrix is ​​an m×s matrix, and the second matrix is ​​an s×n matrix, and the matrix data comprises s groups of data, wherein each group of data comprises a column vector in the first matrix and a row vector in the second matrix with the same index as the column vector, wherein s, m, and n are all positive integers greater than or equal to 1; a computing module, connected to the data storage device, for performing multiplication operations of the first matrix and the second matrix; the method comprises: controlling the data storage device to input the s groups of data into the computing module; performing a vector outer product operation based on the column vector and the row vector in each group of data in the s groups of data to obtain s m×n intermediate result matrices corresponding to the s groups of data; adding corresponding elements in the s m×n intermediate result matrices to obtain the product result of the first matrix and the second matrix.

[0009] The embodiment of the present application proposes a matrix multiplication operation based on a vector outer product. Compared with the traditional matrix multiplication operation based on a vector inner product, the matrix multiplication operation based on a vector outer product has a higher data reuse rate, and thus can reduce the data transmission amount and transmission power consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is an example diagram of matrix multiplication based on vector inner product.

[0011] Figure 2 It is a schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0012] Figure 3 This is an example diagram of the hardware architecture of the computing device provided in an embodiment of the present application.

[0013] Figure 4 This is an example diagram of matrix multiplication based on vector outer product provided in an embodiment of the present application.

[0014] Figure 5 Yes Figure 4 The method is applied to Figure 3 An example diagram of data mapping method when the hardware architecture is shown.

[0015] Figure 6 is an example diagram of a convolution operation.

[0016] Figure 7 The matrix multiplication based on the vector outer product provided in the embodiment of the present application is performed Figure 6 An example diagram of a convolution operation is shown.

[0017] Figure 8This is an example diagram of performing a fully connected operation using the matrix multiplication based on the vector outer product provided in an embodiment of the present application.

[0018] Fig. 9 It is a schematic diagram of the structure of the device provided in the embodiment of the present application.

[0019] Fig.10 It is a schematic flowchart of the method for processing data provided in an embodiment of the present application. DETAILED DESCRIPTION

[0020] To facilitate understanding, some relevant concepts involved in the embodiments of the present application are first introduced.

[0021] Vector inner product

[0022] The inner product of two vectors can also be called the dot product of two vectors, that is, the elements of the two vectors at corresponding positions are multiplied one by one and then summed. The result of the inner product of two vectors is a value (or a scalar). n ) and vector b = (b1, b2, ..., b n ), the inner product of vector a and vector b is calculated as: a1b1+a2b2+...+a n b n .

[0023] Vector outer product

[0024] The result of the outer product operation of two vectors is not a scalar, but a matrix (or a two-dimensional vector). m ) and vector θ = (θ1, θ2,..., θ n ) as an example, the outer product of vector μ and vector θ is calculated as follows:

[0025] Matrix multiplication

[0026] Matrix multiplication can be converted into the inner product of the row and column vectors of the matrix. Figure 1 , taking the first matrix 12 of m×s and the second matrix 14 of s×n as examples, the matrix multiplication operation is illustrated.

[0027] like Figure 1As shown, the first matrix 12 includes m row vectors, and the second matrix 14 includes n column vectors. The result matrix 16 obtained by multiplying the first matrix 12 and the second matrix 14 is an m×n matrix. The multiplication operation of the first matrix 12 and the second matrix 14 can be regarded as a vector inner product operation between the m row vectors of the first matrix and the n column vectors of the second matrix. Each vector inner product operation can obtain the value of an element 161 in the result matrix 16. In a similar manner, after m×n vector inner product operations, the values ​​of all elements in the result matrix 16 can be obtained.

[0028] Matrix Block

[0029] A matrix can be divided into multiple blocks. The multiplication operation between two matrices can be decomposed into the multiplication and addition operations between the blocks of the two matrices. Suppose the matrix matrix Then we can first decompose the matrix A into the following 4 block matrices:

[0030]

[0031]

[0032] Decompose the matrix B into the following 4 block matrices:

[0033]

[0034]

[0035] Next, the multiplication operation of matrix A and matrix B can be converted into a block matrix operation in the following form:

[0036]

[0037] After introducing the above basic concepts, the overall architecture of the computing device involved in the embodiment of the present application is described in detail below. The computing device mentioned in the present application can refer to any type of computing device based on the hardware architecture and capable of performing multiplication operations of two matrices.

[0038] For ease of understanding, first combine Figure 2 , the overall architecture of the computing device 20 mentioned in the embodiment of the present application is introduced.

[0039] like Figure 2 As shown, the computing device 20 may include a data storage 22 , a computing module 24 and a scheduling module 26 .

[0040] The data memory 22 can be used to store matrix data of the first matrix and the second matrix. For ease of description, the following is an example in which the first matrix is ​​an m×s matrix and the second matrix is ​​an s×n matrix. In the first matrix, m represents the row index of the first matrix, and s represents the column index of the first matrix. m and s are generally positive integers greater than 1. However, the embodiments of the present application are not limited to this. In some embodiments, one of m and s may also be equal to 1, in which case the first matrix may be understood as a vector. In the second matrix, s represents the row index of the second matrix, and n represents the column index of the second matrix. s and n are generally positive integers greater than 1. However, the embodiments of the present application are not limited to this. In some embodiments, one of s and n may also be equal to 1, in which case the second matrix may be understood as a vector.

[0041] The present application embodiment does not specifically limit the data content of the matrix data of the first matrix and the second matrix, which is related to the computing task performed by the computing device 20. For example, the computing device 20 performs a convolution operation task (the convolution operation is essentially a matrix multiplication, which will be discussed later). Figure 6 For an example of convolution operation, please refer to Figure 6 ), the data in the first matrix may be the data of the input feature map, and the data in the second matrix may be the weight data in the convolution kernel.

[0042] The data memory 22 may include, for example, a high-speed random access memory (RAM). The data memory 22 may be, for example, a data cache of a neural-network processing unit (NPU).

[0043] The computing module 24 may also be referred to as a processing element (PE) or an arithmetic circuit. The computing module 24 may be connected to the data memory 22. The computing module 24 may be responsible for completing some or all of the computing tasks on the computing device 20. For example, the computing module 24 may be used to perform a multiplication operation of a first matrix and a second matrix.

[0044] There are many ways to implement the calculation module 24. Since matrix multiplication operation is essentially a multiplication-accumulation operation, in some embodiments, the calculation module 24 may be a calculation module based on a multiplication-accumulator.

[0045] Figure 3 A possible implementation of the calculation module 24 is given. Figure 3As shown, the basic unit in the calculation module 24 is a multiplier accumulator 241. The multiplier accumulator 241 may include a multiplier 2411 and an adder 2412. The inputs of the multiplier 2411 are an element of the first matrix and an element of the second matrix, respectively. Taking the convolution operation as an example, the data in the first matrix is ​​the data in the input feature map, and the data in the second matrix is ​​the weight data in the convolution kernel. In this case, the input of the multiplier 2411 can be represented by F and W. Among them, F represents the feature data in the input feature map; W represents the weight data in the convolution kernel.

[0046] The computing module 24 may include m×n multiplication-accumulation trees 242 ( Figure 3 A column of multiplication and accumulation devices in the multiplication and accumulation tree 242 corresponds to a multiplication and accumulation tree 242. The multiplication and accumulation tree 242 may include s multiplication and accumulation devices 241. The s multiplication and accumulation devices 241 are connected end to end through data lines 2413 to form a data operation pipeline. The pipeline can accumulate the calculation results of the s multiplication and accumulation devices in the multiplication and accumulation tree 242 step by step.

[0047] by Figure 3 Take the multiplication-accumulation tree shown by the reference numeral 242 in the figure as an example, the multiplication-accumulation tree 241a in the multiplication-accumulation tree 242 represents the first-level multiplication-accumulation of the multiplication-accumulation tree 242, and the multiplication-accumulation 241s represents the last-level multiplication-accumulation of the multiplication-accumulation tree 242. During the operation process, the multiplication-accumulation tree 242 can use the s multiplication-accumulations inside it to perform s multiplication operations, and then the multiplication-accumulation tree 242 can start from the first-level multiplication-accumulation 241a, and pass the s multiplication results step by step downward to the last-level multiplication-accumulation 241s, and accumulate the multiplication results of the multiplication-accumulations of each level during the transfer process. After the above process, the output of the last-level multiplication-accumulation 241s is the cumulative sum of the s multiplication operations performed by the s multiplication-accumulations in the multiplication-accumulation tree 242.

[0048] See again Figure 2 The scheduling module 26 can be connected to the data memory 22. The scheduling module 26 can be, for example, a controller with a data scheduling function. The scheduling module 26 can control the data memory 22 to input the matrix data of the first matrix and the second matrix to be multiplied to the calculation module 24, so as to use the calculation module 24 to complete the matrix operation of the first matrix and the second matrix.

[0049] Affected by the hardware structure, the line width of the data memory 22 is usually limited. Therefore, the scheduling module 26 usually schedules the matrix data according to a certain data scheduling strategy. As a possible data scheduling method, the scheduling module 26 can be based on Figure 1In the matrix multiplication operation method shown in FIG. 1 , a row vector of the first matrix and a column vector of the second matrix are input to the calculation module 24. After obtaining the row vector of the first matrix and the column vector of the second matrix, the calculation module 24 performs a multiplication-accumulation operation in the manner of vector inner product to obtain the value of an element in the result matrix. For the specific calculation method, see Figure 1 Related description. Figure 3 Taking the calculation module 24 shown in FIG. 2 as an example, the scheduling module 26 can read the row vector (including s elements) of the first matrix and the column vector (including s elements) of the second matrix into Figure 3 The multiplication and accumulation tree 242 is then used to accumulate the s multiplications in the s multiplication and accumulation results to obtain the value of an element in the result matrix.

[0050] In the embodiment of the present application, the above data scheduling and calculation method are referred to as matrix multiplication based on vector inner product. Using matrix multiplication based on vector inner product, each time a vector inner product operation is performed, the calculation module 24 can obtain the value of an element in the result matrix, and perform m×n vector inner product operations to obtain the values ​​of all m×n elements in the result matrix. However, in the process of calculating the values ​​of the m×n elements, each time the value of one of the elements is calculated, it is necessary to input the data corresponding to 2s elements (a row of the first matrix and a column of the second matrix, a total of 2s elements) to the calculation module 26. Therefore, in order to obtain the values ​​of the m×n elements in the result matrix, a total of 2s×m×n elements corresponding to the data amount are required. When performing matrix multiplication based on vector inner product, the row vector of the first matrix and the column vector of the second matrix will be repeatedly scheduled many times. For example, since each row vector of the first matrix needs to be multiplied by the n column vectors of the second matrix, each row vector of the first matrix will be repeatedly scheduled n times; similarly, since each column vector of the second matrix needs to be multiplied by the m row vectors of the first matrix, each column vector of the second matrix will be repeatedly scheduled m times. Such large-scale data repetition scheduling is bound to increase the amount of data to be transmitted and the transmission power consumption.

[0051] In response to the above problems, an embodiment of the present application provides a matrix multiplication based on vector outer product, which can reduce the amount of data that needs to be transmitted and the transmission power consumption during the matrix multiplication operation.

[0052] The following is a detailed description of matrix multiplication based on vector outer product.

[0053] The matrix data of the first matrix and the second matrix may include s groups of data. Each group of data in the s groups of data may include a column vector in the first matrix and a row vector in the second matrix with the same index as the column vector. For example, the i-th group of data in the s groups of data may include the i-th column vector of the first matrix and the i-th row vector of the second matrix.

[0054] The scheduling module 26 may control the data storage 22 to input s groups of data to the calculation module 24. For example, the scheduling module 26 may sequentially input s groups of data to the calculation module 24. Alternatively, the scheduling module 26 may also input multiple groups of data from the s groups of data at one time.

[0055] like Figure 4 As shown, the calculation module 24 can perform a vector outer product operation based on the column vector and the row vector in each of the s groups of data to obtain s m×n intermediate result matrices ( Figure 4 The matrix 162 in the figure represents an m×n intermediate result matrix), and the corresponding elements in the s m×n intermediate result matrices are added to obtain the product result of the first matrix and the second matrix. Exemplarily, the calculation module 24 can start from the 0th column of the first matrix 12 and the 0th row of the second matrix 14, and perform a vector outer product based on the column vector 0 of the first matrix 12 and the row vector 0 of the second matrix. After the calculation is completed, it can be calculated along Figure 4 In the direction of the arrow in the first matrix 12, take the column vector 1 of the first matrix 12 and follow Figure 4 The row vector 1 of the second matrix 14 is taken in the direction of the arrow in the second matrix 14, and then a vector outer product is performed again based on the column vector 1 of the first matrix 12 and the row vector 1 of the second matrix. Similarly, after completing s vector outer product operations, the calculation module 24 can obtain the product result of the first matrix and the second matrix.

[0056] In the embodiment of the present application, the scheduling and calculation method of the above-mentioned matrix data is referred to as matrix multiplication based on vector outer product. Matrix multiplication based on vector outer product requires a column vector of the first matrix and a row vector of the second matrix to perform a vector outer product operation, with a total of (m+n) elements. In order to obtain the product result of the first matrix and the second matrix, the matrix multiplication based on vector outer product needs to perform the above-mentioned vector outer product operation s times, and a total of s×(m+n) elements corresponding to the amount of data are required. As mentioned above, in order to obtain the product result of the first matrix and the second matrix based on the matrix multiplication of the vector inner product, a total of 2s×m×n elements corresponding to the amount of data need to be input. Compared with the two, the matrix multiplication based on vector outer product has a higher data reuse rate, so it can greatly reduce the amount of data that needs to be transmitted to the computing module through the data storage device, thereby reducing the data transmission power. When the computing module is implemented in the form of a chip, the reduction in the amount of data that the data storage device needs to transmit means that the line width of the input data can be reduced, which helps to optimize the area of ​​the chip.

[0057] Combined with the previous article Figure 3 , a possible implementation of the calculation module 24 is given. Figure 3 The hardware architecture shown is applicable to both matrix multiplication based on vector inner product and matrix multiplication based on vector outer product. In other words, matrix multiplication based on vector outer product and matrix multiplication based on vector inner product can share the same hardware architecture.

[0058] Combine the following Figure 3 and Figure 5 , for matrix multiplication based on vector inner product Figure 3 The implementation on the hardware architecture shown is illustrated in more detail.

[0059] Combined with the previous article Figure 3 As mentioned, the computing module 24 includes m×n multiplication-accumulation trees, each of which includes s multiplication-accumulation devices connected in sequence from beginning to end (corresponding to Figure 3 or Figure 5 The multiplication accumulators of the m×n multiplication-accumulation trees can be divided into s groups of multiplication accumulators, each group includes m×n multiplication accumulators, and the m×n multiplication accumulators belong to the m×n multiplication-accumulation trees respectively. That is, any two multiplication accumulators in each group of the s groups of multiplication accumulators come from different multiplication accumulator trees in the m×n multiplication-accumulation trees.

[0060] by Figure 5 For example, you can Figure 5 Each row of multiplication and accumulation devices 243 in is grouped into one group. In this way, Figure 5The s rows of multiplication accumulators in the above-mentioned s groups of multiplication accumulators can be used to form the above-mentioned s groups of multiplication accumulators. In the actual calculation process, the calculation module 24 can perform the above-mentioned s times of vector outer product operations based on the s groups of multiplication accumulators, so as to obtain s m×n intermediate result matrices. For example, in Figure 5 In the example of , the calculation module 24 can control the i-th row multiplication accumulator in the s-row multiplication accumulator to perform the vector outer product operation of the i-th column vector of the first matrix and the i-th row vector of the second matrix, thereby obtaining s m×n intermediate result matrices. After the calculation is completed, the s m×n intermediate result matrices are respectively stored in Figure 5 In the s×m×n multiplier-accumulators shown.

[0061] Next, the computing module 24 can use the data processing pipeline provided by the m×n multiplication-accumulation trees to add the corresponding elements in the s m×n intermediate result matrices, thereby obtaining the product result of the first matrix and the second matrix. For example, the computing module 24 can start from the first-level multiplication-accumulator of each multiplication-accumulation tree, accumulate the multiplication results in each multiplication-accumulation tree, and output a final multiplication-accumulation result in the last-level multiplication-accumulation of each multiplication-accumulation tree. The m×n final results output by the m×n multiplication-accumulation trees are the same as the m×n final results obtained by the matrix multiplication based on the vector inner product.

[0062] In some embodiments, the size of the two matrices to be multiplied (for ease of description, the two matrices are referred to as the third matrix and the fourth matrix below) may be relatively large. Limited by the hardware processing power, the computing device 20 cannot perform the multiplication operation of the two matrices at one time. In the face of this situation, the third matrix and / or the fourth matrix can be processed in blocks according to the matrix block method described above. For example, the third matrix can be divided into A blocks (the size of the A blocks is less than or equal to m×s), and the fourth matrix can be divided into B blocks (the size of the B blocks is less than or equal to s×n), so that the computing device 20 can support the matrix multiplication operation between the A blocks and the B blocks. Then, the computing device 20 can multiply the A blocks and the B blocks in pairs according to the matrix multiplication based on the vector outer product described above, and obtain the product result of the third matrix and the fourth matrix based on the result of the multiplication of the A blocks and the B blocks in pairs.

[0063] The computing device provided in the embodiment of the present application can be applied to various scenarios or computing tasks that require matrix multiplication operations. For example, neural networks are currently widely used in all walks of life. The essence of many operators in neural networks is matrix multiplication. For example, the convolution operations and fully connected operations often used in neural networks are essentially matrix multiplication. Therefore, in some embodiments, the computing device 20 provided in the embodiment of the present application can be used to perform neural network operations, such as for performing convolution operations or fully connected operations. Taking the computing device 20 provided in the embodiment of the present application performing a convolution operation as an example, the computing device 20 may also be referred to as a convolution device, a convolution processor, a convolution accelerator, a convolution acceleration engine, and the like.

[0064] The following is an example of the operation method of the computing device provided in the embodiment of the present application in the convolution operation and full connection operation scenarios.

[0065] Convolution operations are widely used in neural network calculations. The essence of convolution operations is the multiplication of two matrices. Therefore, like the multiplication of two matrices, the core operation of convolution operations is the multiplication-accumulation operation.

[0066] Figure 6 Taking a 1×1 convolution kernel as an example, an example of convolution operation is given. Figure 6 The upper left corner shows the input feature map. M represents the width of the input feature map, and S represents the number of channels of the input feature map (channels may be, for example, R, G, B). Figure 2 The lower left corner of is the convolution kernel, and the data in the convolution kernel can be called weight data. N represents the number of convolution kernels. The number of channels of the convolution kernel is equal to the number of channels of the input feature map, both of which are S. Figure 2 The output feature map is shown on the right. For the convolution operation, the number of channels of the output feature map is equal to the number of convolution kernels.

[0067] The amount of computation required for convolution operations is usually large. Therefore, when using the computing device 20 mentioned above to perform convolution operations, the hardware processing capability of the computing device needs to be considered. For example, Figure 6 Where s, m, and n are the number of input channels, feature image width, and number of convolution kernels that the computing device 20 can process at one time. Therefore, in some embodiments, the input feature map and the weight data in the convolution kernel can be arranged in the form of two matrices to form the following: Figure 7The third matrix and the fourth matrix of the matrix multiplication to be performed are shown. The size of the third matrix is ​​M×S, and the size of the fourth matrix is ​​S×N. Then, the third matrix and / or the fourth matrix can be processed by blocks according to the matrix block method described above. For example, the third matrix can be divided into A blocks (the size of the A blocks is less than or equal to m×s), and the fourth matrix can be divided into B blocks (the size of the B blocks is less than or equal to s×n), so that the computing device 20 can support the matrix multiplication operation between the A blocks and the B blocks. Then, the computing device 20 can multiply the A blocks and the B blocks in pairs according to the scheduling method based on the vector outer product described above, and obtain the product result of the third matrix and the fourth matrix based on the result of the two-by-two multiplication of the A blocks and the B blocks (that is, the corresponding elements in the result of the two-by-two multiplication of the A blocks and the B blocks are accumulated to obtain the product result of the third matrix and the fourth matrix). The product of the third matrix and the fourth matrix can be used as the data in the output feature map.

[0068] It should be noted that Figure 6 Although the convolution operation provided in the embodiment of the present application is illustrated by taking a 1×1 convolution kernel as an example, the embodiment of the present application is not limited to this. For convolution kernels of other sizes, such as 3×3, 5×5, 7×7, etc., their core is also matrix multiplication operation, which can also be converted into matrix operation based on vector outer product to reduce data transmission amount and transmission power consumption.

[0069] Since convolution operations involve a large number of matrix operations, applying the computing device provided in the embodiment of the present application to convolution operations can greatly reduce the amount of data that needs to be transmitted during the convolution operation, thereby reducing the power of transmitting data and the line width of the data storage device.

[0070] The fully connected operation is a special case of matrix multiplication. Figure 8 , taking the matrix multiplication of the m×s first matrix and the s×n second matrix mentioned above as an example, if the value of m is set to 1, the multiplication of the first matrix and the second matrix can be regarded as a fully connected operation.

[0071] Execution using matrix multiplication based on vector outer product Figure 8 The process of the full connection operation is similar to Figure 4 The process shown is basically the same, except that, for the first matrix, the column vector on which the vector outer product operation is performed contains only one element, and the s intermediate result matrices obtained are s vectors of size n.

[0072] It should be noted that the computing device 20 mentioned in the embodiment of the present application may include various other types of components in addition to the data storage device 22, computing module 24 and scheduling module 26 mentioned above. For example, the computing device 20 may also include one or more of the following components: a computing module (or computing circuit, such as an activation circuit for performing an activation operation, a pooling circuit for performing a pooling operation, etc.) capable of performing other operations, a register, an internal memory, etc.

[0073] See also Fig. 9 , the embodiment of the present application further provides a device 90. The device 90 may include the computing device 20 described above. In some embodiments, the device 90 may be a neural network processing device and / or chip. In other embodiments, the device 90 may be, for example, a mobile terminal (such as a mobile phone), a computer, a server, etc.

[0074] Combination of the above Figures 1 to 9 , describes in detail the device embodiment of the present application, and the following is combined with Fig.10 , the method embodiment of the present application is described in detail. It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, so the parts not described in detail can refer to the previous device embodiment.

[0075] Fig.10 It is a schematic flowchart of the method for processing data provided in an embodiment of the present application. Fig.10 The method can be applied to a computing device. The computing device can be the computing device 20 mentioned above. Fig.10 The method includes steps S1010 to S1030.

[0076] In step S1010, the control data memory inputs s groups of data to the calculation module.

[0077] In step S1020, a vector outer product operation is performed based on the column vector and the row vector in each of the s groups of data to obtain s m×n intermediate result matrices corresponding to the s groups of data.

[0078] In step S1030, corresponding elements in s m×n intermediate result matrices are added to obtain a product result of the first matrix and the second matrix.

[0079] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0080] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0081] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0082] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0083] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0084] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A computing device, characterized in that: include: A data memory, used to store matrix data of a first matrix and a second matrix, wherein the first matrix is ​​an m×s matrix, the second matrix is ​​an s×n matrix, the matrix data includes s groups of data, wherein each group of data includes a column vector in the first matrix and a row vector in the second matrix having the same index as the column vector, wherein s, m, and n are all positive integers greater than or equal to 1; a computing module, connected to the data memory, and configured to perform a multiplication operation of the first matrix and the second matrix; a scheduling module connected to the data memory, and used to control the data memory to input the s groups of data to the calculation module, the calculation module performing a vector outer product operation based on the column vector and the row vector in each group of data in the s groups of data to obtain s m×n intermediate result matrices corresponding to the s groups of data one by one, and adding corresponding elements in the s m×n intermediate result matrices to obtain a product result of the first matrix and the second matrix; The computing module includes m×n multiplication-accumulation trees, the multiplication accumulators in the m×n multiplication-accumulation trees form s groups of multiplication accumulators, each group of the s groups of multiplication accumulators includes m×n multiplication accumulators, any two multiplication accumulators in the m×n multiplication accumulators belong to different multiplication-accumulation trees in the m×n multiplication-accumulation trees, each multiplication-accumulation tree includes s multiplication accumulators, and the s multiplication accumulators are connected end to end via data lines; The calculation module performs the s vector outer product operations based on the s groups of multiplication and accumulation units respectively to obtain the s m×n intermediate result matrices, and uses the m×n multiplication and accumulation trees to add corresponding elements in the s m×n intermediate result matrices to obtain the product result of the first matrix and the second matrix.

2. The computing device according to claim 1, wherein: The first matrix belongs to one of the A blocks of the third matrix, the second matrix belongs to one of the B blocks of the fourth matrix, the sizes of the A blocks are less than or equal to m×s, and the sizes of the B blocks are less than or equal to s×n. The calculation module is also used to multiply the A blocks and the B blocks in pairs, and obtain the product result of the third matrix and the fourth matrix based on the results of the multiplication of the A blocks and the B blocks in pairs, wherein A and B are both positive integers greater than 1.

3. The computing device according to claim 2, characterized in that The computing device is used to perform a convolution operation, the elements in the third matrix are data in the input feature map, and the elements in the fourth matrix are weight data in the convolution kernel, wherein the third matrix is ​​an M×S matrix, and the fourth matrix is ​​an S×N matrix, M represents the width of the input feature map, S represents the number of channels of the input feature map, and N represents the number of convolution kernels, and M, S, and N are all positive integers greater than or equal to 1.

4. A neural network processing device, characterized in that: The method comprises a computing device as claimed in any one of claims 1 to 3.

5. A chip, characterized in that: The method comprises a computing device as claimed in any one of claims 1 to 3.

6. A method for processing data, characterized in that The method is applied to a computing device, the computing device comprising: A data memory, used to store matrix data of a first matrix and a second matrix, wherein the first matrix is ​​an m×s matrix, the second matrix is ​​an s×n matrix, the matrix data includes s groups of data, wherein each group of data includes a column vector in the first matrix and a row vector in the second matrix having the same index as the column vector, wherein s, m, and n are all positive integers greater than or equal to 1; a computing module connected to the data memory and configured to perform a multiplication operation of the first matrix and the second matrix; the computing module comprises m×n multiplication-accumulation trees, the multiplication accumulators in the m×n multiplication-accumulation trees form s groups of multiplication accumulators, each group of the s groups of multiplication accumulators comprises m×n multiplication accumulators, any two multiplication accumulators in the m×n multiplication accumulators belong to different multiplication-accumulation trees in the m×n multiplication-accumulation trees, each multiplication-accumulation tree comprises s multiplication accumulators, and the s multiplication accumulators are connected end to end via data lines; The method comprises: Controlling the data storage to input the s groups of data into the calculation module; Perform a vector outer product operation based on the column vector and the row vector in each of the s groups of data to obtain s m×n intermediate result matrices corresponding to the s groups of data one by one; The corresponding elements in the s m×n intermediate result matrices are added together to obtain the product result of the first matrix and the second matrix; wherein, The calculation module performs the s vector outer product operations based on the s groups of multiplication and accumulation units respectively to obtain the s m×n intermediate result matrices, and uses the m×n multiplication and accumulation trees to add corresponding elements in the s m×n intermediate result matrices to obtain the product result of the first matrix and the second matrix.

7. The method according to claim 6, characterized in that The first matrix belongs to one of the A blocks of the third matrix, the second matrix belongs to one of the B blocks of the fourth matrix, the sizes of the A blocks are less than or equal to m×s, and the sizes of the B blocks are less than or equal to s×n, The method further comprises: Control the calculation module to multiply the A blocks and the B blocks in pairs, and obtain the product result of the third matrix and the fourth matrix based on the result of the multiplication of the A blocks and the B blocks in pairs, wherein A and B are both positive integers greater than 1.

8. The method according to claim 7, characterized in that The computing device is used to perform a convolution operation, the elements in the third matrix are data in the input feature map, and the elements in the fourth matrix are weight data in the convolution kernel, wherein the third matrix is ​​an M×S matrix, and the fourth matrix is ​​an S×N matrix, M represents the width of the input feature map, S represents the number of channels of the input feature map, and N represents the number of convolution kernels, and M, S, and N are all positive integers greater than or equal to 1.

Images