Matrix operation circuit, device and method

Abstract

The embodiments of the present disclosure disclose a matrix operation circuit, device and method. The matrix operation circuit includes: a control circuit; an operation unit array, the operation unit array includes multiple operation units, the operation unit includes a first input register, a second input register and an output register; the first input register is used to receive data of a first input matrix, and the second input register is used to receive data of a second input matrix; the control circuit is used to receive a matrix operation instruction, and in response to the instruction, controls at least one of the multiple operation units to perform an operation operation on the first input matrix and the second input matrix according to the instruction, wherein the instruction is a single instruction; the output register is used to store the operation result of the operation operation. Through the above method, the technical problems of low computational efficiency and high power consumption in the prior art when performing convolution calculations are solved.

Description

Technical Field

[0001] The present disclosure relates to the field of neural network computing, and in particular to a matrix operation circuit, device, and method. Background Art

[0002] With the advancement of science and technology, human society is rapidly entering the intelligent era. A key characteristic of this era is the increasing variety and volume of data available, coupled with the increasing demand for faster data processing. Chips are the cornerstone of data processing, fundamentally determining our ability to process data. From an application perspective, there are two main approaches to chip development: general-purpose chips, such as CPUs (Central Processing Units), offer significant flexibility but exhibit relatively low effective computing power when processing domain-specific algorithms. Specialized chips, such as TPUs (Tensor Processing Units), offer high effective computing power in specific domains, but struggle with or even fail to handle flexible, general-purpose applications. Given the diverse and massive amounts of data in the intelligent era, chips must possess both extreme flexibility to handle diverse and rapidly evolving algorithms and robust processing capabilities to rapidly process vast and rapidly growing volumes of data.

[0003] Convolution is often required in artificial intelligence computing. There are two common solutions for implementing convolution:

[0004] (1) CPU solution: In this solution, if it is a single-core CPU, the matrix will be decomposed into scalars for operation, and the convolution operation will be implemented by combining scalar instructions; if it is a multi-core CPU, it may be possible to execute the scalar instructions of multiple cores in parallel to implement the convolution operation. However, this solution has the following disadvantages: the underlying program is complex, and generally requires multiple layers of loops to implement the convolution operation; the convolution operation is implemented through general computing instructions, which is inefficient and requires multiple branches and jumps; the CPU cache is limited, and implementing a relatively large convolution operation requires multiple off-chip data transfers, which affects efficiency; the CPU needs to access data multiple times, which increases the computation time to implement the convolution operation; the CPU needs to access data multiple times, which increases the computation power consumption to implement the convolution operation; if multi-core parallel computing is used, the communication between cores is complex, and communication performance may become a bottleneck.

[0005] (2) GPU (Graphics Processing Unit) solution: In this solution, the GPU will decompose the convolution operation into multiple instructions, which are mainly vector instructions. The convolution operation is implemented by combining and executing vector instructions. However, this solution has the following disadvantages: the underlying program is complex and generally requires multiple layers of loops to implement the convolution operation; the convolution operation is implemented by combining vector instructions multiple times, which is inefficient; the GPU needs to access data multiple times, which increases the computational time and power consumption of the convolution operation; the GPU has limited cache memory, and implementing relatively large convolution operations requires multiple off-chip transfers, which affects efficiency. Summary of the Invention

[0006] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] In a first aspect, an embodiment of the present disclosure provides a matrix operation circuit, comprising:

[0008] control circuit;

[0009] an arithmetic unit array, the arithmetic unit array comprising a plurality of arithmetic units, the arithmetic units comprising a first input register, a second input register, and an output register;

[0010] The first input register is used to receive data of a first input matrix, and the second input register is used to receive data of a second input matrix;

[0011] The control circuit is configured to receive a matrix operation instruction and, in response to the instruction, control at least one of the plurality of operation units to perform an operation on the first input matrix and the second input matrix according to an instruction of the instruction, wherein the instruction is a single instruction;

[0012] The output register is used to store the operation result of the operation.

[0013] Furthermore, the instruction includes an instruction name, a first address of a first input matrix, a first address of a second input matrix, and a first address of an output matrix.

[0014] Furthermore, the matrix operation instruction is a matrix multiplication instruction.

[0015] Furthermore, the operation unit includes an operator, which includes at least a multiplier and an adder; the operation unit is used to combine the multiplier and the adder according to the matrix multiplication instruction to perform a matrix multiplication operation.

[0016] Further, in response to the matrix multiplication instruction, for each operation unit that executes the matrix multiplication instruction:

[0017] Reading data of the first input matrix in the first input register;

[0018] Reading data of the second input matrix in the second input register;

[0019] Calculating the product of the data of the first input matrix and the data of the second input matrix by a multiplier;

[0020] Calculating the accumulated value of the product by an adder;

[0021] The accumulated value is stored in the output register.

[0022] Furthermore, the matrix multiplication instruction is used to implement a matrix convolution operation, wherein:

[0023] The data of the first input matrix is ​​the row vector data of the first input matrix;

[0024] The data of the second input matrix is ​​the row vector data of the second input matrix.

[0025] In a second aspect, an embodiment of the present disclosure provides a matrix computing device, comprising:

[0026] A memory, configured to store a matrix operation instruction, a first input matrix, a second input matrix, and an output matrix;

[0027] an instruction fetch module, connected to the memory, and configured to fetch the matrix operation instruction from the memory;

[0028] A decoding module, connected to the instruction fetch module, for decoding the matrix operation instruction obtained by the instruction fetch module;

[0029] a register, configured to store attribute data of the first input matrix, the second input matrix, and the output matrix;

[0030] An execution module is connected to the decoding module, the memory and the register, and includes the matrix operation circuit according to claims 1-6, and is used to execute the decoded matrix operation instruction.

[0031] Furthermore, the execution module obtains the decoded matrix operation instruction from the decoding module; the execution module obtains the attribute data of the first input matrix, the attribute data of the second input matrix and the attribute data of the output matrix from the register; the execution module obtains the data of the first input matrix and the data of the second input matrix used for calculation from the memory according to the attribute data of the first input matrix and the second input matrix; the execution module calculates the data of the first input matrix and the second input matrix according to the decoded matrix operation instruction to obtain the data of the output matrix; the execution module stores the data of the output matrix in the memory according to the attribute data of the output matrix.

[0032] Furthermore, the attribute data of the first input matrix includes the number of rows, the number of columns and the row vector interval of the first input matrix; the attribute data of the second input matrix includes the number of rows, the number of columns and the row vector interval of the second input matrix; and the attribute data of the output matrix includes the number of rows, the number of columns and the row vector interval of the output matrix.

[0033] Furthermore, the execution module obtains data of the first input matrix and data of the second input matrix for calculation from the memory according to attribute data of the first input matrix and the second input matrix, including:

[0034] The execution module reads the data of the first input matrix according to a preset first reading method and the attribute data of the first input matrix;

[0035] The execution module reads the data of the second input matrix according to a preset second reading method and attribute data of the second input matrix.

[0036] Furthermore, the first reading method is reading by row or reading by column; the second reading method is reading by row or reading by column.

[0037] In a third aspect, an embodiment of the present disclosure provides a matrix operation method, which is a matrix operation method based on any matrix operation circuit described in the first aspect, including:

[0038] Fetch matrix operation instructions from memory;

[0039] Decoding the matrix operation instruction and sending the decoded instruction operation instruction to the matrix operation circuit;

[0040] Based on the decoded matrix operation instruction, the matrix operation circuit obtains the data of the first input matrix and the data of the second input matrix from the memory and performs operation, and stores the operation result in the memory after the operation is completed.

[0041] In a fourth aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory for storing computer-readable instructions; and one or more processors for executing the computer-readable instructions, so that the processors implement any of the matrix operation methods described in the third aspect when running.

[0042] In a fifth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any matrix operation method described in the third aspect.

[0043] In a sixth aspect, an embodiment of the present disclosure provides a computer program product, comprising computer instructions. When the computer instructions are executed by a computing device, the computing device can execute any matrix operation method described in the aforementioned third aspect.

[0044] In a seventh aspect, an embodiment of the present disclosure provides a chip comprising any matrix operation circuit described in the first aspect.

[0045] In an eighth aspect, an embodiment of the present disclosure provides a computing device comprising any chip described in the seventh aspect.

[0046] The embodiments of the present disclosure disclose a matrix operation circuit, device and method. The matrix operation circuit includes: a control circuit; an operation unit array, the operation unit array includes multiple operation units, the operation unit includes a first input register, a second input register and an output register; the first input register is used to receive data of a first input matrix, and the second input register is used to receive data of a second input matrix; the control circuit is used to receive a matrix operation instruction, and in response to the instruction, controls at least one of the multiple operation units to perform an operation operation on the first input matrix and the second input matrix according to the instruction, wherein the instruction is a single instruction; the output register is used to store the operation result of the operation operation. Through the above method, the technical problems of low computational efficiency and high power consumption in the prior art when performing convolution calculations are solved.

[0047] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0049] Figure 1 A schematic diagram of the structure of a matrix operation circuit provided in an embodiment of the present disclosure;

[0050] Figure 2 A schematic diagram of the structure of a computing unit provided in an embodiment of the present disclosure;

[0051] Figure 3 A further structural diagram of the computing unit provided in the embodiment of the present disclosure;

[0052] Figure 4a-4b A schematic diagram of a convolution operation in an embodiment of the present disclosure;

[0053] Figure 5-Figure 8 The calculation process of partial convolution provided by the embodiment of the present disclosure;

[0054] Figure 9 A schematic diagram of the structure of a matrix computing device provided in an embodiment of the present disclosure;

[0055] Figures 10a-10d A schematic diagram of the storage order and format of the matrix in the present disclosure;

[0056] Figure 11-Figure 15 Schematic diagram of data overlap in convolution operation;

[0057] Figure 16 A schematic diagram of a specific example of convolution calculation in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0059] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0060] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0061] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0062] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0063] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0064] Figure 1 This is a schematic diagram of the structure of the matrix operation circuit provided by the embodiment of the present disclosure. Figure 1 As described above, the matrix operation circuit 100 includes a control circuit 101 and an operation unit array 102, wherein the operation unit array includes a plurality of operation units (PUs) 103, and the operation units 103 are as follows: Figure 2 As shown, it includes a first input register (Rin1) 104, a second input register (Rin2) 105 and an output register (Rout) 106, wherein the first input register 104 is used to receive data of a first input matrix, and the second input register 105 is used to receive data of a second input matrix; the control circuit 101 is used to receive a matrix operation instruction, and in response to the instruction, controls at least one operation unit 103 of the multiple operation units 103 to perform an operation operation on the first input matrix and the second input matrix according to the instruction, wherein the instruction is a single instruction; the output register 106 is used to store the operation result of the operation.

[0065] In the present disclosure, the single instruction includes the instruction name, the first address of the first input matrix, the first address of the second input matrix, and the first address of the output matrix. The following table shows an exemplary instruction format:

[0066]

[0067] The instruction name corresponds to the instruction meaning, format, and operation implemented. The first address of the first input matrix and the first address of the second input matrix define the read addresses of the two source operands of the instruction respectively, and the first address of the output matrix defines the storage address of the destination operand of the instruction. The above exemplary instruction is a matrix multiplication instruction, which implements the multiplication of two matrices. Specifically:

[0068]

[0069] C in =Σ(a ij *b jn )

[0070] Figure 3 This is a further structural diagram of the computing unit provided in the embodiment of the present disclosure. Figure 3 As shown, in addition to the first input register, the second input register and the output register, the operation unit also includes an operator, which includes at least a multiplier 301 and an adder 302, wherein the operation unit is used to combine the multiplier and the adder according to the matrix multiplication instruction to perform a matrix multiplication operation.

[0071] Specifically, in response to the matrix multiplication instruction, for each operation unit executing the matrix multiplication instruction: read the data of the first input matrix in the first input register; read the data of the second input matrix in the second input register; calculate the product of the data of the first input matrix and the data of the second input matrix through a multiplier; calculate the accumulated value of the product through an adder; and store the accumulated value into the output register. The data of the first input matrix in the first input register is the data sequentially read into the first input register according to the first address of the first input matrix; the data of the second input matrix in the second input register is the data sequentially read into the second input register according to the first address of the second input matrix.

[0072] For example, after reading in the data, the product of the data in the first input register and the second input register is calculated by the multiplier. As in the above example, in one clock cycle, if the data received in the first input register is a11 and the data received in the second input register is b11, the multiplier in the operation unit calculates the product of a11*b11, and then sends the product of a11*b11 to the adder for accumulation. Since the accumulator has no input in the previous clock cycle at this time, the accumulated result of the accumulator is a11*b11; then in the next clock cycle, the above calculation continues. In this case, if the data received in the first input register is a12 and the data received in the second input register is b21, the multiplier in the arithmetic unit calculates the product of a12*b21 and then feeds this product into the adder for accumulation. The input to the accumulator in the previous clock cycle was a11*b11, so a11*b11+a12*b21 is calculated in the current clock cycle. This operation is continued until one row of the first input matrix and one column of the second input matrix are calculated, obtaining the final accumulated value, which is then stored in the output register. The control circuit stores the accumulated value in the output register in the system memory according to the output matrix starting address in the instruction.

[0073] In the present disclosure, the matrix multiplication instruction can implement a matrix convolution operation. The matrix convolution operation is the cumulative sum of the products of the point-by-point multiplication of the data in two matrices.

[0074] An example convolution operation is as follows:

[0075] Figure 4a-4b FIG is a schematic diagram of a convolution operation in an embodiment of the present disclosure. Figure 4a This is a schematic diagram of the overall convolution operation process, where the diagrams are explained as follows:

[0076] Win: width of input feature map;

[0077] Hin: the height of the input feature map;

[0078] Cin: The number of channels of the input feature map, hereinafter referred to as the depth of the input feature map;

[0079] Kw: width of the convolution kernel;

[0080] Kh: height of the convolution kernel

[0081] Wout: width of the output feature map;

[0082] Hout: the height of the output feature map;

[0083] Cout: The number of channels of the output feature map, hereinafter referred to as the depth of the output feature map.

[0084] The feature points on the input feature map form the first input matrix; the points on the convolution kernel form the second input matrix; the output feature map forms the output matrix, and each feature point on the output feature map is a piece of data on the output matrix. During the convolution operation, the convolution kernel slides across the input feature map. Each time it slides, it multiplies and accumulates the corresponding data in the input feature map to extract an output feature point, which is a piece of data on the output matrix.

[0085] Figure 4b This is a calculation diagram of an input feature point with depth. Figure 4b As shown in the figure, the convolution kernel slides on the input feature map. When it stops at a position, it multiplies and accumulates the corresponding data with the feature point in the input feature map at that position to obtain the output feature point corresponding to that position. There are Cout convolution kernels in total. Each convolution kernel multiplies and accumulates the data with the feature point in the input feature map at the same position to obtain Cout output feature points in the depth direction. The Cout output feature points constitute a feature point with depth on the entire output feature map, and the depth of this point is Cout. The convolution kernel slides over the entire input feature map to obtain the entire output feature map.

[0086] For a convolution kernel at depth l (1 <= l <= Cout), the formula for feature extraction is as follows:

[0087]

[0088] The data corresponding to the convolution kernel in the input feature map, the superscript i corresponds to the depth of the input feature map, j and k correspond to the width and height of the convolution kernel respectively; w is the convolution kernel, its superscript l and i correspond to the depth of the output feature map and the depth of the input feature map respectively, j and k correspond to the width and height of this convolution kernel respectively.

[0089] For a convolution kernel of size Kh*Kw*Cin, it can be divided into Kh partial convolution kernels of Kw*Cin for partial feature extraction. Each time, 1 / Kh of the entire feature extraction is achieved, that is, the feature corresponding to the partial convolution kernel of Kw*Cin. The partial results obtained are: Finally, add up these Kh partial results to get the final result

[0090] in, It can be divided into Kw steps, each step is realized Then add up the Kw partial results to get the final result The implementation is to multiply a row of input data matrix (i.e., part of the first input matrix) and a column of weight matrix (i.e., part of the convolution kernel), which is implemented as follows Figure 5 shown. The implementation is also a multiplication of a row of input data matrix and a column of weight matrix, but the number of data in the rows and columns is Kw times the number of data in the , which is achieved as follows Figure 6 As shown. The number of convolution kernels is Cout, so the depth of the output feature point is Cout. You can multiply the input data matrix of one row with the convolution kernel matrix of the Cout column composed of Cout convolution kernels, that is, the weight matrix, to obtain a feature point with depth. This feature point is a vector, and the length of the vector is the depth Cout of the output feature point. It is implemented as follows Figure 7 shown.

[0091] Since the process of implementing convolution or partial convolution of the neural network is the sliding process of the convolution kernel or part of the convolution kernel on the input feature map, it can be regarded as a process in which the data of the input feature map changes with sliding while the weight remains unchanged. In this way, the process of implementing convolution of the neural network becomes the multiplication of the input data matrix of the Wout row and the weight matrix of the Cout column, and the output data matrix of the Wout row is obtained. Its implementation is as follows Figure 8 shown.

[0092] In the above process, Kw points out of the Kh*Kw points in the entire convolution kernel have been calculated. Since the entire convolution kernel is divided into Kh parts, each part is Kw points, the final result is to add up the Kh calculation results to get the true convolution result.

[0093] The above-mentioned convolution operation only needs to use a single instruction in the present disclosure, that is, the above-mentioned matrix multiplication instruction can complete the entire convolution process, and it only needs to pre-set the order of data reading in the upper-level program. Specifically, the matrix multiplication instruction is used to implement the convolution operation of the matrix, wherein: the data of the first input matrix is ​​the row vector data of the first input matrix; the data of the second input matrix is ​​the row vector data of the second input matrix. In other words, the data in the first input register and the data in the second input register both need to be the row vector data of the matrix, so that the calculated data is the convolution result. As Figure 16 The figure shows a schematic diagram of the above convolution calculation. For example, the first input matrix is ​​a 4*4*2 matrix, the second input matrix is ​​two 3*3*2 convolution kernels, the step size is 1, and the output matrix is ​​a 2*2*2 matrix. When calculating, the partial convolution method is used to first calculate the intermediate value of each point in the output matrix, such as Figure 16As shown, when partial convolution is performed, a row of the second input matrix slides on the first input matrix, and the data of the first input matrix read out is shown in 1601. Each row is the data of the first input matrix corresponding to a position of the second input matrix 1602, including 3 numbers with depth, a total of 6 data, a column in 1602 is a row of 3 numbers including depth of the second input matrix, a total of 6 data, a row of data in 1601 and a column of data in 1602 are multiplied and added to obtain a point in 1603, and a point in 1603 is the result of partial convolution. In this example, the result of the partial convolution is part of the value of a point in the output matrix. Finally, the three rows of the second input matrix are respectively slid on the second input matrix to extract the data calculation results and accumulate them to obtain the value of a point in the output matrix (one of the two values ​​of the point with a depth of 2).

[0094] Combine Figure 1-3 The PU array shown briefly explains how the PU array implements convolution calculations.

[0095] Will Figure 16 The value 1 of the first input matrix data 1601 is sent to Rin1 of PU11 and Rin1 of PU12, and the value 3 of the first input matrix data 1601 is sent to Rin1 of PU21 and Rin1 of PU22; the value 0.1 in the second input matrix 1602 is sent to Rin2 of PU11 and Rin2 of PU21, and the value 1.9 in the second input matrix 1602 is sent to Rin2 of PU12 and Rin2 of PU22; PU11, PU12, PU21, and PU22 perform data multiplication operations and send the results to the output register for storage; in the next clock cycle, the The value 2 of the data 1601 of the first input matrix is ​​sent to Rin1 of PU11 and Rin1 of PU12, and the value 4 of the data 1601 of the first input matrix is ​​sent to Rin1 of PU21 and Rin1 of PU22; the value 0.2 in the second input matrix 1602 is sent to Rin2 of PU11 and Rin2 of PU21, and the value 2.0 in the second input matrix 1602 is sent to Rin2 of PU12 and Rin2 of PU22. PU11, PU12, PU21, and PU22 perform the data multiplication operation and send the result to the output register for accumulation with the last saved result.

[0096] By analogy, the multiplication and accumulation results of the partial convolution are finally obtained.

[0097] Figure 9 This is a schematic diagram of the structure of the matrix computing device provided in the embodiment of the present disclosure. Figure 9As shown, the matrix calculation device 900 includes: a memory 901, which is used to store matrix operation instructions, a first input matrix, a second input matrix and an output matrix; an instruction fetch module 902, which is connected to the memory 901 and is used to obtain the matrix operation instructions from the memory 901; a decoding module 903, which is connected to the instruction fetch module 902 and is used to decode the matrix operation instructions obtained by the instruction fetch module 902; a register 904, which is used to store attribute data of the first input matrix, the second input matrix and the output matrix; and an execution module 905, which is connected to the decoding module 903, the memory 901 and the register 904, and includes the matrix operation circuit in the above embodiment, and is used to execute the decoded matrix operation instructions.

[0098] In one embodiment, the execution module obtains the decoded matrix operation instruction from the decoding module; the execution module obtains the attribute data of the first input matrix, the attribute data of the second input matrix, and the attribute data of the output matrix from the register; the execution module obtains the data of the first input matrix and the data of the second input matrix for calculation from the memory according to the attribute data of the first input matrix and the second input matrix; the execution module calculates the data of the first input matrix and the second input matrix according to the decoded matrix operation instruction to obtain the data of the output matrix; the execution module stores the data of the output matrix in the memory according to the attribute data of the output matrix. Wherein, the attribute data of the first input matrix includes the number of rows, the number of columns, and the row vector spacing of the first input matrix; the attribute data of the second input matrix includes the number of rows, the number of columns, and the row vector spacing of the second input matrix; the attribute data of the output matrix includes the number of rows, the number of columns, and the row vector spacing of the output matrix. Among them, the number of rows and columns defines the size of the matrix, and the row vector interval defines the storage address difference between two adjacent rows of the matrix. For example, each row of the matrix has 10 int8 matrix elements. If the two rows are stored continuously, then the row vector interval is 10 bytes. If the two rows are stored at a certain interval, such as 20 bytes, then 10 bytes are matrix elements, and the other 10 bytes do not belong to this matrix, which may be invalid data or data for other purposes.

[0099] Optionally, the execution module acquires data of the first input matrix and data of the second input matrix for calculation from the memory based on the attribute data of the first input matrix and the second input matrix, including: the execution module reads the data of the first input matrix based on a preset first reading method and the attribute data of the first input matrix; the execution module reads the data of the second input matrix based on a preset second reading method and the attribute data of the second input matrix. The first reading method is reading by row or by column; and the second reading method is reading by row or by column.

[0100] For example, if the attribute data defines that the number of rows of the first input matrix is ​​5 rows, the number of columns is 5 columns, and the row spacing is 5 bytes, and the preset reading method is to read by row, then the first row of the first input matrix is ​​read according to the first address of the first input matrix in the instruction and the row spacing. The number of columns shows that the first row has 5 matrix elements. Then, the first address plus the row spacing is used as the new first address, and the second row is read, which also has 5 matrix elements. After reading five times in sequence, the execution module can obtain all the data of the first input matrix.

[0101] Similarly, the data of the second input matrix is ​​obtained in the same manner. Optionally, the execution module stores the data of the output matrix in the memory according to the attribute data of the output matrix, including: the execution module stores the data of the output matrix in the memory according to a preset storage method and the attribute data of the output matrix. The preset storage method is row-by-row storage or column-by-column storage. The specific storage method is similar to the reading method, except that the direction is opposite, and will not be repeated here.

[0102] Figure 10a Schematic diagram of the storage order and format of the first input matrix in the present disclosure. Figure 10a As shown, it is an example of the first input matrix in the above embodiment. When it is stored in the memory, it is stored in the order of depth Cin first, then width Win, and finally height Hin. Taking Cin=2, Win=3, Hin=3 as an example, the first point of the first row in Hin=3 rows is stored first. Due to the depth, the point contains 2 data. Then the second point is stored, which also contains 2 data, until the third point is stored. This row has a total of 3*2=6 data, including 3 points in the Win direction; then the first point of the second row in Hin=3 rows is stored, and the storage is carried out in this way until all the points are stored. An example of the storage order and format of the first input matrix is ​​as follows: Figure 10b shown.

[0103] Figure 10c is a schematic diagram of the storage order and format of the second input matrix of the present disclosure, such as Figure 10c As shown, it is an example of the second input matrix in the above embodiment. When it is stored in the memory, it is stored in rows with priority given to Cout (the number of convolution kernels), and one convolution kernel is stored in each column. In the column direction, it is stored with priority given to the depth Cin of the convolution kernel, followed by the width Kw, and then the height Kh. Taking Cout=2, Cin=2, Kw=2, Kh=2 as an example, a total of Cin*Kw*Kh=8 ​​rows of data will be stored, with 2 data stored in each row. First, the first data of the point with a depth of 2 in the first row and first column of the first convolution kernel is stored, and then the first data of the point with a depth of 2 in the first row and first column of the second convolution kernel is stored to complete the storage of the first row of data of the second input matrix; then, the second data of the point with a depth of 2 in the first row and first column of the first convolution kernel is stored, and then the second data of the point with a depth of 2 in the first row and first column of the second convolution kernel is stored to complete the storage of the second row of data of the second input matrix, and so on, and so on, until all points are stored. An example of the storage order and format of the second input matrix is ​​as follows: Figure 10d shown.

[0104] like Figure 10a RS1 shown in is the first address of the first input matrix. The data of the matrix read in can be controlled by setting the storage format and the reading method. For example, if the row vector data of the first input matrix is ​​read out according to the row order storage in the above example, combined with row reading and row interval, the data can be stored. Figure 10c RS2 is the starting address of the second input matrix. By setting the storage format and read mode, the matrix data to be read can be controlled. For example, if the storage order is followed in the example above, combined with row-by-row read and row spacing, the row vector data of the second input matrix can be read. After the data of the first and second input matrices are read, the above-mentioned matrix multiplication instruction is used to complete the convolution operation in a single instruction.

[0105] Before calling the above matrix multiplication instruction, the attribute data of the first input matrix, the second input matrix and the output matrix can be set. In the present disclosure, multiple registers are defined to store the attribute data of each matrix. An example configuration of the registers is shown in the following table:

[0106]

[0107] Figure 11-Figure 15 This is a schematic diagram of data overlap in convolution operation. Figure 11 As shown in , in the convolution operation, the data used to calculate the previous output feature point often overlaps with the data used to calculate the next output feature point. Figure 11In the example, we use a 3*3 convolution kernel and a stride (the distance the convolution kernel slides each time) of 1 to perform convolution and calculate the second point in the second row. After calculating the second point in the second row, as shown in Figure 12 As shown in , the convolution kernel slides one point to the right to calculate the third point in the second row. From the sliding process of the input data, when calculating the two points before and after, the input data partially overlaps. The overlapping part is as follows Figure 13 As shown in the gray part in . When only considering part of the convolution, such as Figure 14 As shown in the figure, the gray part is the overlapping data between the two calculations. This is equivalent to the fact that when the convolution kernel slides, two-thirds of the input matrix data is repeated in the convolution calculations at two adjacent positions.

[0108] Assume that the partial convolution input data is matrix 1, and the data in matrix 1 is recorded as x, y and z are Figure 14 The coordinates of the coordinate system in , and assuming that the weight of the partial convolution is matrix 2, the data in matrix 2 is recorded as w has one more dimension of Cout, so a dimension is added to its superscript, and the data in the output matrix is ​​recorded as The matrix calculation process is as follows Figure 15 When calculating the second point in the second row of the output feature map, the data in the second row of matrix one is multiplied and accumulated with each column of matrix two to obtain a second point with depth on the output feature map. The depth of the second point is 8. Figure 15 The second row of the output matrix is ​​represented as It is a point in the output feature map, the depth of this point is Cout, and each data sequence in the depth corresponds to each data of the output matrix; when calculating the third point in the first row of the output feature map, the data in the third row of matrix one is multiplied and accumulated with each column of matrix two to obtain the third point with depth on the output feature map. The third point is a point with a depth of 8. Figure 15 It can be seen that when calculating the second point of the second row with depth and the third point of the second row with depth, the 8 data in the gray part are overlapped. If they are stored in the way of matrix 1, a large amount of memory will be wasted.

[0109] The above-mentioned problem in the calculation can be solved by setting the row interval. Taking the above-mentioned embodiment as an example, the number of row data of the first input matrix is ​​4*3=12 (4 is the depth of the first input matrix, and 3 is the number of columns of the second data matrix). Before using the above-mentioned matrix multiplication instruction, the row interval represented by the 15:0 bits in the above-mentioned register Stride2 is set to 4. Then, between the two rows, 12-4=8 data will be repeatedly filled into the above-mentioned matrix calculation device, thereby realizing the process of realizing the entire partial convolution with one instruction, and there is no need to repeatedly store the repeated data during storage, saving memory space. At the same time, the register of the output matrix is ​​set so that the output data is also stored sequentially, in the same order as the input data. Then, if there is a convolution or other operation later, there is no need to adjust the data shape and it can be used directly, saving the time and power consumption required for data adjustment.

[0110] An embodiment of the present disclosure also provides a matrix operation method, which is a matrix operation method based on any of the aforementioned matrix operation circuits, including: taking out a matrix operation instruction from a memory; decoding the matrix operation instruction, and sending the decoded instruction operation instruction to the matrix operation circuit; based on the decoded matrix operation instruction, the matrix operation circuit obtains the data of the first input matrix and the second input matrix data from the memory and performs an operation, and stores the operation result in the memory after the operation is completed.

[0111] An embodiment of the present disclosure further provides an electronic device, comprising: a memory for storing computer-readable instructions; and one or more processors for executing the computer-readable instructions, so that the processors implement any of the matrix operation methods in the aforementioned embodiments when executed.

[0112] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, which stores computer instructions for causing a computer to execute any of the matrix operation methods described in the aforementioned embodiments.

[0113] An embodiment of the present disclosure provides a computer program product, including computer instructions. When the computer instructions are executed by a computing device, the computing device can perform any of the matrix operation methods in the aforementioned embodiments.

[0114] An embodiment of the present disclosure provides a chip, comprising the matrix operation circuit described in any one of the aforementioned embodiments.

[0115] An embodiment of the present disclosure provides a computing device, comprising the chip described in any one of the aforementioned embodiments.

[0116] The flowcharts and block diagrams in the accompanying drawings of the present disclosure illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0117] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0118] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0119] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Claims

1. A matrix operation circuit comprising: control circuit; an arithmetic unit array, the arithmetic unit array comprising a plurality of arithmetic units, the arithmetic units comprising a first input register, a second input register, and an output register; The first input register is used to receive data of a first input matrix, and the second input register is used to receive data of a second input matrix; The control circuit is configured to receive a matrix operation instruction and, in response to the instruction, control at least one of the plurality of operation units to perform an operation on the first input matrix and the second input matrix according to an instruction of the instruction, wherein the instruction is a single instruction and has a corresponding preset reading order of the first input matrix and the second input matrix; The output register is used to store the operation result of the operation; In which, the matrix operation circuit obtains the data of the first input matrix and the second input matrix used for calculation from the corresponding memory according to the attribute data of the first input matrix and the second input matrix, the attribute data includes the number of rows, the number of columns and the row vector interval, the row vector interval represents the storage address difference between two adjacent rows of the corresponding matrix, and the value of the row vector interval is configured to avoid repeated storage of data.

2. The matrix operation circuit according to claim 1, wherein: The instruction includes an instruction name, a first address of a first input matrix, a first address of a second input matrix, and a first address of an output matrix.

3. The matrix operation circuit according to claim 1 or 2, wherein: The matrix operation instruction is a matrix multiplication instruction.

4. The matrix operation circuit according to claim 3, wherein: The arithmetic unit includes an arithmetic unit, and the arithmetic unit includes at least a multiplier and an adder; The operation unit is used to combine the multiplier and the adder according to the matrix multiplication instruction to perform a matrix multiplication operation.

5. The matrix operation circuit according to claim 4, wherein: In response to the matrix multiplication instruction, for at least one operation unit executing the matrix multiplication instruction: Reading data of the first input matrix in the first input register; Reading data of the second input matrix in the second input register; Calculating the product of the data of the first input matrix and the data of the second input matrix by a multiplier; Calculating the accumulated value of the product by an adder; The accumulated value is stored in the output register.

6. The matrix operation circuit according to claim 5, wherein: The matrix multiplication instruction is used to implement a matrix convolution operation, where: The data of the first input matrix is ​​the row vector data of the first input matrix; The data of the second input matrix is ​​the row vector data of the second input matrix.

7. A matrix computing device, comprising: A memory, configured to store a matrix operation instruction, a first input matrix, a second input matrix, and an output matrix; an instruction fetch module, connected to the memory, and configured to fetch the matrix operation instruction from the memory; A decoding module, connected to the instruction fetch module, for decoding the matrix operation instruction obtained by the instruction fetch module; a register, configured to store attribute data of the first input matrix, the second input matrix, and the output matrix; an execution module, connected to the decoding module, the memory and the register, comprising the matrix operation circuit according to any one of claims 1 to 6, and configured to execute the decoded matrix operation instruction; In which, the execution module obtains the data of the first input matrix and the second input matrix used for calculation from the memory according to the attribute data of the first input matrix and the second input matrix, the attribute data includes the number of rows, the number of columns and the row vector interval, the row vector interval represents the storage address difference between two adjacent rows of the corresponding matrix, and the value of the row vector interval is configured to avoid repeated storage of data.

8. The matrix calculation device according to claim 7, wherein: The execution module obtains the decoded matrix operation instruction from the decoding module; The execution module obtains the attribute data of the first input matrix, the attribute data of the second input matrix, and the attribute data of the output matrix from the register; The execution module calculates the data of the first input matrix and the data of the second input matrix according to the decoded matrix operation instruction to obtain the data of the output matrix; The execution module stores the data of the output matrix into the memory according to the attribute data of the output matrix.

9. The matrix calculation device according to claim 8, wherein: The attribute data of the first input matrix includes the number of rows, the number of columns and the row vector interval of the first input matrix; The attribute data of the second input matrix includes the number of rows, the number of columns and the row vector interval of the second input matrix; The attribute data of the output matrix includes the number of rows, the number of columns and the row vector interval of the output matrix.

10. The matrix calculation device according to claim 8, wherein: The execution module acquires data of the first input matrix and the second input matrix for calculation from the memory according to attribute data of the first input matrix and the second input matrix, including: The execution module reads the data of the first input matrix according to a preset first reading method and the attribute data of the first input matrix; The execution module reads the data of the second input matrix according to a preset second reading method and attribute data of the second input matrix.

11. The matrix calculation device according to claim 10, wherein: The first reading mode is reading by row or reading by column; The second reading method is reading by row or reading by column.

12. A matrix operation method based on the matrix operation circuit according to any one of claims 1 to 6, characterized in that: include: Fetch matrix operation instructions from memory; Decoding the matrix operation instruction and sending the decoded instruction operation instruction to the matrix operation circuit; Based on the decoded matrix operation instruction, the matrix operation circuit obtains the data of the first input matrix and the second input matrix data from the memory and performs operation, and stores the operation result in the memory after the operation is completed.

Images