Computing apparatus and method, and device, chip and system

Abstract

Disclosed are a computing apparatus and method, and a device, a chip and a system, which relate to the field of computers. The computing apparatus provided in the present application executes the computation of compressed sparse-format matrices on the basis of compression identifiers corresponding to element values in the compressed sparse-format matrices (for example, a matrix inner product and a matrix outer product), and outputs a matrix inner product result or a matrix outer product result of the compressed sparse-format matrices, such that the compressed sparse-format matrices are efficiently computed by using the large computing power of a computing unit that supports matrix computation, and the computing unit is expanded to support the matrix computation of unstructured sparse matrices, thereby improving the computational efficiency of the sparse matrices. Moreover, a merging circuit is provided inside the computing apparatus to accumulate intermediate result matrices of a matrix outer product in the computing apparatus, and a register is provided in the computing apparatus to cache intermediate results of the compressed sparse-format matrices, thereby reducing the storage space and access bandwidth required for computing the sparse matrices and further improving the computational efficiency of the sparse matrices.

Description

Computing device, method, equipment, chip and system

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on July 21, 2023, with application number 202310911966.X and application name “Computing Devices, Methods, Equipment, Chips and Systems”, all of the contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of computers, and in particular to a computing device, method, equipment, chip, and system. Background Art

[0003] Currently, matrix calculations are used in applications such as artificial intelligence, scientific computing, and graphics computing. In recent years, matrix-specific processors have become commonplace, increasing the speed of matrix calculations. However, these processors are inefficient for sparse matrix calculations.

[0004] Summary of the Invention

[0005] The present application provides computing devices, methods, equipment, chips and systems to improve the computing efficiency of sparse matrices.

[0006] In a first aspect, a computing device is provided, which includes a computing unit and a merging circuit. The computing unit is used to calculate the matrix outer product of the first matrix and the second matrix based on the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices. The first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices. The compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix. The compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix. The merging circuit is used to accumulate the element values ​​with the same compression identifier in the multiple intermediate result matrices to obtain the matrix outer product result.

[0007] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculations on the decompressed matrix, or performing matrix calculations on a compressed sparse format matrix based on the position coordinates of the compressed sparse format matrix, which results in a low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a merging circuit. By utilizing the large computing power of the computing unit that supports matrix calculations, the matrix outer product of the compressed sparse format matrix is ​​efficiently calculated based on the compression identifier of the compressed sparse format matrix, so that the computing unit is expanded to support matrix calculations of unstructured sparse matrices. The merging circuit is used to realize the accumulation of intermediate result matrices of the matrix outer product within the computing device, effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate result matrices of the compressed sparse format matrix are cached within the computing device, reducing the storage space and access bandwidth required for sparse matrix calculations, further improving the computational efficiency of the compressed sparse format matrix.

[0008] In one possible implementation, the merging circuit is specifically used to: accumulate the element values ​​of the same compression identifier in multiple intermediate result matrices to obtain multiple intermediate merged matrices, where the intermediate merged matrices are compressed sparse format matrices; accumulate the element values ​​of the same compression identifier in multiple intermediate merged matrices to obtain a matrix outer product result.

[0009] In another possible implementation, the merging circuit is specifically configured to accumulate element values ​​with the same compression identifier in multiple intermediate merging matrices obtained from the memory to obtain a matrix outer product result.

[0010] In another possible implementation, the merging circuit includes a sorting unit and an addition processing unit: the sorting unit is used to sort the elements in multiple intermediate result matrices according to the order of compression identifiers from small to large, or to sort the elements in multiple intermediate merge matrices, where the elements include element values ​​and compression identifiers; the addition processing unit is used to accumulate the element values ​​of the same compression identifiers in the sorted elements.

[0011] In another possible implementation, the merging circuit further includes an arranging unit; the arranging unit is configured to arrange the elements whose valid values ​​indicate validity, according to the valid values ​​contained in the elements in the intermediate merging matrix.

[0012] In another possible implementation, the merging circuit further includes a de-bounding unit; the de-bounding unit is used to remove elements corresponding to compression identifiers other than the minimum compression identifier in the maximum compression identifier of the two vectors in the accumulated vector of the two vectors, the two vectors are vectors other than the last two vectors in the matrix to be merged, and the matrix to be merged includes either an intermediate merging matrix or an intermediate result matrix.

[0013] In another possible implementation, the computing device further includes a first cache unit; the first cache unit is configured to store an intermediate result matrix, where the intermediate result matrix includes an element value product matrix, a compression identification matrix, and a valid value matrix.

[0014] In another possible implementation, the merging circuit further includes a second cache unit; the second cache unit is configured to store the intermediate merging matrix.

[0015] In a second aspect, a computing device is provided, which includes a computing unit and a register; the computing unit is used to calculate the matrix inner product of the first matrix and the second matrix based on the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the computing unit is also used to accumulate the element values ​​of the same position coordinates in multiple intermediate results to obtain the matrix inner product result; the register is used to store multiple intermediate results and the matrix inner product results.

[0016] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculations on the decompressed matrix, or performing matrix calculations on a compressed sparse format matrix based on the position coordinates of the compressed sparse format matrix, which results in a low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a register, which utilizes the large computing power of the computing unit that supports matrix calculations, and efficiently calculates the matrix inner product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is expanded to support matrix calculations of unstructured sparse matrices, and the computing unit is used to realize the accumulation of intermediate results of the matrix inner product within the computing device, thereby effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate results of the compressed sparse format matrix are cached within the computing device, reducing the storage space and access bandwidth required for sparse matrix calculations, further improving the computational efficiency of the compressed sparse format matrix.

[0017] In a possible implementation, the computing unit is specifically configured to perform a vector inner product calculation on element values ​​corresponding to the same compression identifier in the row vector and the column vector to obtain an intermediate result.

[0018] In another possible implementation, the computing unit is specifically configured to add the first intermediate result output by the computing unit to the value indicated by the same position coordinate in the register to obtain a second intermediate result.

[0019] In another possible implementation, the computing unit includes a sorting circuit, a multiplication processing circuit, and an aggregation circuit; the sorting circuit is used to sort the element values ​​of the row vector and the element values ​​of the column vector according to the compression identifier of the row vector and the compression identifier of the column vector; the multiplication processing circuit is used to multiply the element values ​​with the same compression identifier in the sorted elements to obtain multiple calculation results, where the elements include element values ​​and compression identifiers; the aggregation circuit is used to accumulate multiple calculation results and values ​​determined in the register based on the same position coordinates.

[0020] In another possible implementation, the multiplication processing circuit includes N multipliers and N selectors, one multiplier is connected to one selector, N represents the number of row vectors and the number of column vectors of the compressed sparse format matrix; the i-th selector is used to determine the value of the input multiplier based on the elements adjacent to the n-th element and the compression identifiers of the elements adjacent to the n-th element, the elements adjacent to the n-th element include the n+1-th element and the n-1-th element, the value range of i is 1-N, and the value range of n is an odd or even number in 1-2N; the i-th multiplier is used to perform a multiplication calculation on the value of the n-th element and the value output by the selector.

[0021] In another possible implementation, the aggregation circuit includes N adders; the N adders are used to accumulate output results of the N multipliers and values ​​indicated by the same position coordinates in the register.

[0022] In another possible implementation, when the matrix inner product calculation is completed, the intermediate result stored in the register is transferred to the memory.

[0023] It should be noted that the compressed sparse format matrix described in the above aspects or any possible implementation of each aspect includes a compressed sparse column format matrix and a compressed sparse row format matrix.

[0024] In a third aspect, a calculation method is provided, which is executed by a calculation device, the calculation device including a calculation unit and a merging circuit, the method including: the calculation unit calculates the matrix outer product of the first matrix and the second matrix based on the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the merging circuit accumulates the element values ​​with the same compression identifier in multiple intermediate result matrices to obtain the matrix outer product result.

[0025] In a fourth aspect, a computing device is provided, and the method is executed by the computing device, the computing device includes a computing unit and a register, and the method includes: the computing unit calculates the matrix inner product of the first matrix and the second matrix based on the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the computing unit accumulates the element values ​​of the same position coordinates in multiple intermediate results to obtain the matrix inner product result; the register stores multiple intermediate results and the matrix inner product results.

[0026] In a fifth aspect, a chip is provided, comprising: a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to implement the function of the computing device in the first aspect or any possible implementation of the first aspect; the processor is used to implement the function of the computing device in the second aspect or any possible implementation of the second aspect.

[0027] In a sixth aspect, a computer device is provided, which includes at least one processor and a memory, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, it implements the functions of the computing device in the first aspect or any possible implementation of the first aspect, or implements the functions of the computing device in the second aspect or any possible implementation of the second aspect.

[0028] In a seventh aspect, a computing system is provided, comprising a processor and a computing device. The computing device is configured to implement the functionality of the computing device in the first aspect or any possible implementation of the first aspect, or to implement the functionality of the computing device in the second aspect or any possible implementation of the second aspect.

[0029] In an eighth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a computer device, the computer device implements the functions of a computing device as in the first aspect or any possible implementation of the first aspect, or implements the functions of a computing device as in the second aspect or any possible implementation of the second aspect.

[0030] In the ninth aspect, a computer program product is provided. When the computer program product runs on a computer, it enables the computer to implement the functions of the computing device in the first aspect or any possible implementation of the first aspect, or to implement the functions of the computing device in the second aspect or any possible implementation of the second aspect.

[0031] The technical effects brought about by any design method from the third aspect to the ninth aspect can be referred to the technical effects brought about by the first aspect to the second aspect or different design methods, and will not be repeated here.

[0032] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] FIG1 is a schematic diagram of a dense matrix and a sparse matrix provided in this application;

[0034] FIG2 is a schematic diagram of a compressed sparse format provided by the present application;

[0035] FIG3 is a schematic structural diagram of a computer device provided by the present application;

[0036] FIG4 is a schematic diagram of the layout of a computing device provided by the present application;

[0037] FIG5 is a schematic diagram of the structure of a computing device provided by the present application;

[0038] FIG6 is a schematic diagram of calculating a vector outer product provided by this application;

[0039] FIG7 is a schematic diagram of a merging process provided by this application;

[0040] FIG8 is a schematic structural diagram of a merging circuit provided by the present application;

[0041] FIG9 is a schematic structural diagram of a sorting circuit provided by the present application;

[0042] FIG10 is a schematic structural diagram of an arrangement circuit provided by the present application;

[0043] FIG11 is a schematic diagram of the structure of a computing device provided by the present application;

[0044] FIG12 is a schematic diagram of calculating a vector inner product provided by this application;

[0045] FIG13 is a schematic diagram of the structure of a computing unit provided by the present application;

[0046] FIG14 is a schematic structural diagram of a multiplication processing circuit provided by the present application;

[0047] FIG15 is a schematic diagram of the structure of a computing device provided by the present application;

[0048] FIG16 is a flow chart of a calculation method provided by the present application;

[0049] FIG17 is a flow chart of a calculation method provided in this application. DETAILED DESCRIPTION

[0050] To facilitate understanding, the main terms involved in this application are first explained.

[0051] Scalar: Also known as a pure quantity, it refers to a quantity that has only size but no direction and can be expressed by real numbers.

[0052] Vector: Also known as a Euclidean vector or geometric vector, a vector is a quantity with both magnitude and direction. A vector can be represented as a line segment with an arrow, where the arrowhead indicates the direction of the vector and the length of the line segment indicates the magnitude of the vector. Vectors are fundamental concepts in many natural sciences, including mathematics, physics, and engineering. In theoretical mathematics, a vector is defined as any element in a vector space.

[0053] Row vector: In linear algebra, a row vector is a matrix of dimension 1×m, i.e., a matrix consisting of a row of m elements, where m is an arbitrary positive integer. A row vector is represented by the following equation (1). X = [x1 x2 ... x m Formula (1)

[0054] Column vector: In linear algebra, a column vector is a matrix of dimension m × 1, i.e., a matrix consisting of a column containing m elements, i.e., a column vector, where m is any positive integer. The transpose of a row vector is a column vector. A column vector is shown in Equation (2).

[0055] Vector inner product: Also known as dot product, scalar product, or scalar product, it is a row vector multiplied by a column vector, resulting in a scalar. The vector inner product is shown in Equation (3). a = [a1, a2, ... an] b = [b1, b2, ... bn] a·b = a1·b1 + a2·b2 + ... + an·bn Equation (3)

[0056] Vector outer product: A column vector multiplied by a row vector produces a matrix. The vector outer product is shown in equation (4). a=[a1,a2,…an]b=[b1,b2,…bn]

[0057] Matrix: A set of complex or real numbers arranged in a rectangular array. For example, an m×n matrix is ​​a rectangular array of elements arranged in m rows and n columns. Matrix A is shown in Equation (5) below, and Matrix B is shown in Equation (6) below.

[0058] Matrix addition: Matrices of the same size (same number of rows and columns) can be added or subtracted from each other. This means that elements at the same coordinates can be added or subtracted. For example, if both matrix A and matrix B are m × n matrices, the sum of matrix A and matrix B yields matrix C. Matrix C is represented by equation (7).

[0059] Matrix multiplication: When two matrices are multiplied, the number of columns in the first matrix is ​​the same as the number of rows in the second matrix. For example, if matrix A is an m×n matrix and matrix B is an n×p matrix, then the product of matrix A and matrix B is an m×p matrix. The product of matrix A and matrix B is shown in Equation (8).

[0060] Matrix inner product: A matrix is ​​decomposed into row vectors and column vectors, and the resulting matrix is ​​calculated using the vector inner product. The element at position (i, j) in the matrix resulting from the inner product of matrix A and matrix B is the result of the inner product of the i-th row vector of matrix A and the j-th column vector of matrix B. The resulting matrix of the matrix inner product is shown in Equation (9).

[0061] Matrix outer product: Decompose a matrix into row vectors and column vectors. Calculate the vector outer product to obtain multiple intermediate matrices. Add the elements with the same coordinates in these intermediate matrices to obtain the matrix of the matrix outer product of matrix A and matrix B. The matrix of the matrix outer product is shown in Equation (10).

[0062] A dense matrix is ​​one in which the proportion of nonzero elements is relatively high. A dense matrix is ​​often referred to as a matrix in an uncompressed format; it contains only the values ​​of the matrix elements, not their coordinates.

[0063] Sparse matrix: A matrix in which zero elements make up a large proportion of all elements. For example, Figure 1(a) shows a dense matrix, while Figure 1(b) shows a sparse matrix. A structured sparse matrix contains regularly distributed zero elements. An unstructured sparse matrix is ​​a matrix of arbitrary sparsity, containing randomly distributed non-zero elements.

[0064] Large sparse matrices are often encountered when solving linear models in science and engineering. When computers store and manipulate sparse matrices, standard algorithms cannot be used to calculate large sparse matrices. Therefore, standard algorithms need to be modified to take advantage of the sparse structure of matrices.

[0065] Due to the inherent sparse nature of matrices, compressing sparse matrices can save memory costs. Storing the nonzero elements of a sparse matrix in sparse matrix format, while not storing the zero elements, is the process of compressing the sparse matrix. A matrix stored in compressed format is called a compressed sparse format matrix.

[0066] Common sparse matrix formats include Coordinate (COO), Compressed Sparse Column (CSC), and Compressed Sparse Row (CSR).

[0067] Coordinate format: The sparse matrix is ​​represented based on triples. The triple contains three values, namely the row number, column number and element value. Among them, the row number is used to identify the row coordinate of the element value in the sparse matrix. The column number is used to identify the column coordinate of the element value in the sparse matrix. The row number and column number are used to identify the position of the element value in the sparse matrix. For example, the triple is represented as (row number, column number, element value), or the triple is represented as (element value, row number, column number), and the arrangement order of the three values ​​in the triple is not limited. When the sparse matrix is ​​compressed in coordinate format, the elements in the compressed sparse format matrix contain row number, column number and element value.

[0068] For example, as shown in Figure 2(a), this is the coordinate format of a sparse matrix. Sparse matrix E contains five non-zero elements: 2, 5, 9, 1, and 6. The element value 2 is located in the first row and first column of sparse matrix E, and the corresponding triplet is (1, 1, 2). The element value 5 is located in the third row and fourth column of sparse matrix E, and the corresponding triplet is (3, 4, 5).

[0069] Compressed sparse column format: Sparse matrices are represented based on three types of data: element values, indices, and column offsets. The element values ​​and indices in the compressed sparse column format are similar to the element values ​​and row numbers in the coordinate format described above. The index is used to indicate the row coordinate or row number of the element value in the sparse matrix. The column offset value indicates the offset of the first non-zero element of the column in the sparse matrix in the element value, and the last element indicates the number of non-zero elements in the sparse matrix. The number of column offset values ​​is the number of columns plus 1. When a sparse matrix is ​​compressed in the compressed sparse column format, the elements in the compressed sparse format matrix contain element values, row numbers, and column offsets.

[0070] For example, as shown in (b) of Figure 2, it is a compressed sparse column format of a sparse matrix. Among them, the sparse matrix D contains 7 non-zero elements. The element values ​​include the non-zero elements 8, 2, 5, 7, 1, 9, and 2 arranged in columns in the sparse matrix D. For example, the row number corresponding to the element value 8 and the element value 2 is 0. The row number corresponding to the element value 5 is 1. The non-zero element 8 in the first column of the sparse matrix D has an offset of 0 in the element value. The non-zero element 2 in the third column of the sparse matrix D has an offset of 1 in the element value. The non-zero element 1 in the fourth column of the sparse matrix D has an offset of 4 in the element value. The non-zero element 2 in the fifth column of the sparse matrix D has an offset of 6 in the element value.

[0071] Compressed sparse row format: Sparse matrices are represented based on three types of data: element values, indices, and row offsets. The element values ​​and column numbers in the compressed sparse row format are similar to those in the coordinate format described above. The index indicates the column coordinate or column number of the element value in the sparse matrix. The row offset indicates the offset of the first non-zero element in the row of the sparse matrix in the element value, and the last element indicates the number of non-zero elements in the matrix. The number of row offsets is the number of rows plus 1. When a sparse matrix is ​​compressed using the compressed sparse row format, the elements in the compressed sparse format matrix contain element values, column numbers, and row offsets.

[0072] For example, as shown in (c) in Figure 2, it is a compressed sparse row format of a sparse matrix. Among them, the sparse matrix D contains 7 non-zero elements. The element values ​​include the non-zero elements 8, 2, 5, 7, 1, 2, and 9 arranged in rows in the sparse matrix D. For example, the column number corresponding to the element value 8 is 0. The column number corresponding to the element value 2 is 2. The offset of the non-zero element 8 in the first row of the sparse matrix D in the element value is 0. The offset of the non-zero element 5 in the second row of the sparse matrix D in the element value is 2. The offset of the non-zero element 7 in the fifth row of the sparse matrix D in the element value is 3. The offset of the non-zero element 9 in the seventh row of the sparse matrix D in the element value is 6.

[0073] Multiply Accumulate (MAC): A special operation in digital signal processors (DSPs) and some microprocessors. The hardware circuit unit that implements this operation is called a multiply-accumulate (MAC). A MAC operation involves adding the product of the multiplier to the value stored in the accumulator and storing the result in the accumulator.

[0074] Arithmetic Intensity (AI): This refers to the number of operations per byte. For example, arithmetic intensity is the ratio of work (W) to memory traffic (Q), or AI = W / Q, where AI represents the number of operations per byte of memory traffic. When work (W) is expressed in FLOPs, the resulting arithmetic intensity (AI) is the ratio of floating-point operations to total data movement (FLOPs / Byte). Higher arithmetic intensity indicates higher data reuse.

[0075] Single instruction multiple data (SIMD) is a technology that achieves spatial parallelism by performing the same operation on each piece of data in a set of data (also known as a "data vector"). SIMD is an extension of the basic instruction set of the central processing unit (CPU).

[0076] Scalable Vector Extension (SVE) is a SIMD instruction set for the ARM AArch64 architecture, designed to accelerate high-performance computing. It supports variable vector lengths, per-lane prediction, gather-load and scatter-store operations, and horizontal operations. SVE differs from SIMD in that SIMD processes fixed-length data, while SVE processes variable-length data.

[0077] Scalable Matrix Extension (SME) is an instruction set based on the Scalable Vector Extension (SVE and SVE2) under the ARMv9-A architecture that supports matrix calculations. It can be understood that the processor provides a new mode, that is, the processor provides matrix calculation capabilities, making it easier for the processor to use different vector sizes when processing matrix calculations, as well as SVE or SIMD calculations. SME defines an architecture capable of storing two-dimensional matrices and a series of instructions related to matrix calculations, including: ① Instructions for calculating the outer product of two vectors to implement matrix multiplication; ② Instructions for reading, storing, and moving vectors from a matrix. SME supports vector lengths ranging from 128 to 2048 bits.

[0078] In order to solve the problem of low computational efficiency of sparse matrices, the present application provides a computing device and method, that is, the elements in the compressed sparse format matrix include element values ​​and compression identifiers, and since the compression identifier indicates the row coordinates or column coordinates of the element value in the sparse matrix, the calculation of the compressed sparse format matrix (such as matrix inner product, matrix outer product) is performed based on the compression identifier corresponding to the element value, and the matrix inner product result or matrix outer product result of the compressed sparse format matrix is ​​output, that is, the vector outer product calculation and the vector inner product calculation are performed based on the compression identifier in the compressed sparse format matrix to realize the calculation of the sparse matrix, thereby utilizing the large computing power of the computing unit that supports matrix calculation to efficiently calculate the compressed sparse format matrix, so that the computing unit expands the matrix calculation that supports unstructured sparse matrices, and improves the computational efficiency of the sparse matrix. In addition, a merging circuit is set in the computing device to realize the matrix accumulation of the intermediate results of the matrix outer product in the computing device, and a register is set in the computing device to cache the intermediate results of the compressed sparse format matrix, reducing the storage space and access bandwidth required for the sparse matrix calculation, and further improving the computational efficiency of the sparse matrix.

[0079] The calculation method provided in this application can cover more sparse matrix calculation scenarios, such as unstructured sparse pruning technology (unstructured pruning), graph computing (graph), graph neural network (GNN), scientific computing, multi-grid solution and other typical sparse matrix multiplication and addition calculations.

[0080] The following describes in detail the implementation of the calculation method and device provided by this application with reference to the accompanying drawings.

[0081] FIG3 is a schematic diagram of the structure of a computer device provided in this application. The computer device can be a device with computing capabilities, such as a server. For example, as shown in FIG3 , the computer device 300 includes a processor 310, a bus 320, a memory 330, a communication interface 340, and a memory unit 350 (also referred to as a main memory unit). The processor 310, the memory 330, the memory unit 350, and the communication interface 340 are connected via a bus 320.

[0082] The processor 310 is the control center of the computer device 300. The processor 310 can be a computing unit with computing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), or a neural processing unit (NPU). The computer device 300 may include multiple processors. The processor here can refer to one or more devices, circuits, and / or computing units with computing capabilities for processing data (such as computer program instructions). The processor 310 includes one processor core or multiple processor cores.

[0083] Among them, the instructions that the processor core can process include SIMD instructions, SVE instructions, and SME instructions. The processor core uses different data types and performs different calculations when processing different instructions. For example, when the processor processes SIMD instructions, it processes fixed-length data (such as integer data, floating-point data) and performs addition, subtraction, or other calculations on the data. For another example, when the processor processes SVE instructions, it processes variable-length data (such as integer data, floating-point data) and performs addition, subtraction, or other calculations on the data. For another example, when the processor processes SME instructions, it performs matrix calculations on matrices (such as matrix inner product, matrix outer product).

[0084] In this embodiment, the processor 310 may also be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0085] The memory 330 can be used to store data and applications, and mainly includes a program storage area and a data storage area. The program storage area can store an operating system, software applications required for at least one function, and intermediate layer software. The data storage area can store data created when the device is in use. For example, the operating system can include a Linux operating system, a Unix operating system, or a Windows operating system. The software applications required for the at least one function can include applications related to artificial intelligence (AI), high performance computing (HPC), deep learning, or scientific computing, and the intermediate layer software can include linear algebra library functions. In one possible exemplary embodiment, the memory 330 includes, but is not limited to, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), or high-speed random access memory. Furthermore, the memory 330 can also include other non-volatile memories, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0086] The communication interface 340 is used to enable communication between the computer device 300 and an external device or device. Among them, the communication interface 340 may include an input interface and an output interface. The input interface can be used to obtain vectors (such as row vectors or column vectors) in the compressed sparse format matrix in the following embodiments. The output interface can be used to output the calculation results in the following embodiments. In some embodiments, the calculation results of the compressed sparse format matrix can be directly output by the processor or can be first stored in a memory. The computer device may include one or more input interfaces and one or more output interfaces.

[0087] The memory unit 350 may be a volatile memory pool or a non-volatile memory pool, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronized DRAM (SLDRAM), and direct RAM bus RAM (DR RAM). Memory unit 350 is used to store matrix calculation results (e.g., matrix outer product results, matrix inner product results) of compressed sparse format matrices.

[0088] The bus 320 may include a path for transmitting information between the above-mentioned components (such as the processor 310, the memory unit 350, and the memory 330). In addition to the data bus, the bus 320 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 320 in the figure. The bus 320 may be a Peripheral Component Interconnect Express (PCIe) bus, an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a compute express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 320 can be divided into an address bus, a data bus, a control bus, etc.

[0089] The connection between the processor core and other components in the processor in Figure 3 is merely illustrative. In one possible implementation, the processor core is connected to other components in the processor via a ring bus, and accesses other components in the processor via the ring bus. In another possible implementation, the processor core is connected to other components in the processor via a mesh bus, and the processor core accesses other components in the processor via the mesh bus.

[0090] It is worth noting that Figure 3 illustrates computer device 300 as including one processor 310 and one memory 330. Here, processor 310 and memory 330 are each used to indicate a type of device or equipment. In specific embodiments, the number of each type of device or equipment can be determined based on business requirements. The device structure shown in Figure 3 does not limit the computer device and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0091] The computer device 300 may include a computing device configured to provide a matrix inner product or matrix outer product computing function. The computing device may be disposed inside or outside the processor, without limitation.

[0092] For example, as shown in FIG4(a), the computing device 313 is integrated into the processor 310. As shown in FIG4(b), the computing device 313 is provided outside the processor 310, for example, the computing device is provided in the form of a PCIe card. The processor 310 may also include a cache memory 311, a register 312, a controller 314, one or more arithmetic logic units (ALUs) 315, and a memory management unit (MMU) 316.

[0093] The calculation device 313 is used to calculate the matrix inner product or matrix outer product of the sparse matrix based on the compression identifier contained in the elements of the compressed sparse format matrix. For a specific explanation of the matrix inner product or matrix outer product calculation of the sparse matrix, please refer to the detailed description of the following embodiment.

[0094] The cache memory 311 is used to store instructions or data that may be accessed multiple times by the processor core in the processor 310 , thereby increasing the speed at which the processor processes data and preventing the processor from frequently accessing the memory unit 350 .

[0095] In terms of physical form, the cache memory 311 can be a random access memory (RAM), a static random-access memory (SRAM), a dynamic random access memory (DRAM), or other types of storage devices that can store information and instructions.

[0096] In terms of logical form, the cache memory 311 can be a level 1 cache (L1 cache), a level 2 cache (L2 cache), a level 3 cache (L3 cache), or any other level of cache. For example, the cache memory 311 disposed inside the processor core can be a level 1 cache (L1 cache) or a level 2 cache (L2 cache). The cache memory 311 disposed outside the processor core can be a level 3 cache (L3 cache).

[0097] Register 312 is used to store instructions or data that may be accessed multiple times by the processor core in processor 310. Because register access speed is higher than cache memory access speed, instructions or data that may be accessed multiple times by the processor core can be stored in registers first, further improving the processor's data processing speed.

[0098] In the present application, when a computing device calculates the matrix outer product of two compressed sparse format matrices, it first obtains N column vectors of the first compressed sparse format matrix and N row vectors of the second compressed sparse format matrix, calculates the vector outer product of the N column vectors and the N row vectors, and obtains N intermediate result matrices. Among them, the column vector includes the first element value and the compression identifier of the first element value, and the compression identifier of the first element value is used to indicate the row coordinate of the first element value in the first sparse matrix. The row vector includes the second element value and the compression identifier of the second element value, and the compression identifier of the second element value is used to indicate the column coordinate of the second element value in the second sparse matrix. The first compressed sparse format matrix is ​​the compressed sparse format matrix of the first sparse matrix. The second compressed sparse format matrix is ​​the compressed sparse format matrix of the second sparse matrix. The intermediate result matrix includes the third element value and the compression identifier of the third element value, and the compression identifier includes the compression identifier of the first element value and the compression identifier of the second element value.

[0099] Furthermore, the computing device accumulates the third element values ​​having the same compression identifier of the third element values ​​in multiple intermediate result matrices according to the compression identifier of the third element value to obtain a matrix outer product result.

[0100] It should be noted that all matrices involved in the matrix outer product calculation process described in this application are compressed sparse format matrices. For example, the matrix for calculating the matrix outer product, the intermediate result matrix, the intermediate merge matrix, and the matrix outer product result are all compressed sparse format matrices.

[0101] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculations on the decompressed matrix, or performing matrix calculations on a compressed sparse format matrix based on the position coordinates of the compressed sparse format matrix, which results in a low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a merging circuit. By utilizing the large computing power of the computing unit that supports matrix calculations, the matrix outer product of the compressed sparse format matrix is ​​efficiently calculated based on the compression identifier of the compressed sparse format matrix, so that the computing unit is expanded to support matrix calculations of unstructured sparse matrices. The merging circuit is used to realize the accumulation of intermediate result matrices of the matrix outer product within the computing device, effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate result matrices of the compressed sparse format matrix are cached within the computing device, reducing the storage space and access bandwidth required for sparse matrix calculations, further improving the computational efficiency of the compressed sparse format matrix.

[0102] FIG5 is a schematic diagram of the structure of a computing device provided by the present application. As shown in FIG5 , the computing device 500 includes a computing unit 510 , a first cache unit 520 , and a merging circuit 530 .

[0103] The computing unit 510 may be an SME unit in an ARM processor. The input to the computing unit 510 includes column vectors in a first matrix and row vectors in a second matrix. Both the first matrix and the second matrix are compressed sparse format matrices. Elements in the compressed sparse format matrices include element values, compression indices, and predicates.

[0104] The compression identifier is used to indicate the column coordinates or row coordinates of an element value in a sparse matrix. For example, if the first matrix is ​​a compressed sparse column format matrix, the compression identifier is used to indicate the row coordinates of the element value in the first sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. For another example, if the second matrix is ​​a compressed sparse row format matrix, the compression identifier is used to indicate the column coordinates of the element value in the second sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix.

[0105] The valid value indicates whether the element value is valid. For example, when the valid value is 1, it indicates that the element value is valid; when the valid value is 0, it indicates that the element value is invalid.

[0106] Optionally, the first matrix may be a submatrix of a first compressed sparse format matrix. The second matrix may be a submatrix of a second compressed sparse format matrix. The sizes of the first matrix and the second matrix may be determined by the size of a cache unit in the system. The following description uses the example of N×N matrices, where both the first matrix and the second matrix are N×N matrices.

[0107] The calculation unit 510 is used to calculate the vector outer product of each column vector of the N column vectors of the first matrix and the N row vectors of the second matrix to obtain multiple intermediate result matrices.

[0108] The calculation unit 510 is specifically configured to calculate the matrix outer product of the first matrix and the second matrix based on the compression identifiers of the column vectors in the first matrix and the compression identifiers of the row vectors in the second matrix to obtain a plurality of intermediate result matrices. The intermediate result matrices are all compressed sparse format matrices.

[0109] The calculation unit 510 obtains a column vector of the first matrix and a row vector of the second matrix each time, and calculates the element values, compression flags, and effective values ​​in the column vector and row vector respectively to obtain an intermediate result matrix. The intermediate result matrix includes an element value product matrix, a compression flag matrix, and an effective value matrix.

[0110] For example, as shown in FIG6 , it is assumed that the calculation unit 510 calculates the vector outer product of the column vector A and the row vector B.

[0111] The calculation unit 510 is used to sequentially multiply each element value (A_value) of the column vector A with the element value (B_value) of the row vector B to obtain an element value product matrix (C_value). The element value product matrix can be a compressed sparse coordinate format matrix.

[0112] The calculation unit 510 is used to combine the row coordinates (A_index) of the element values ​​of the column vector A and the column coordinates (B_index) of the element values ​​of the row vector B in sequence to obtain a compressed identification matrix (C_index) corresponding to the element value product matrix, where the compressed identification matrix includes coordinates corresponding to the product of the element values ​​in the product matrix (coordinates in coordinate format).

[0113] The calculation unit 510 is configured to ensure that the value at the corresponding position in the effective value matrix (C_predicate) is valid only when the effective values ​​(A_predicate, B_predicate) corresponding to the element values ​​of the column vector A and the row vector B are both valid.

[0114] The first cache unit 520 is configured to store an intermediate result matrix. Optionally, the first cache unit 520 is also configured to store an intermediate merge matrix. Optionally, the computing device 500 may further include a selector 540 that selects whether to cache the intermediate result matrix output by the computing unit 510 in the first cache unit 520 or to cache the intermediate merge matrix in the memory in the first cache unit 520.

[0115] The merging circuit 530 is used to accumulate the values ​​of elements with the same compression identifier in multiple intermediate result matrices to obtain a matrix outer product result.

[0116] Optionally, the intermediate result matrix stored in the first cache unit 520 can be stored in a memory for subsequent merging processing, or the intermediate result matrix stored in the first cache unit 520 can be stored in a merging circuit 530. The merging circuit 530 includes a register whose size is larger than the size of the first cache unit 520. The merging circuit 530 is used to store the merging result.

[0117] Optionally, when the remaining capacity of the merging circuit 530 is less than a threshold, that is, there is no remaining storage space in the merging circuit 530 to store the merging result, the merging result can be stored in a memory, which includes multiple ordered data blocks whose size does not exceed the size of the merging circuit 530.

[0118] In some embodiments, as shown in FIG7 , the merge circuit 530 performs an intra-chip merge process and a memory-to-internal-chip merge process. The intra-chip merge process includes the merge circuit 530 merging the intermediate result matrix output by the first cache unit 520 with the intermediate result matrix stored by the merge circuit 530, and / or the merge circuit 530 merging the intermediate result matrix output by the first cache unit 520 with the intermediate merged matrix stored by the merge circuit 530 to obtain an updated intermediate merged matrix. The memory-to-internal-chip merge process includes merging the intermediate merged matrix read from the memory.

[0119] It is understood that when the merging circuit 530 performs an intra-chip merging process, the input of the merging circuit 530 includes the intermediate result matrix output by the first cache unit 520 and at least one of the intermediate result matrix and the intermediate merging matrix stored by the merging circuit 530. When the merging circuit 530 performs a memory-to-intra-chip merging process, the input of the merging circuit 530 includes the intermediate merging matrix stored in the memory. The intermediate merging matrix stored in the memory can be the intermediate merging matrix output by the merging circuit 530.

[0120] Merging circuit 530 is specifically configured to accumulate element values ​​with the same compression identifier in multiple intermediate result matrices to obtain multiple intermediate merged matrices, and accumulate element values ​​with the same compression identifier in multiple intermediate merged matrices to obtain a matrix outer product result. It is understood that the elements in the intermediate result matrices and the intermediate merged matrices include element values ​​and compression identifiers. The intermediate result matrices and the intermediate merged matrices are compressed sparse format matrices. The compression identifier can be a coordinate in a compressed sparse coordinate format matrix, comprising a row coordinate or a column coordinate. The same compression identifier in multiple intermediate result matrices can indicate that the row coordinates and column coordinates included in the compression identifier are the same. The same compression identifier in multiple intermediate merged matrices can indicate that the row coordinates and column coordinates included in the compression identifier are the same. The size of the intermediate merged matrix can be equal to or greater than the size of the intermediate result matrix. For example, if the multiple intermediate result matrices have the same size and contain the same compression identifier, the size of the intermediate merged matrix is ​​equal to the size of the intermediate result matrix. For another example, if the multiple intermediate result matrices have the same size and contain at least one compression identifier that is different from all other compression identifiers, the size of the intermediate merged matrix is ​​greater than the size of the intermediate result matrix. When the merging circuit 530 performs the memory-to-on-chip merging process, the merged intermediate merging matrix obtained by merging multiple intermediate merging matrices may be equal to or larger than the intermediate merging matrix.

[0121] In some embodiments, as shown in FIG. 8 , the merging circuit 530 includes a sorting unit 531 , an addition processing unit 532 , an arrangement unit 533 , a boundary removal unit 534 , and a second cache unit 535 .

[0122] The sorting unit 531 is used to sort the elements in the multiple intermediate result matrices according to the order of compression identifiers in the intermediate result matrices from small to large when merging multiple intermediate result matrices.

[0123] Optionally, the sorting unit 531 is further configured to sort the elements in the plurality of intermediate merge matrices according to the order of compression identifiers in the intermediate merge matrices from small to large when merging the intermediate merge matrices.

[0124] Optionally, the sorting unit 531 is further configured to sort the elements in the intermediate result matrix and the intermediate merged matrix according to the order of the compression flags in the intermediate result matrix and the compression flags in the intermediate merged matrix from small to large when merging the intermediate result matrix and the intermediate merged matrix.

[0125] In some embodiments, the sorting unit can be implemented using a merge sort method. Two vectors containing N elements are connected end to end to form a vector of length 2N. In the first round, the elements 0 to N-1 are compared with the elements N to 2N-1 respectively. If the valid value of the lower element is 0 and the valid value of the higher element is 1, or the valid values ​​of both elements are 1 but the lower compression flag is greater than the higher compression flag, the positions of the two elements are swapped; in the second round, the elements N / 2 to N-1 are compared with the elements N to 3N / 2-1 respectively, and the swap principle is the same as the first round; in the third round, the elements N / 4 to N / 2-1, 3N / 4 to N-1, and 5N / 4 to 3N / 2-1 are compared with the elements N / 4 positions apart, and the swap principle is the same as the first round; after completing multiple rounds of calculations, a set of vectors with ordered compression flags can be obtained, and elements with the same compression flags are in adjacent positions.

[0126] Figure 9 shows a schematic diagram of an 8+8 sorting circuit design provided by this application. The two connected black dots represent a comparator. After four rounds of comparison, the elements in the two vectors are sorted in ascending order based on the compression identifiers.

[0127] The addition processing unit 532 is configured to accumulate element values ​​with the same compression identifier in the sorted elements.

[0128] In some embodiments, the addition processing unit 532 may include N adders and N selectors.

[0129] A selector is used to determine a value of an input adder according to elements adjacent to an nth element and compression identifiers of elements adjacent to the nth element, where the elements adjacent to the nth element include an n+1th element and an n-1th element.

[0130] An adder is configured to perform a cumulative calculation on the value of the nth element and the value output by the selector. Optionally, when the value output by the selector is 0, indicating that there is no compression identifier that is the same as the compression identifier of the nth element, that is, the compression identifier of the nth element is different from the compression identifier of the (n+1)th element and the compression identifier of the (n-1)th element, and the adder directly outputs the element value of the nth element.

[0131] The arrangement unit 533 is configured to arrange the elements indicated as valid by the valid values ​​according to the valid values ​​contained in the elements in the intermediate merge matrix, thereby reducing the storage capacity occupied by the process of merging the matrices.

[0132] In some embodiments, the arrangement unit 533 is used to shift valid elements to lower bits based on their valid values, and set the remaining elements to 0. For example, FIG10 shows a schematic diagram of a compact arrangement design for 16 elements. Valid elements are sorted in ascending order based on compression identifiers, and invalid elements are set to 0. To facilitate timing optimization, multi-level (e.g., two-level) compaction is implemented, and one level can be used if the timing meets the requirements.

[0133] A de-bounding unit 534 is configured to remove, from the accumulated vector of two vectors, elements corresponding to compression identifiers other than the minimum compression identifier of the maximum compression identifier of the two vectors. The two vectors are other than the last two vectors in the matrix to be merged, and the matrix to be merged includes either an intermediate merge matrix or an intermediate result matrix. When the last two vectors in the matrix to be merged are accumulated, the compression identifiers of both vectors are retained.

[0134] For example, if matrix 1 and matrix 2 to be merged are 2×2 matrices, matrix 1 and matrix 2 to be merged are merged row by row. The first row vector of matrix 1 contains two elements with compression identifiers 1 and 2, and the second row vector of matrix 1 contains two elements with compression identifiers 3 and 4. The first row vector of matrix 2 contains two elements with compression identifiers 1 and 3, and the second row vector of matrix 2 contains two elements with compression identifiers 5 and 6.

[0135] Merge the first row vector of matrix 1 to be merged and the first row vector of matrix 2 to be merged, accumulate the element value corresponding to compression identifier 2 in the first row vector of matrix 1 to be merged and the element value corresponding to compression identifier 2 in the first row vector of matrix 2 to be merged, remove the element value corresponding to compression identifier 3 in the first row vector of matrix 2 to be merged, and retain the element value corresponding to compression identifier 1 in the first row vector of matrix 1 to be merged.

[0136] Merge the second row vector of the matrix to be merged 1 and the first row vector of the matrix to be merged 2. Since the element value corresponding to the compression identifier 1 in the first row vector of the matrix to be merged 2 and the compression identifier 1 in the first row vector of the matrix to be merged are already merged, there is no need to merge the compression identifier 1 in the first row vector of the matrix to be merged 1. Accumulate the element value corresponding to the compression identifier 3 in the second row vector of the matrix to be merged 1 and the element value corresponding to the compression identifier 3 in the first row vector of the matrix to be merged 2, and remove the element value corresponding to the compression identifier 4 in the second row vector of the matrix to be merged 1.

[0137] Merge the second row vector of the matrix to be merged 1 and the second row vector of the matrix to be merged 2. Since the element value corresponding to the compression identifier 3 in the second row vector of the matrix to be merged 1 and the compression identifier 3 in the first row vector of the matrix to be merged 2 have already been merged, there is no need to merge the compression identifier 3 in the second row vector of the matrix to be merged 1. The element value corresponding to the compression identifier 4 in the second row vector of the matrix to be merged 1 is retained, and the element values ​​corresponding to the compression identifiers 5 and 6 in the second row vector of the matrix to be merged 2 are retained.

[0138] It should be understood that the elements contained in the intermediate merge matrix stored in the memory are ordered, and the merging process is also carried out in sequence. When two vectors are merged, the minimum value IDX of the two maximum compression identifiers (IDX=Min(AindexA(max), BindexB(max)))) is removed from the boundary and can be directly stored in the memory. Because the compression identifiers of the subsequently merged elements are all greater than IDX, the size of the merged matrix is ​​not limited by the size of the cache unit in the merging circuit.

[0139] The second cache unit 535 is used to store the intermediate merge matrix.

[0140] It should be noted that the merging circuit may not include the sorting unit and the permutation unit. The merging circuit 530 compares the compression identifiers in the intermediate result matrix one by one, and accumulates the values ​​of the elements with the same compression identifier in the intermediate result matrix.

[0141] In the present application, when a computing device calculates the matrix inner product of two compressed sparse format matrices, it first obtains N row vectors of the first compressed sparse format matrix and N column vectors of the second compressed sparse format matrix, calculates the vector inner product of the N row vectors and the N column vectors, and obtains N intermediate results. Among them, the row vector contains the first element value and the compression identifier of the first element value, and the compression identifier of the first element value is used to indicate the column coordinate of the first element value in the first sparse matrix. The column vector contains the second element value and the compression identifier of the second element value, and the compression identifier of the second element value is used to indicate the row coordinate of the second element value in the second sparse matrix. The first compressed sparse format matrix is ​​the compressed sparse format matrix of the first sparse matrix. The second compressed sparse format matrix is ​​the compressed sparse format matrix of the second sparse matrix.

[0142] Furthermore, the computing device accumulates the element values ​​of the same position coordinates in the plurality of intermediate results to obtain a matrix inner product result.

[0143] It should be noted that all matrices involved in the matrix inner product calculation process described in this application are compressed sparse format matrices. For example, the matrix used to calculate the matrix outer product and the matrix inner product result are both compressed sparse format matrices.

[0144] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculations on the decompressed matrix, or performing matrix calculations on a compressed sparse format matrix based on the position coordinates of the compressed sparse format matrix, which results in a low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a register, which utilizes the large computing power of the computing unit that supports matrix calculations, and efficiently calculates the matrix inner product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is expanded to support matrix calculations of unstructured sparse matrices, and the computing unit is used to realize the accumulation of intermediate results of the matrix inner product within the computing device, thereby effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate results of the compressed sparse format matrix are cached within the computing device, reducing the storage space and access bandwidth required for sparse matrix calculations, further improving the computational efficiency of the compressed sparse format matrix.

[0145] 11 is a schematic diagram of the structure of a computing device provided by the present application. The computing device 1100 includes a computing unit 1110 and a register 1120.

[0146] The input of the calculation unit 1110 includes the row vectors in the first matrix and the column vectors in the second matrix. The first matrix and the second matrix are both compressed sparse format matrices. The elements in the compressed sparse format matrix include element values, compression flags, and valid values.

[0147] The compression identifier is used to indicate the column coordinates or row coordinates of an element value in a sparse matrix. For example, if the first matrix is ​​a compressed sparse row format matrix, the compression identifier is used to indicate the column coordinates of the element value in the first sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. For another example, if the second matrix is ​​a compressed sparse column format matrix, the compression identifier is used to indicate the row coordinates of the element value in the second sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix.

[0148] Optionally, the first matrix may be a submatrix of a first compressed sparse format matrix. The second matrix may be a submatrix of a second compressed sparse format matrix. The sizes of the first matrix and the second matrix may be determined by the size of a cache unit in the system. The following description uses the example of N×N matrices, where both the first matrix and the second matrix are N×N matrices.

[0149] The calculation unit 1110 is used to calculate the vector inner product of each of the N row vectors of the first matrix and the N column vectors of the second matrix to obtain multiple intermediate results.

[0150] The calculation unit 1110 is used to calculate the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results.

[0151] Computing unit 1110 is specifically configured to perform a vector inner product calculation on element values ​​corresponding to the same compression identifier in a row vector and a column vector to obtain an intermediate result. The compression identifier of the row vector is used to indicate the column coordinates of the element values ​​in the row vector in the first sparse matrix. The compression identifier of the column vector is used to indicate the row coordinates of the element values ​​in the column vector in the second sparse matrix.

[0152] Register 1120 is used to store multiple intermediate results and matrix inner product results. Optionally, when the matrix inner product calculation is completed, the intermediate results stored in register 1120 are transferred to the memory.

[0153] Understandably, the intermediate result may be an element in the matrix stored in register 1120, the element containing the element value. The element value is the result of the vector inner product of the row vector and the column vector, that is, the result of multiplying and accumulating the element values ​​with the same compression identifier in the row vector and the column vector.

[0154] The calculation unit 1110 is further configured to accumulate the values ​​of elements at the same position coordinates in multiple intermediate results to obtain a matrix inner product result.

[0155] Computing unit 1110 is specifically configured to add the first intermediate result output by the computing unit to the value indicated by the same position coordinate in the register to obtain a second intermediate result. It is understood that the first intermediate result output by computing unit 1110 is added to the value at the position in register 1120 where the first intermediate result should be stored, and the result of the addition is stored in the corresponding position in register 1120.

[0156] Registers in the computing device cache the matrix inner product results of small matrix blocks, allowing the matrix inner product results of small matrices to be directly accumulated within the computing unit. As shown in Figure 12, assume that the first compressed sparse format matrix includes sub-matrices A00-A03 and sub-matrices A10-A12, and the second compressed sparse format matrix includes sub-matrices B00-B20 and sub-matrices B01-B21. Matrix C00 is the result of the cross-multiplication and accumulation of A00-A03 and B00-B20. After the matrix inner product of the first compressed sparse format matrix and the second compressed sparse format matrix is ​​calculated, the C00 block of matrix C is stored in memory, and then C01, C10, and C11 are calculated in sequence. Thus, the register caches intermediate results rather than directly outputting scalars, avoiding the need to store intermediate results in memory and reducing memory access overhead during the process of downloading them from memory to the computing unit for accumulation.

[0157] 13 , computing unit 1110 includes a sorting circuit 1111, a multiplication processing circuit 1112, and an aggregation circuit 1113. Optionally, the multipliers and adders used by multiplication processing circuit 1112 and aggregation circuit 1113 may reuse the multipliers and adders of a computing unit for computing dense matrices.

[0158] Sorting circuit 1111 is configured to sort the values ​​of row and column vector elements based on the compression identifiers of the row and column vectors. Specifically, the elements of row vector A and column vector B input to computing unit 1110 are sorted in ascending order of compression identifiers, resulting in elements sorted by compression identifiers. A maximum of two elements may have the same compression identifier, and these elements must be adjacent to each other after the sorting. For an explanation of the sorting process, refer to the description of sorting unit 331 above.

[0159] Optionally, the computing unit 1110 further includes a first cache unit 1114 . The sorting circuit 1111 stores the sorted elements in the first cache unit 1114 .

[0160] Multiplication processing circuit 1112 is configured to multiply element values ​​of the sorted elements with the same compression identifier to obtain multiple calculation results, where each element includes an element value and a compression identifier. Specifically, the compression identifier of each element is compared with the compression identifier of the next element. If the compression identifiers of the two elements are the same, the element values ​​of the two elements are multiplied, the element value of the current element is modified to the calculation result of the multiplication of the element values ​​of the two elements, and the element value of the next element is set to 0. If the compression identifiers of the two elements are different, the element value of the current element is modified to 0. After processing, the calculation result of the multiplication of the element values ​​with the same compression identifier is obtained.

[0161] Since elements with the same compression identifier are adjacent after sorting, and each element has the same compression identifier as at most one other element, if both the row vector and the column vector contain N elements, N multipliers are required to implement the multiplication processing of the element values ​​with the same compression identifier after sorting. In some embodiments, the multiplication processing circuit 1112 includes N multipliers and N selectors, with each multiplier connected to one selector.

[0162] The i-th selector is configured to determine the value of the input multiplier based on the compressed identifiers of the elements adjacent to the n-th element and the elements adjacent to the n-th element, where the elements adjacent to the n-th element include the n+1-th element and the n-1-th element. The value of i ranges from 1 to N, and the value of n ranges from 1 to 2N, whichever is odd or even.

[0163] The i-th multiplier is used to perform a multiplication calculation on the n-th element value and the value output by the selector.

[0164] Optionally, the selector can determine the input of the multiplier based on the compression flag same flag (index same flag). Index same flag [n] represents the indicator bit that the compression flags of the nth element and the n+1th element are the same. If index same flag [n] is 1, indicating that the compression flag of the nth element and the compression flag of the n+1th element are the same, the multiplier selects the element value of the n+1th element as input; if index same flag [n] is 0, indicating that the compression flag of the nth element and the compression flag of the n+1th element are different, the multiplier selects the element value of the n-1th element as input; if the compression flag of the nth element is not equal to the compression flag of the n-1th element and the compression flag of the n+1th element, the multiplier input is 0.

[0165] Optionally, a controller in the computing system (such as the controller 314 shown in FIG4 ) can determine that the compression identifier of the nth element is the same as the compression identifier of the n+1th element, and that the compression identifier of the nth element is the same as the compression identifier of the n-1th element.

[0166] For example, as shown in FIG14 , the multiplication processing circuit 1112 includes eight multipliers and eight selectors. The multiplication processing circuit 1112 multiplies the values ​​of the elements with the same compression identifier among the 16 elements after sorting to obtain multiple calculation results. The i-th multiplier performs a multiplication calculation on the value of the n-th element and the value output by the i-th selector.

[0167] For example, the first multiplier obtains the element value of the second element. When identifier 0 is 1, indicating that the compression identifier of the second element is the same as the compression identifier of the third element, the first selector outputs the element value of the third element, and the first multiplier performs a multiplication calculation on the second element value and the element value of the third element output by the first selector. When identifier 1 is 1, indicating that the compression identifier of the second element is the same as the compression identifier of the first element, the first selector outputs the element value of the first element, and the first multiplier performs a multiplication calculation on the second element value and the element value of the first element output by the first selector. When identifier 1 and identifier 0 are both 0, indicating that the compression identifier of the second element is different from the compression identifier of the first element and the compression identifier of the third element, the first selector outputs 0, and the first multiplier performs a multiplication calculation on the second element value and the 0 output by the first selector.

[0168] Optionally, the calculation unit 1110 further includes a second cache unit 1115 . The sorting circuit 1111 stores the calculation results of the element values ​​corresponding to the same compression identifier in the second cache unit 1115 .

[0169] The aggregation circuit 1113 is used to accumulate multiple calculation results and values ​​in the register determined based on the same position coordinates.

[0170] In some embodiments, aggregation circuit 1113 includes N adders. The N adders are configured to accumulate the output results of the N multipliers and the value indicated by the same position coordinate in the register. For example, the first adder accumulates the calculation results of the first and second multipliers. The second adder accumulates the calculation results of the third and fourth multipliers. The third adder accumulates the calculation results of the fifth and sixth multipliers. The fourth adder accumulates the calculation results of the seventh and eighth multipliers. The fifth adder accumulates the output results of the first and second adders. The sixth adder accumulates the output results of the third and fourth adders. The seventh adder accumulates the output results of the fifth and sixth adders. The eighth adder accumulates the result of the seventh adder and the result indicated by the same position coordinate in the register.

[0171] In some embodiments, as shown in Figure 15, the computing device may further include multiple registers and multiple computing units. The multiple computing units process the vector inner product of the row vectors in the first matrix and the column vectors in the second matrix in parallel to implement the vector inner product of the first matrix and the second matrix and obtain the vector inner product result.

[0172] Figure 16 is a flow chart of a calculation method provided by the present application. The method is executed by the calculation device for calculating the matrix outer product of the above-mentioned embodiment. The calculation device includes a calculation unit and a merging circuit. The method includes: the calculation unit calculates the matrix outer product of the first matrix and the second matrix based on the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices (step 1601). The merging circuit accumulates the element values ​​with the same compression identifier in the multiple intermediate result matrices to obtain the matrix outer product result (step 1602).

[0173] Figure 17 is a flow chart of a calculation method provided by the present application. The method is executed by the calculation device for calculating the matrix inner product of the above embodiment. The calculation device includes a calculation unit and a register. The method includes: the calculation unit calculates the matrix inner product of the first matrix and the second matrix based on the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results (step 1701). The calculation unit accumulates the element values ​​of the same position coordinates in the multiple intermediate results to obtain the matrix inner product result (step 1702). The register stores the multiple intermediate results and the matrix inner product result.

[0174] The process of calculating the matrix outer product and the matrix inner product can be described with reference to the above-mentioned relevant embodiments and will not be elaborated on here.

[0175] It is understood that in order to implement the functions in the above embodiments, the device includes hardware structures and / or software modules that perform the corresponding functions. Those skilled in the art should easily appreciate that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a computer software-driven hardware manner depends on the specific application scenario and design constraints of the technical solution.

[0176] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and storage medium can also exist as discrete components in a computing device.

[0177] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD). The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A computing device, characterized in that: The computing device comprises a computing unit and a merging circuit; The calculation unit is used to calculate the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The merging circuit is used to accumulate the element values ​​of the same compression identifier in the multiple intermediate result matrices to obtain the matrix outer product result.

2. The device according to claim 1, characterized in that The merging circuit is specifically used for: Accumulating the element values ​​of the same compression identifier in the plurality of intermediate result matrices to obtain a plurality of intermediate merged matrices, wherein the intermediate merged matrices are compressed sparse format matrices; The matrix outer product result is obtained by accumulating the element values ​​of the same compression identifier in the multiple intermediate merge matrices.

3. The device according to claim 2, characterized in that The merging circuit is specifically used for: The matrix outer product result is obtained by accumulating the element values ​​of the same compression identifier in multiple intermediate merge matrices obtained from the memory.

4. The device according to claim 2 or 3, characterized in that The merging circuit includes a sorting unit and an adding processing unit: The sorting unit is used to sort the elements in the multiple intermediate result matrices according to the order of compression identifiers from small to large, or to sort the elements in the multiple intermediate merged matrices, wherein the elements include element values ​​and compression identifiers; The adding processing unit is used to accumulate the element values ​​of the same compression identifier in the sorted elements.

5. The device according to claim 4, characterized in that The merging circuit further comprises an arranging unit; the arranging unit is used to arrange the elements indicated as valid by the valid values ​​according to the valid values ​​contained in the elements in the intermediate merging matrix.

6. The device according to claim 4 or 5, characterized in that The merging circuit also includes a boundary removal unit; The boundary removal unit is used to remove elements corresponding to compression identifiers other than the minimum compression identifier in the maximum compression identifiers of the two vectors in the accumulated vector of the two vectors, wherein the two vectors are vectors other than the last two vectors in the matrix to be merged, and the matrix to be merged includes either the intermediate merge matrix or the intermediate result matrix.

7. The device according to any one of claims 1 to 6, characterized in that The computing device further includes a first cache unit; The first cache unit is used to store the intermediate result matrix, which includes an element value product matrix, a compression identification matrix and a valid value matrix.

8. The device according to any one of claims 2 to 7, characterized in that: The merging circuit further includes a second cache unit; The second cache unit is used to store the intermediate merging matrix.

9. A computing device, characterized in that: The computing device comprises a computing unit and a register; The calculation unit is used to calculate the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The calculation unit is further used to accumulate the element values ​​of the same position coordinates in the multiple intermediate results to obtain a matrix inner product result; The register is used to store the multiple intermediate results and the matrix inner product result.

10. The device according to claim 9, characterized in that The computing unit is specifically used for: Vector inner product calculation is performed on element values ​​corresponding to the same compression identifier in the row vector and the column vector to obtain the intermediate result.

11. The device according to claim 9, characterized in that The computing unit is specifically used for: The first intermediate result output by the calculation unit is added to the value indicated by the same position coordinate in the register to obtain a second intermediate result.

12. The device according to any one of claims 9 to 11, characterized in that The computing unit includes a sorting circuit, a multiplication processing circuit and an aggregation circuit; The sorting circuit is used to sort the element values ​​of the row vector and the element values ​​of the column vector according to the compression identifier of the row vector and the compression identifier of the column vector; The multiplication processing circuit is used to multiply the element values ​​with the same compression identifier in the sorted elements to obtain multiple calculation results, wherein the elements include element values ​​and compression identifiers; The aggregation circuit is used to accumulate the multiple calculation results and the value in the register determined according to the same position coordinates.

13. The device according to claim 12, characterized in that The multiplication processing circuit includes N multipliers and N selectors, one multiplier is connected to one selector, and N represents the number of row vectors and the number of column vectors of the compressed sparse format matrix; an i-th selector, used to determine a value input to the multiplier according to an element adjacent to an n-th element and a compression identifier of an element adjacent to the n-th element, wherein the element adjacent to the n-th element includes an n+1-th element and an n-1-th element, the value range of i is 1-N, and the value range of n is an odd or even number in 1-2N; The i-th multiplier is used to perform a multiplication calculation on the n-th element value and the value output by the selector.

14. The device according to claim 13, characterized in that The aggregation circuit includes N adders; The N adders are used to accumulate the output results of the N multipliers and the value indicated by the same position coordinate in the register.

15. The device according to any one of claims 9 to 14, characterized in that When the matrix inner product calculation is completed, the matrix inner product result stored in the register is transferred to the memory.

16. A calculation method, characterized in that: The method is performed by a computing device, the computing device includes a computing unit and a merging circuit, and the method includes: The calculation unit calculates the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The merging circuit accumulates the element values ​​of the same compression identifier in the multiple intermediate result matrices to obtain a matrix outer product result.

17. A calculation method, characterized in that: The method is performed by a computing device, the computing device comprising a computing unit and a register, and the method comprises: The calculation unit calculates the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The calculation unit accumulates the element values ​​of the same position coordinates in the multiple intermediate results to obtain a matrix inner product result; The register stores the plurality of intermediate results and the matrix inner product result.

18. A chip, characterized in that: The chip includes a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to implement the function of the computing device described in any one of claims 1 to 8, or to implement the function of the computing device described in any one of claims 9 to 15.

19. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a set of computer instructions; when the processor executes the set of computer instructions, the function of the computing device described in any one of claims 1 to 8 is implemented, or the function of the computing device described in any one of claims 9 to 15 is implemented.

20. A computing system, characterized in that: The computing system comprises a processor and a computing device, and the computing device is used to implement the function of the computing device described in any one of claims 1 to 8, or to implement the function of the computing device described in any one of claims 9 to 15.

Images