Computing device, method, equipment, chip and system

Abstract

The invention discloses a computing device, method, equipment, chip and system, and relates to the field of computers. The calculation device provided by the invention performs calculation (such as a matrix inner product and a matrix outer product) of the compressed sparse format matrix based on the compression identifiers corresponding to the element values in the compressed sparse format matrix, and outputs the matrix inner product result or the matrix outer product result of the compressed sparse format matrix, so that the calculation efficiency of the compressed sparse format matrix is improved by utilizing the large calculation power of the calculation unit supporting the matrix calculation. The sparse format matrix is efficiently calculated and compressed, so that the calculation unit expands and supports the matrix calculation of the unstructured sparse matrix, and the calculation efficiency of the sparse matrix is improved. Besides, a merging circuit is arranged in the computing device to realize matrix accumulation of an intermediate result of a matrix outer product in the computing device, and a register is arranged in the computing device to cache and compress the intermediate result of the sparse format matrix, so that the storage space and the access bandwidth required by sparse matrix computing are reduced, and the computing efficiency of the sparse matrix is further improved.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular to a computing device, method, equipment, chip and system. Background Art

[0002] At present, matrix calculations are included in application scenarios such as artificial intelligence, scientific computing, and graphics computing. In recent years, matrix-type processors have been commonly used to improve the processing speed of matrix calculations. However, such processors have low calculation efficiency for sparse matrices. Summary of the invention

[0003] The present application provides computing devices, methods, equipment, chips and systems, thereby improving the computing efficiency of sparse matrices.

[0004] In a first aspect, a computing device is provided, which includes a computing unit and a merging circuit. The computing unit is used to calculate the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix, so as to obtain multiple intermediate result matrices. The first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices. The compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix. The compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix. The merging circuit is used to accumulate the element values ​​with the same compression identifier in multiple intermediate result matrices to obtain the matrix outer product result.

[0005] Compared with decompressing a compressed sparse format matrix first and then performing matrix calculation on the decompressed matrix, or performing matrix calculation on a compressed sparse format matrix according to the position coordinates of the compressed sparse format matrix, which leads to the problem of low calculation efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a merging circuit, which utilizes the large computing power of the computing unit supporting matrix calculation, and efficiently calculates the matrix outer product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is extended to support the matrix calculation of unstructured sparse matrices, and the merging circuit is used to realize the accumulation of the intermediate result matrix of the matrix outer product in the computing device, which effectively improves the calculation efficiency of the compressed sparse format matrix. In addition, the intermediate result matrix of the compressed sparse format matrix is ​​cached in the computing device, which reduces the storage space and access bandwidth required for the sparse matrix calculation, and further improves the calculation efficiency of the compressed sparse format matrix.

[0006] In a possible implementation, the merging circuit is specifically used to: accumulate the element values ​​of the same compression identifier in multiple intermediate result matrices to obtain multiple intermediate merged matrices, where the intermediate merged matrices are compressed sparse format matrices; accumulate the element values ​​of the same compression identifier in multiple intermediate merged matrices to obtain a matrix outer product result.

[0007] In another possible implementation, the merging circuit is specifically used to accumulate element values ​​with the same compression identifier in multiple intermediate merging matrices obtained from the memory to obtain a matrix outer product result.

[0008] In another possible implementation, the merging circuit includes a sorting unit and an adding processing unit: the sorting unit is used to sort the elements in multiple intermediate result matrices according to the order of compression identifiers from small to large, or to sort the elements in multiple intermediate merging matrices, where the elements include element values ​​and compression identifiers; the adding processing unit is used to accumulate the element values ​​of the same compression identifiers in the sorted elements.

[0009] In another possible implementation, the merging circuit further includes an arranging unit; the arranging unit is used to arrange the elements whose valid values ​​indicate validity according to the valid values ​​contained in the elements in the intermediate merging matrix.

[0010] In another possible implementation, the merging circuit also includes a boundary removal unit; the boundary removal unit is used to remove elements corresponding to compression identifiers other than the minimum compression identifier in the maximum compression identifiers of the two vectors in the accumulated vector of the two vectors, the two vectors are vectors other than the last two vectors in the matrix to be merged, and the matrix to be merged includes either an intermediate merging matrix or an intermediate result matrix.

[0011] In another possible implementation, the computing device further includes a first cache unit; the first cache unit is used to store an intermediate result matrix, the intermediate result matrix including an element value product matrix, a compression identification matrix and a valid value matrix.

[0012] In another possible implementation, the merging circuit further includes a second cache unit; the second cache unit is used to store the intermediate merging matrix.

[0013] In a second aspect, a computing device is provided, which includes a computing unit and a register; the computing unit is used to calculate the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the computing unit is also used to accumulate the element values ​​of the same position coordinates in multiple intermediate results to obtain the matrix inner product result; the register is used to store multiple intermediate results and the matrix inner product results.

[0014] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculation on the decompressed matrix, or performing matrix calculation on a compressed sparse format matrix according to the position coordinates of the compressed sparse format matrix, which leads to the problem of low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a register, and utilizes the large computing power of the computing unit that supports matrix calculation, and efficiently calculates the matrix inner product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is extended to support the matrix calculation of unstructured sparse matrices, and the computing unit is used to realize the accumulation of intermediate results of the matrix inner product in the computing device, effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate results of the compressed sparse format matrix are cached in the computing device, reducing the storage space and access bandwidth required for sparse matrix calculation, and further improving the computational efficiency of the compressed sparse format matrix.

[0015] In a possible implementation, the computing unit is specifically used to perform vector inner product calculation on element values ​​corresponding to the same compression identifier in the row vector and the column vector to obtain an intermediate result.

[0016] In another possible implementation, the calculation unit is specifically configured to add the first intermediate result output by the calculation unit to the value indicated by the same position coordinate in the register to obtain a second intermediate result.

[0017] In another possible implementation, the computing unit includes a sorting circuit, a multiplication processing circuit and an aggregation circuit; the sorting circuit is used to sort the element values ​​of the row vector and the element values ​​of the column vector according to the compression identifier of the row vector and the compression identifier of the column vector; the multiplication processing circuit is used to multiply the element values ​​with the same compression identifier in the sorted elements to obtain multiple calculation results, and the elements include element values ​​and compression identifiers; the aggregation circuit is used to accumulate multiple calculation results and values ​​determined in the register according to the same position coordinates.

[0018] In another possible implementation, the multiplication processing circuit includes N multipliers and N selectors, one multiplier is connected to one selector, N represents the number of row vectors and the number of column vectors of the compressed sparse format matrix; the i-th selector is used to determine the value of the input multiplier based on the elements adjacent to the n-th element and the compression identifiers of the elements adjacent to the n-th element, the elements adjacent to the n-th element include the n+1-th element and the n-1-th element, the value range of i is 1-N, and the value range of n is an odd or even number in 1-2N; the i-th multiplier is used to perform a multiplication calculation on the value of the n-th element and the value output by the selector.

[0019] In another possible implementation, the aggregation circuit includes N adders; the N adders are used to accumulate output results of the N multipliers and the value indicated by the same position coordinate in the register.

[0020] In another possible implementation, when the matrix inner product calculation is completed, the intermediate result stored in the register is transferred to the memory.

[0021] It should be noted that the compressed sparse format matrix described in the above aspects or any possible implementation manner of each aspect includes a compressed sparse column format matrix and a compressed sparse row format matrix.

[0022] In a third aspect, a calculation method is provided, which is executed by a calculation device, the calculation device including a calculation unit and a merging circuit, the method including: the calculation unit calculates the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the merging circuit accumulates the element values ​​with the same compression identifier in multiple intermediate result matrices to obtain the matrix outer product result.

[0023] In a fourth aspect, a computing device is provided, and the method is executed by the computing device, the computing device includes a computing unit and a register, and the method includes: the computing unit calculates the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; the computing unit accumulates the element values ​​of the same position coordinates in multiple intermediate results to obtain the matrix inner product result; the register stores multiple intermediate results and matrix inner product results.

[0024] In a fifth aspect, a chip is provided, comprising: a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to implement the function of a computing device in the first aspect or any possible implementation of the first aspect; the processor is used to implement the function of a computing device in the second aspect or any possible implementation of the second aspect.

[0025] In a sixth aspect, a computer device is provided, which includes at least one processor and a memory, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, the function of the computing device in the first aspect or any possible implementation of the first aspect is implemented, or the function of the computing device in the second aspect or any possible implementation of the second aspect is implemented.

[0026] In a seventh aspect, a computing system is provided, the computing system comprising a processor and a computing device. The computing device is used to implement the function of the computing device in the first aspect or any possible implementation of the first aspect, or to implement the function of the computing device in the second aspect or any possible implementation of the second aspect.

[0027] In an eighth aspect, a computer-readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a computer device, the computer device implements the function of a computing device in the first aspect or any possible implementation of the first aspect, or implements the function of a computing device in the second aspect or any possible implementation of the second aspect.

[0028] In the ninth aspect, a computer program product is provided. When the computer program product runs on a computer, it enables the computer to implement the functions of the computing device in the first aspect or any possible implementation of the first aspect, or to implement the functions of the computing device in the second aspect or any possible implementation of the second aspect.

[0029] The technical effects brought about by any design method from the third aspect to the ninth aspect can refer to the technical effects brought about by the first aspect to the second aspect or different design methods, and will not be repeated here.

[0030] Based on the implementations provided in the above aspects, the present application can be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A schematic diagram of a dense matrix and a sparse matrix provided for this application;

[0032] Figure 2 A schematic diagram of a compressed sparse format provided for this application;

[0033] Figure 3 A schematic diagram of the structure of a computer device provided for this application;

[0034] Figure 4 A schematic diagram of the layout of a computing device provided in this application;

[0035] Figure 5 A schematic diagram of the structure of a computing device provided by the present application;

[0036] Figure 6 A schematic diagram of calculating a vector outer product provided for this application;

[0037] Figure 7 A schematic diagram of a merging process provided for this application;

[0038] Figure 8 A schematic diagram of the structure of a merging circuit provided in this application;

[0039] Fig. 9 A schematic diagram of the structure of a sorting circuit provided in this application;

[0040] Fig.10 A schematic diagram of the structure of an arrangement circuit provided in this application;

[0041] Fig.11 A schematic diagram of the structure of a computing device provided by the present application;

[0042] Fig.12 A schematic diagram of calculating a vector inner product provided for this application;

[0043] Fig.13 A schematic diagram of the structure of a computing unit provided in this application;

[0044] Fig.14 A schematic diagram of the structure of a multiplication processing circuit provided in this application;

[0045] Fig.15 A schematic diagram of the structure of a computing device provided by the present application;

[0046] Fig.16 A schematic diagram of a calculation method provided in this application;

[0047] Fig.17 A flowchart of a calculation method provided in this application. DETAILED DESCRIPTION

[0048] To facilitate understanding, the main terms involved in this application are first explained.

[0049] Scalar: Also known as a pure quantity, it refers to a quantity that has only size but no direction and can be expressed by real numbers.

[0050] Vector: Also known as Euclidean vector or geometric vector, it refers to a quantity with magnitude and direction. A vector can be represented as a line segment with an arrow, where the direction of the arrow represents the direction of the vector and the length of the line segment represents the magnitude of the vector. Vector is a basic concept in many natural sciences, such as mathematics, physics, and engineering science. In theoretical mathematics, a vector is defined as any element in a vector space.

[0051] Row vector: In linear algebra, a row vector refers to a matrix of dimension 1×m, i.e., a matrix consists of a row with m elements, i.e., a row vector, where m is any positive integer. The row vector is shown in the following formula (1).

[0052] X=[x1 x2 ... x m Formula (1)

[0053] Column vector: In linear algebra, a column vector refers to a matrix of dimension m×1, that is, a matrix consisting of a column containing m elements, namely a column vector, where m is an arbitrary positive integer. The transpose of a row vector is a column vector. A column vector is shown in the following formula (2).

[0054]

[0055] Vector inner product: Also known as dot product, scalar product or scalar product, it refers to the multiplication of a row vector by a column vector, and the result is a scalar. The vector inner product is shown in the following formula (3).

[0056] a=[a1,a2,…an] b=[b1,b2,…bn]

[0057] a·b=a1·b1+a2·b2+…+an·bn Formula (3)

[0058] Vector outer product: A column vector multiplied by a row vector, the result is a matrix. The vector outer product is shown in equation (4).

[0059] a=[a1,a2,…an] b=[b1,b2,…bn]

[0060]

[0061] Matrix: refers to a set of complex or real numbers arranged in a rectangular array. For example, a matrix of dimension m×n is a rectangular array of elements arranged in m rows and n columns. Matrix A is shown in equation (5) below, and matrix B is shown in equation (6) below.

[0062]

[0063] Matrix addition: Matrices of the same size (the same number of rows and columns) can be added and subtracted, that is, elements at the same position coordinates can be added and subtracted. For example, matrix A and matrix B are both matrices of dimension m×n, and matrix A and matrix B are added to obtain matrix C. Matrix C is shown in equation (7).

[0064]

[0065] Matrix multiplication: When two matrices are multiplied, the number of columns of the first matrix is ​​the same as the number of rows of the second matrix. For example, if matrix A is an m×n matrix and matrix B is an n×p matrix, then the multiplication of matrix A and matrix B is an m×p matrix. The product of matrix A and matrix B is shown in formula (8).

[0066]

[0067]

[0068] Matrix inner product: The matrix is ​​decomposed into row vectors and column vectors, and the result matrix is ​​obtained by calculating the vector inner product. The element at position coordinate (i, j) in the result matrix of the matrix inner product of matrix A and matrix B is the result of calculating the inner product of the i-th row vector of matrix A and the j-th column vector of matrix B. The result matrix of the matrix inner product is shown in formula (9).

[0069]

[0070] Matrix outer product: The matrix is ​​decomposed into row vectors and column vectors, and multiple intermediate matrices are obtained by calculating the vector outer product. The elements with the same position coordinates in the multiple intermediate matrices are accumulated to obtain the matrix of the matrix outer product of matrix A and matrix B. The matrix of the matrix outer product is shown in formula (10).

[0071]

[0072] Dense matrix: refers to a matrix in which non-zero elements account for a large proportion of all elements. Generally, a matrix in an uncompressed format is called a dense matrix, which only contains the values ​​of the matrix elements, not the coordinate values ​​of the elements.

[0073] Sparse matrix: refers to a matrix in which zero elements account for a large proportion of all elements. For example, Figure 1 As shown in (a) in , it is a dense matrix, such as Figure 1 (b) in the figure is a sparse matrix. A structured sparse matrix includes regularly distributed zero elements. An unstructured sparse matrix is ​​a matrix of arbitrary sparsity, including randomly distributed non-zero elements.

[0074] Large sparse matrices often appear when solving linear models in science and engineering. When computers store and operate sparse matrices, they cannot use standard algorithms to calculate large sparse matrices, so standard algorithms need to be modified to take advantage of the sparse structure of the matrix.

[0075] Due to the sparse nature of the matrix itself, the memory cost of the sparse matrix can be saved by compressing the sparse matrix. The non-zero elements in the sparse matrix are stored in the sparse matrix format, but the zero elements are not stored. This is the process of compressing the sparse matrix. The matrix stored in compression can be called a compressed sparse format matrix.

[0076] Common sparse matrix formats include coordinate format (Coordinate, COO), compressed sparse column format (Compressed Sparse Column, CSC) and compressed sparse row format (Compressed Sparse Row, CSR).

[0077] Coordinate format: Sparse matrices are represented based on triplets. A triple contains three values, namely the row number, column number, and element value. The row number is used to identify the row coordinates of the element value in the sparse matrix. The column number is used to identify the column coordinates of the element value in the sparse matrix. The row number and column number are used to identify the position of the element value in the sparse matrix. For example, a triple is represented as (row number, column number, element value), or a triple is represented as (element value, row number, column number), and the order of the three values ​​in the triple is not limited. When a sparse matrix is ​​compressed in coordinate format, the elements in the compressed sparse format matrix include row numbers, column numbers, and element values.

[0078] For example, Figure 2 (a) in the figure shows the coordinate format of the sparse matrix. The sparse matrix E contains 5 non-zero elements, namely 2, 5, 9, 1, and 6. The element value 2 is located in the 1st row and 1st column of the sparse matrix E, and the corresponding triplet can be (1, 1, 2). The element value 5 is located in the 3rd row and 4th column of the sparse matrix E, and the corresponding triplet can be (3, 4, 5).

[0079] Compressed sparse column format: Sparse matrices are represented based on three types of data: element values, indices, and column offset values. The element values ​​and indices in the compressed sparse column format are similar to the element values ​​and row numbers in the coordinate format described above. The index is used to indicate the row coordinates or row numbers of the element values ​​in the sparse matrix. The column offset value indicates the offset of the first non-zero element of the column in the sparse matrix in the element value, and the last element indicates the number of non-zero elements in the sparse matrix. The number of column offset values ​​is the number of columns plus 1. When a sparse matrix is ​​compressed in the compressed sparse column format, the elements in the compressed sparse format matrix contain element values, row numbers, and column offset values.

[0080] For example, Figure 2 As shown in (b), it is the compressed sparse column format of the sparse matrix. Among them, the sparse matrix D contains 7 non-zero elements. The element values ​​include the non-zero elements 8, 2, 5, 7, 1, 9, 2 arranged in columns in the sparse matrix D. For example, the row number corresponding to the element value 8 and the element value 2 is 0. The row number corresponding to the element value 5 is 1. The non-zero element 8 in the first column of the sparse matrix D has an offset of 0 in the element value. The non-zero element 2 in the third column of the sparse matrix D has an offset of 1 in the element value. The non-zero element 1 in the fourth column of the sparse matrix D has an offset of 4 in the element value. The non-zero element 2 in the fifth column of the sparse matrix D has an offset of 6 in the element value.

[0081] Compressed sparse row format: Sparse matrices are represented based on three types of data: element values, indexes, and row offset values. The element values ​​and column numbers in the compressed sparse row format are similar to those in the coordinate format described above. The index is used to indicate the column coordinates or column numbers of the element value in the sparse matrix. The row offset value indicates the offset of the first non-zero element of the row in the sparse matrix in the element value, and the last element indicates the number of non-zero elements in the matrix. The number of row offset values ​​is the number of rows plus 1. When a sparse matrix is ​​compressed in the compressed sparse row format, the elements in the compressed sparse format matrix contain element values, column numbers, and row offset values.

[0082] For example, Figure 2 As shown in (c), it is the compressed sparse row format of the sparse matrix. Among them, the sparse matrix D contains 7 non-zero elements. The element values ​​include the non-zero elements 8, 2, 5, 7, 1, 2, 9 arranged in rows in the sparse matrix D. For example, the column number corresponding to the element value 8 is 0. The column number corresponding to the element value 2 is 2. The non-zero element 8 in the first row of the sparse matrix D has an offset of 0 in the element value. The non-zero element 5 in the second row of the sparse matrix D has an offset of 2 in the element value. The non-zero element 7 in the fifth row of the sparse matrix D has an offset of 3 in the element value. The non-zero element 9 in the seventh row of the sparse matrix D has an offset of 6 in the element value.

[0083] Multiply Accumulate (MAC): refers to a special operation in a digital signal processor or some microprocessors. The hardware circuit unit that implements this operation can be called a multiply-accumulate or multiply-accumulate. The multiply-accumulate operation refers to adding the product result of the multiplier and the value stored in the accumulator, and then storing it in the accumulator.

[0084] Arithmetic Intensity (AI): refers to the number of operations per byte. For example, arithmetic intensity refers to the ratio of work W to memory traffic Q, that is, AI = W / Q, where AI represents the number of operations per byte of memory traffic. When work W represents FLOPs, the resulting arithmetic intensity AI is the ratio of floating-point operations to total data movement (FLOPs / Byte). The higher the arithmetic intensity, the higher the data reuse rate.

[0085] Single instruction multiple data (SIMD) is a technology that performs the same operation on each data in a set of data (also called "data vector") to achieve spatial parallelism. SIMD is an extension of the basic instruction set of the central processing unit (CPU).

[0086] Scalable vector extension (SVE) is a SIMD instruction set under the AArch64 architecture of advanced reduced instruction set computing machines (ARM), designed to accelerate high-performance computing. It supports variable vector length, per-channel prediction, gather load and scatter store, and lateral operation. The difference between SVE and SIMD is that SIMD is used to process fixed-length data; SVE is used to process variable-length data.

[0087] Scalable matrix extension (SME) is an instruction set that supports matrix calculations based on scalable vector extensions (such as SVE and SVE2) under the ARMv9-A architecture. Understandably, the processor provides a new mode, that is, the processor provides matrix calculation functions, making it easier for the processor to use different vector sizes when processing matrix calculations, as well as SVE or SIMD calculations. SME defines an architecture that can store two-dimensional matrices and a series of instructions related to matrix calculations, including: ① Instructions for calculating the outer product of two vectors to achieve matrix multiplication; ② Instructions for reading, storing, and moving vectors from matrices. SME supports vector lengths ranging from 128 to 2048 bits.

[0088] In order to solve the problem of low computational efficiency of sparse matrices, the present application provides a computing device and method, that is, the elements in the compressed sparse format matrix include element values ​​and compression identifiers, and since the compression identifier indicates the row coordinates or column coordinates of the element value in the sparse matrix, the calculation of the compressed sparse format matrix (such as: matrix inner product, matrix outer product) is performed based on the compression identifier corresponding to the element value, and the matrix inner product result or matrix outer product result of the compressed sparse format matrix is ​​output, that is, the vector outer product calculation and the vector inner product calculation are performed based on the compression identifier in the compressed sparse format matrix to realize the calculation of the sparse matrix, thereby utilizing the large computing power of the computing unit supporting matrix calculation to efficiently calculate the compressed sparse format matrix, so that the computing unit expands the matrix calculation supporting unstructured sparse matrices, and improves the computational efficiency of the sparse matrix. In addition, a merging circuit is set in the computing device to realize the matrix accumulation of the intermediate result matrix of the matrix outer product in the computing device, and a register is set in the computing device to cache the intermediate result of the compressed sparse format matrix, reduce the storage space and access bandwidth required for the sparse matrix calculation, and further improve the computational efficiency of the sparse matrix.

[0089] The calculation method provided in this application can cover more sparse matrix calculation scenarios, such as unstructured sparse pruning technology (unstructured pruning), graph computing (graph), graph neural network (Graph Neural Network, GNN), scientific computing, multi-grid solution and other typical sparse matrix multiplication and addition calculations.

[0090] The implementation of the calculation method and device provided by the present application is described in detail below with reference to the accompanying drawings.

[0091] Figure 3 The present application provides a schematic diagram of a computer device. The computer device may be a server or other device with computing capabilities. For example, Figure 3 As shown, the computer device 300 includes a processor 310 , a bus 320 , a memory 330 , a communication interface 340 , and a memory unit 350 (also referred to as a main memory unit). The processor 310 , the memory 330 , the memory unit 350 , and the communication interface 340 are connected via the bus 320 .

[0092] The processor 310 is the control center of the computer device 300. The processor 310 may be a computing unit with computing capabilities, such as a central processing unit (CPU), a graphics processing unit (GPU), a data processing unit (DPU), a neural processing unit (NPU), etc. The computer device 300 may include multiple processors. The processor here may refer to one or more devices, circuits, and / or computing units with computing capabilities for processing data (such as computer program instructions). The processor 310 includes one processor core (core) or multiple processor cores.

[0093] Among them, the instructions that the processor core can process include SIMD instructions, SVE instructions and SME instructions. The processor core uses different data types and performs different calculations when processing different instructions. For example, when the processor processes SIMD instructions, it processes fixed-length data (such as integer data, floating-point data), and performs addition, subtraction or other calculations on the data. For another example, when the processor processes SVE instructions, it processes variable-length data (such as integer data, floating-point data), and performs addition, subtraction or other calculations on the data. For another example, when the processor processes SME instructions, it performs matrix calculations on matrices (such as matrix inner product, matrix outer product).

[0094] In this embodiment, the processor 310 may also be other general-purpose processors, digital signal processors (DSP), ASICs, FPGAs or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0095] The memory 330 can be used to store data and applications, mainly including a program storage area and a data storage area. The program storage area can store an operating system, software applications and intermediate layer software required for at least one function, etc. The data storage area can store data created when the device is used, etc. For example, the operating system can include a Linux operating system, a Unix operating system, or a Window operating system, etc. The software applications required for the at least one function can include applications related to artificial intelligence, applications related to high performance computing (HPC), applications related to deep learning, or applications related to scientific computing, etc. The intermediate layer software can include linear algebra library functions, etc. In a possible exemplary embodiment, the memory 330 includes but is not limited to static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM) or high-speed random access memory, etc. Further, the memory 330 can also include other non-volatile memories, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0096] The communication interface 340 is used to realize the communication between the computer device 300 and an external device or device. Among them, the communication interface 340 may include an input interface and an output interface. The input interface can be used to obtain a vector (such as a row vector or a column vector) in a compressed sparse format matrix in the following embodiments. The output interface can be used to output the calculation results in the following embodiments. In some embodiments, the calculation results of the compressed sparse format matrix can be directly output by the processor, or can be first stored in a memory. The computer device may include one or more input interfaces, and one or more output interfaces.

[0097] The memory unit 350 may be a volatile memory pool or a non-volatile memory pool, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM). The memory unit 350 is used to store matrix calculation results (such as matrix outer product results, matrix inner product results) of compressed sparse format matrices, etc.

[0098] The bus 320 may include a path for transmitting information between the above components (such as the processor 310, the memory unit 350, and the memory 330). In addition to the data bus, the bus 320 may also include a power bus, a control bus, and a status signal bus. However, for the sake of clarity, various buses are labeled as bus 320 in the figure. The bus 320 may be a Peripheral Component Interconnect Express (PCIe) bus, or an extended industry standard architecture (EISA) bus, a unified bus (Ubus or UB), a computer express link (CXL), a cache coherent interconnect for accelerators (CCIX), etc. The bus 320 can be divided into an address bus, a data bus, a control bus, etc.

[0099] Figure 3 The connection method between the processor core and other devices in the processor is only a schematic illustration. In one possible implementation, the processor core is connected to other devices in the processor through a ring bus, and accesses other devices in the processor through the ring bus. In another possible implementation, the processor core and other devices in the processor are connected through a mesh bus, and the processor core accesses other devices in the processor through the mesh bus.

[0100] It is worth mentioning that Figure 3 In the example, a computer device 300 including one processor 310 and one memory 330 is taken. Here, the processor 310 and the memory 330 are respectively used to indicate a type of device or equipment. In a specific embodiment, the number of each type of device or equipment can be determined according to business requirements. Figure 3 The device structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0101] The computer device 300 may include a computing device, which is used to provide a matrix inner product or matrix outer product computing function. The computing device may be disposed inside or outside the processor, without limitation.

[0102] For example, Figure 4 As shown in (a) of FIG. 1 , the computing device 313 is integrated into the processor 310. Figure 4As shown in (b), the computing device 313 is arranged outside the processor 310, for example, the computing device is arranged in the form of a PCIe card. The processor 310 may also include a cache memory 311, a register 312, a controller 314, one or more arithmetic logic units (ALU) 315 and a memory management unit (MMU) 316, etc.

[0103] The calculation device 313 is used to implement the matrix inner product or matrix outer product calculation of the sparse matrix based on the compression identifier contained in the elements of the compressed sparse format matrix. For a specific explanation of the matrix inner product or matrix outer product calculation of the sparse matrix, please refer to the detailed description of the following embodiment.

[0104] The cache memory 311 is used to store instructions or data that may be accessed multiple times by the processor core in the processor 310 , thereby increasing the speed at which the processor processes data and preventing the processor from frequently accessing the memory unit 350 .

[0105] In physical form, cache memory 311 may be random access memory (RAM), static random-access memory (SRAM), dynamic RAM (DRAM), or other types of storage devices that can store information and instructions.

[0106] In terms of logical form, the cache memory 311 may be a first-level cache (L1 cache), a second-level cache (L2 cache), a third-level cache (L3 cache), or a cache device at any level. For example, the cache memory 311 disposed inside the processor core may be a first-level cache (L1 cache) and a second-level cache (L2 cache). The cache memory 311 disposed outside the processor core may be a third-level cache (L3 cache).

[0107] Register 312 is used to store instructions or data that may be accessed multiple times by the processor core in processor 310. Since the access speed of the register is higher than the access speed of the cache memory, the instructions or data that may be accessed multiple times by the processor core can be stored in the register first, which can further improve the speed of the processor processing data.

[0108] In the present application, when the computing device calculates the matrix outer product of two compressed sparse format matrices, first, N column vectors of the first compressed sparse format matrix and N row vectors of the second compressed sparse format matrix are obtained, and the vector outer products of the N column vectors and the N row vectors are calculated to obtain N intermediate result matrices. Among them, the column vector includes the first element value and the compression mark of the first element value, and the compression mark of the first element value is used to indicate the row coordinate of the first element value in the first sparse matrix. The row vector includes the second element value and the compression mark of the second element value, and the compression mark of the second element value is used to indicate the column coordinate of the second element value in the second sparse matrix. The first compressed sparse format matrix is ​​the compressed sparse format matrix of the first sparse matrix. The second compressed sparse format matrix is ​​the compressed sparse format matrix of the second sparse matrix. The intermediate result matrix includes the third element value and the compression mark of the third element value, and the compression mark includes the compression mark of the first element value and the compression mark of the second element value.

[0109] Furthermore, the computing device accumulates the third element values ​​having the same compression identifier of the third element values ​​in multiple intermediate result matrices according to the compression identifier of the third element value to obtain a matrix outer product result.

[0110] It should be noted that all matrices involved in the matrix outer product calculation process described in this application are compressed sparse format matrices. For example, the matrix for calculating the matrix outer product, the intermediate result matrix, the intermediate merge matrix and the matrix outer product result are all compressed sparse format matrices.

[0111] Compared with decompressing a compressed sparse format matrix first and then performing matrix calculation on the decompressed matrix, or performing matrix calculation on a compressed sparse format matrix according to the position coordinates of the compressed sparse format matrix, which leads to the problem of low calculation efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a merging circuit, which utilizes the large computing power of the computing unit supporting matrix calculation, and efficiently calculates the matrix outer product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is extended to support the matrix calculation of unstructured sparse matrices, and the merging circuit is used to realize the accumulation of the intermediate result matrix of the matrix outer product in the computing device, which effectively improves the calculation efficiency of the compressed sparse format matrix. In addition, the intermediate result matrix of the compressed sparse format matrix is ​​cached in the computing device, which reduces the storage space and access bandwidth required for the sparse matrix calculation, and further improves the calculation efficiency of the compressed sparse format matrix.

[0112] Figure 5 This is a schematic diagram of the structure of a computing device provided by this application. Figure 5 As shown, the computing device 500 includes a computing unit 510 , a first cache unit 520 and a merging circuit 530 .

[0113] The computing unit 510 may be an SME unit in an ARM processor. The input of the computing unit 510 includes a column vector in a first matrix and a row vector in a second matrix. Both the first matrix and the second matrix are compressed sparse format matrices. The elements in the compressed sparse format matrix include an element value, a compression identifier, and a valid value.

[0114] The compression identifier is used to indicate the column coordinates or row coordinates of the element value in the sparse matrix. For example, the first matrix is ​​a compressed sparse column format matrix, and the compression identifier is used to indicate the row coordinates of the element value in the first sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. For another example, the second matrix is ​​a compressed sparse row format matrix, and the compression identifier is used to indicate the column coordinates of the element value in the second sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix.

[0115] The valid value is used to indicate whether the element value is valid. For example, when the valid value is 1, it means the element value is valid; when the valid value is 0, it means the element value is invalid.

[0116] Optionally, the first matrix may be a submatrix in a first compressed sparse format matrix. The second matrix may be a submatrix in a second compressed sparse format matrix. The size of the first matrix and the size of the second matrix may be determined by the size of a cache unit in a system. The following description is made by taking the first matrix and the second matrix both being N×N matrices as an example.

[0117] The calculation unit 510 is used to calculate the vector outer product of each column vector of the N column vectors of the first matrix and the N row vectors of the second matrix to obtain multiple intermediate result matrices.

[0118] The calculation unit 510 is specifically used to calculate the matrix outer product of the first matrix and the second matrix according to the compression mark of the column vector in the first matrix and the compression mark of the row vector in the second matrix to obtain multiple intermediate result matrices. The intermediate result matrices are all compressed sparse format matrices.

[0119] The calculation unit 510 obtains a column vector of the first matrix and a row vector of the second matrix each time, and calculates the element values, compression marks and effective values ​​in the column vector and the row vector respectively to obtain an intermediate result matrix. The intermediate result matrix includes an element value product matrix, a compression mark matrix and an effective value matrix.

[0120] For example, Figure 6 As shown, it is assumed that the calculation unit 510 calculates the vector outer product of the column vector A and the row vector B.

[0121] The calculation unit 510 is used to sequentially multiply each element value (A_value) of the column vector A with the element value (B_value) of the row vector B to obtain an element value product matrix (C_value). The element value product matrix may be a compressed sparse coordinate format matrix.

[0122] The calculation unit 510 is used to combine the row coordinates (A_index) of the element values ​​of the column vector A and the column coordinates (B_index) of the element values ​​of the row vector B in sequence to obtain a compressed identification matrix (C_index) corresponding to the element value product matrix, and the compressed identification matrix includes coordinates corresponding to the product of the element values ​​in the product matrix (coordinates in coordinate format).

[0123] The calculation unit 510 is used for determining that the value at the corresponding position in the effective value matrix (C_predicate) is valid only when the effective values ​​(A_predicate, B_predicate) corresponding to the element values ​​of the column vector A and the row vector B are both valid.

[0124] The first cache unit 520 is used to store the intermediate result matrix. Optionally, the first cache unit 520 is also used to store the intermediate merge matrix. Optionally, the computing device 500 may further include a selector 540, which selects to cache the intermediate result matrix output by the computing unit 510 to the first cache unit 520, or cache the intermediate merge matrix in the memory to the first cache unit 520.

[0125] The merging circuit 530 is used to accumulate the element values ​​of the same compression identifier in multiple intermediate result matrices to obtain the matrix outer product result.

[0126] Optionally, the intermediate result matrix stored in the first cache unit 520 can be stored in a memory to wait for subsequent merging processing, or the intermediate result matrix stored in the first cache unit 520 can be stored in a merging circuit 530, and the merging circuit 530 includes a register, and the size of the register is larger than the size of the first cache unit 520. The merging circuit 530 is used to store the merging result.

[0127] Optionally, when the remaining capacity of the merging circuit 530 is less than a threshold, that is, there is no remaining storage space in the merging circuit 530 to store the merging result, the merging result can be stored in a memory, which includes multiple ordered data blocks whose size does not exceed the size of the merging circuit 530.

[0128] In some embodiments, Figure 7As shown, the merging circuit 530 performs an on-chip merging process and a memory-to-on-chip merging process. The on-chip merging process includes the merging circuit 530 merging the intermediate result matrix output by the first cache unit 520 and the intermediate result matrix stored by the merging circuit 530, and / or the merging circuit 530 merging the intermediate result matrix output by the first cache unit 520 and the intermediate merged matrix stored by the merging circuit 530 to obtain an updated intermediate merged matrix. The memory-to-on-chip merging process includes merging the intermediate merged matrix read from the memory.

[0129] It is understandable that when the merging circuit 530 performs the on-chip merging process, the input of the merging circuit 530 includes the intermediate result matrix output by the first cache unit 520, and at least one of the intermediate result matrix or the intermediate merging matrix stored by the merging circuit 530. When the merging circuit 530 performs the memory-to-on-chip merging process, the input of the merging circuit 530 includes the intermediate merging matrix stored in the memory. The intermediate merging matrix stored in the memory may be the intermediate merging matrix output by the merging circuit 530.

[0130] The merging circuit 530 is specifically used for: accumulating the element values ​​of the same compression identifier in multiple intermediate result matrices to obtain multiple intermediate merged matrices, and accumulating the element values ​​of the same compression identifier in multiple intermediate merged matrices to obtain the matrix outer product result. It can be understood that the elements in the intermediate result matrix and the intermediate merged matrix contain element values ​​and compression identifiers. The intermediate result matrix and the intermediate merged matrix are compressed sparse format matrices. The compression identifier can be a coordinate in a compressed sparse coordinate format matrix, and the compression identifier contains row coordinates or column coordinates. The same compression identifier in multiple intermediate result matrices can indicate that the row coordinates and column coordinates contained in the compression identifier are the same. The same compression identifier in multiple intermediate merged matrices can indicate that the row coordinates and column coordinates contained in the compression identifier are the same. The size of the intermediate merged matrix may be equal to the size of the intermediate result matrix or greater than the size of the intermediate result matrix. For example, if the sizes of multiple intermediate result matrices are the same and the multiple intermediate result matrices contain the same compression identifier, then the size of the intermediate merged matrix is ​​equal to the size of the intermediate result matrix. For another example, if the sizes of multiple intermediate result matrices are the same and the multiple intermediate result matrices contain at least one compression identifier that is different from the other compression identifiers, then the size of the intermediate merged matrix is ​​greater than the size of the intermediate result matrix. When the merging circuit 530 performs the memory-to-on-chip merging process, the merged intermediate merging matrix obtained by merging multiple intermediate merging matrices may be equal to or larger than the intermediate merging matrix.

[0131] In some embodiments, Figure 8 As shown, the merging circuit 530 includes a sorting unit 531 , an addition processing unit 532 , an arrangement unit 533 , a boundary removal unit 534 and a second cache unit 535 .

[0132] The sorting unit 531 is used to sort the elements in the multiple intermediate result matrices according to the order of compression identifiers in the intermediate result matrices from small to large when merging multiple intermediate result matrices.

[0133] Optionally, the sorting unit 531 is further configured to sort the elements in the plurality of intermediate merged matrices according to the order of the compression identifiers in the intermediate merged matrices from small to large when merging the intermediate merged matrices.

[0134] Optionally, the sorting unit 531 is further used to sort the elements in the intermediate result matrix and the intermediate merged matrix according to the compression flags in the intermediate result matrix and the compression flags in the intermediate merged matrix from small to large when merging the intermediate result matrix and the intermediate merged matrix.

[0135] In some embodiments, the sorting unit can be implemented by merge sorting. Two vectors containing N elements are connected end to end to form a vector of length 2N. In the first round, the elements of 0 to N-1 are compared with the elements of N to 2N-1 respectively. If the effective value of the low-order element is 0 and the effective value of the high-order element is 1, or the effective values ​​of both elements are 1, but the compression mark of the low-order is greater than the compression mark of the high-order, the positions of the two elements are swapped; in the second round, the elements of N / 2 to N-1 are compared with the elements of N to 3N / 2-1 respectively, and the exchange principle is the same as the first round; in the third round, the elements of N / 4 to N / 2-1, 3N / 4 to N-1, and 5N / 4 to 3N / 2-1 are compared with the elements separated by N / 4 positions in turn, and the exchange principle is the same as the first round; after completing multiple rounds of calculations, a set of vectors with ordered compression marks can be obtained, and the elements with the same compression marks are in adjacent positions.

[0136] like Fig. 9 As shown, it is a schematic diagram of a design scheme of an 8+8 sorting circuit provided by the present application. The two connected black dots represent a comparator. After four rounds of comparison, the elements in the two vectors are sorted in ascending order according to the compression identifier.

[0137] The adding processing unit 532 is used to accumulate the element values ​​of the same compression identifier in the sorted elements.

[0138] In some embodiments, the addition processing unit 532 may include N adders and N selectors.

[0139] A selector is used to determine a value of an input adder according to elements adjacent to an n-th element and compression identifiers of elements adjacent to the n-th element, wherein the elements adjacent to the n-th element include an n+1-th element and an n-1-th element.

[0140] The adder is used to perform cumulative calculation on the nth element value and the value output by the selector. Optionally, when the value output by the selector is 0, it indicates that there is no compression identifier that is the same as the compression identifier of the nth element, that is, the compression identifier of the nth element is different from the compression identifier of the n+1th element and the compression identifier of the n-1th element, and the adder directly outputs the element value of the nth element.

[0141] The arrangement unit 533 is used to arrange the elements indicated as valid by the valid values ​​according to the valid values ​​contained in the elements in the intermediate merge matrix, thereby reducing the storage capacity occupied by the process of merging the matrices.

[0142] In some embodiments, the arrangement unit 533 is used to move valid elements to lower bits according to the valid values ​​of the elements, and set the remaining elements to 0. For example, Fig.10 As shown in FIG. 1 , a schematic diagram of a compact arrangement design of 16 elements is shown. Valid elements are sorted in ascending order according to the compression identifier, and invalid elements are set to 0. In order to optimize the timing, it is divided into multiple levels (such as two levels) of compactness, and the timing can be achieved through level 1 if the conditions are met.

[0143] The boundary removal unit 534 is used to remove the elements corresponding to the compression identifiers other than the minimum compression identifier in the maximum compression identifiers of the two vectors in the accumulated vector of the two vectors. The two vectors are vectors other than the last two vectors in the matrix to be merged, and the matrix to be merged includes any one of the intermediate merge matrix and the intermediate result matrix. When the last two vectors in the matrix to be merged are accumulated, the compression identifiers in the two vectors are retained.

[0144] For example, matrix 1 to be merged and matrix 2 to be merged are 2×2 matrices, and matrix 1 to be merged and matrix 2 to be merged are merged row by row. Two elements in the first row vector of matrix 1 to be merged include compression identifier 1 and compression identifier 2, and two elements in the second row vector of matrix 1 to be merged include compression identifier 3 and compression identifier 4. Two elements in the first row vector of matrix 2 to be merged include compression identifier 1 and compression identifier 3, and two elements in the second row vector of matrix 2 to be merged include compression identifier 5 and compression identifier 6.

[0145] Merge the first row vector of matrix 1 to be merged and the first row vector of matrix 2 to be merged, accumulate the element value corresponding to compression identifier 2 in the first row vector of matrix 1 to be merged and the element value corresponding to compression identifier 2 in the first row vector of matrix 2 to be merged, remove the element value corresponding to compression identifier 3 in the first row vector of matrix 2 to be merged, and retain the element value corresponding to compression identifier 1 in the first row vector of matrix 1 to be merged.

[0146] Merge the second row vector of matrix 1 to be merged and the first row vector of matrix 2 to be merged. Since the element value corresponding to compression identifier 1 in the first row vector of matrix 2 to be merged has been merged with compression identifier 1 in the first row vector of matrix 1 to be merged, there is no need to merge compression identifier 1 in the first row vector of matrix 1 to be merged. Accumulate the element value corresponding to compression identifier 3 in the second row vector of matrix 1 to be merged and the element value corresponding to compression identifier 3 in the first row vector of matrix 2 to be merged, and remove the element value corresponding to compression identifier 4 in the second row vector of matrix 1 to be merged.

[0147] Merge the second row vector of matrix 1 to be merged and the second row vector of matrix 2 to be merged. Since the element value corresponding to compression identifier 3 in the second row vector of matrix 1 to be merged is merged with compression identifier 3 in the first row vector of matrix 2 to be merged, there is no need to merge compression identifier 3 in the second row vector of matrix 1 to be merged. Keep the element value corresponding to compression identifier 4 in the second row vector of matrix 1 to be merged, and keep the element values ​​corresponding to compression identifier 5 and compression identifier 6 in the second row vector of matrix 2 to be merged.

[0148] It should be understood that the elements contained in the intermediate merge matrix stored in the memory are ordered, and the merging process is also carried out in order. When two vectors are merging, they can be directly stored in the memory after removing the boundary according to the minimum value IDX of the two maximum compression identifiers (IDX=Min(AindexA(max), BindexB(max)))). Because the compression identifiers of the subsequently merged elements are greater than IDX, the size of the merged matrix is ​​not limited by the size of the cache unit in the merging circuit.

[0149] The second cache unit 535 is used to store the intermediate merge matrix.

[0150] It should be noted that the merging circuit may not include the sorting unit and the arranging unit. The merging circuit 530 compares the compression identifiers in the intermediate result matrix one by one, and accumulates the values ​​of the elements with the same compression identifier in the intermediate result matrix.

[0151] In the present application, when the computing device calculates the matrix inner product of two compressed sparse format matrices, first, N row vectors of the first compressed sparse format matrix and N column vectors of the second compressed sparse format matrix are obtained, and the vector inner product of the N row vectors and the N column vectors is calculated to obtain N intermediate results. Among them, the row vector includes a first element value and a compression identifier of the first element value, and the compression identifier of the first element value is used to indicate the column coordinate of the first element value in the first sparse matrix. The column vector includes a second element value and a compression identifier of the second element value, and the compression identifier of the second element value is used to indicate the row coordinate of the second element value in the second sparse matrix. The first compressed sparse format matrix is ​​a compressed sparse format matrix of the first sparse matrix. The second compressed sparse format matrix is ​​a compressed sparse format matrix of the second sparse matrix.

[0152] Furthermore, the computing device accumulates the element values ​​of the same position coordinates in multiple intermediate results to obtain a matrix inner product result.

[0153] It should be noted that all matrices involved in the matrix inner product calculation process described in this application are compressed sparse format matrices. For example, the matrix for calculating the matrix outer product and the matrix inner product result are both compressed sparse format matrices.

[0154] Compared with first decompressing a compressed sparse format matrix and then performing matrix calculation on the decompressed matrix, or performing matrix calculation on a compressed sparse format matrix according to the position coordinates of the compressed sparse format matrix, which leads to the problem of low computational efficiency of the sparse matrix, the computing device provided by the present application includes a computing unit and a register, and utilizes the large computing power of the computing unit that supports matrix calculation, and efficiently calculates the matrix inner product of the compressed sparse format matrix based on the compression identifier of the compressed sparse format matrix, so that the computing unit is extended to support the matrix calculation of unstructured sparse matrices, and the computing unit is used to realize the accumulation of intermediate results of the matrix inner product in the computing device, effectively improving the computational efficiency of the compressed sparse format matrix. In addition, the intermediate results of the compressed sparse format matrix are cached in the computing device, reducing the storage space and access bandwidth required for sparse matrix calculation, and further improving the computational efficiency of the compressed sparse format matrix.

[0155] Fig.11 The schematic diagram of the structure of a computing device provided in the present application is as follows: The computing device 1100 includes a computing unit 1110 and a register 1120 .

[0156] The input of the calculation unit 1110 includes the row vectors in the first matrix and the column vectors in the second matrix. The first matrix and the second matrix are both compressed sparse format matrices. The elements in the compressed sparse format matrix include element values, compression flags and valid values.

[0157] The compression identifier is used to indicate the column coordinates or row coordinates of the element value in the sparse matrix. For example, the first matrix is ​​a compressed sparse row format matrix, and the compression identifier is used to indicate the column coordinates of the element value in the first sparse matrix. The first matrix is ​​a compressed sparse format matrix of the first sparse matrix. For another example, the second matrix is ​​a compressed sparse column format matrix, and the compression identifier is used to indicate the row coordinates of the element value in the second sparse matrix. The second matrix is ​​a compressed sparse format matrix of the second sparse matrix.

[0158] Optionally, the first matrix may be a submatrix in a first compressed sparse format matrix. The second matrix may be a submatrix in a second compressed sparse format matrix. The size of the first matrix and the size of the second matrix may be determined by the size of a cache unit in a system. The following description is made by taking the first matrix and the second matrix both being N×N matrices as an example.

[0159] The calculation unit 1110 is used to calculate the vector inner product of each of the N row vectors of the first matrix and the N column vectors of the second matrix to obtain multiple intermediate results.

[0160] The calculation unit 1110 is used to calculate the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results.

[0161] The calculation unit 1110 is specifically used to perform vector inner product calculation on the element values ​​corresponding to the same compression identifier in the row vector and the column vector to obtain an intermediate result. The compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix. The compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix.

[0162] Register 1120 is used to store multiple intermediate results and matrix inner product results. Optionally, when the matrix inner product calculation is completed, the intermediate results stored in register 1120 are transferred to the memory.

[0163] Understandably, the intermediate result may be an element in the matrix stored in register 1120, the element including the element value. The element value is the result of the vector inner product of the row vector and the column vector, that is, the result of multiplying and accumulating the element values ​​of the same compression identifier in the row vector and the column vector.

[0164] The calculation unit 1110 is further used to accumulate the element values ​​of the same position coordinates in multiple intermediate results to obtain a matrix inner product result.

[0165] The calculation unit 1110 is specifically used to add the first intermediate result output by the calculation unit to the value indicated by the same position coordinate in the register to obtain a second intermediate result. It can be understood that the first intermediate result output by the calculation unit 1110 is added to the value of the position where the register 1120 should store the first intermediate result, and the addition result is stored in the corresponding position of the register 1120.

[0166] The register in the computing device caches the matrix inner product results of the small matrix blocks, so that the matrix inner product results of the small matrix can be directly accumulated in the computing unit. Fig.12As shown, it is assumed that the first compressed sparse format matrix includes submatrices A00-A03 and submatrices A10-A12, and the second compressed sparse format matrix includes submatrices B00-B20 and submatrices B01-B21. Matrix C00 is the result of cross-multiplication of A00-A03 and B00-B20 and accumulation. After the calculation of the C00 block of the matrix C of the inner product of the first compressed sparse format matrix and the second compressed sparse format matrix is ​​completed, C00 is stored in the memory, and then C01, C10 and C11 are calculated in sequence. Therefore, the register caches the intermediate results instead of directly outputting the scalar, avoiding the storage of the intermediate results in the memory, and reducing the memory access overhead in the process of downloading from the memory to the computing unit for accumulation.

[0167] In some embodiments, Fig.13 As shown, the computing unit 1110 includes a sorting circuit 1111, a multiplication processing circuit 1112, and an aggregation circuit 1113. Optionally, the multipliers and adders used by the multiplication processing circuit 1112 and the aggregation circuit 1113 can reuse the multipliers and adders of the computing unit for computing dense matrices.

[0168] The sorting circuit 1111 is used to sort the element values ​​of the row vector and the element values ​​of the column vector according to the compression identifier of the row vector and the compression identifier of the column vector. It can be understood that the elements in the row vector A and the column vector B input to the calculation unit 1110 are sorted in the order of the compression identifier from small to large, and the elements after the compression identifier is sorted from small to large are obtained. There are at most two elements with the same compression identifier, and they are in adjacent positions of the sorted elements. For an explanation of the sorting process, refer to the description of the above-mentioned sorting unit 331.

[0169] Optionally, the computing unit 1110 further includes a first cache unit 1114. The sorting circuit 1111 stores the sorted elements in the first cache unit 1114.

[0170] The multiplication processing circuit 1112 is used to multiply the element values ​​with the same compression identifier in the sorted elements to obtain multiple calculation results, where the elements include element values ​​and compression identifiers. Understandably, the compression identifier of each element is compared with the compression identifier of the next element. If the compression identifiers of the two elements are the same, the element values ​​of the two elements are multiplied, the element value of the current element is modified to the calculation result of the multiplication of the element values ​​of the two elements, and the element value of the next element is set to 0. If the compression identifiers of the two elements are different, the element value of the current element is modified to 0, and the calculation result of the multiplication of the element values ​​of the same compression identifier is obtained after processing.

[0171] Since the elements with the same compression identifier after sorting are adjacent, and each element has the same compression identifier as at most one other element. If the row vector and the column vector both contain N elements, N multipliers are needed to implement the element value multiplication processing with the same compression identifier after sorting. In some embodiments, the multiplication processing circuit 1112 includes N multipliers and N selectors, and one multiplier is connected to one selector.

[0172] The i-th selector is used to determine the value of the input multiplier according to the elements adjacent to the n-th element and the compression identifiers of the elements adjacent to the n-th element, and the elements adjacent to the n-th element include the n+1-th element and the n-1-th element. The value range of i is 1-N, and the value range of n is an odd or even number in 1-2N.

[0173] The i-th multiplier is used to perform a multiplication calculation on the n-th element value and the value output by the selector.

[0174] Optionally, the selector can determine the input of the multiplier based on the compression flag same flag (index same flag). Index same flag[n] represents the indicator bit that the compression flags of the nth element and the n+1th element are the same. If index same flag[n] is 1, indicating that the compression flag of the nth element and the compression flag of the n+1th element are the same, the multiplier uses the element value of the n+1th element as input; if index same flag[n] is 0, indicating that the compression flag of the nth element and the compression flag of the n+1th element are different, the multiplier uses the element value of the n-1th element as input; if the compression flag of the nth element is not equal to the compression flag of the n-1th element and the compression flag of the n+1th element, the multiplier input is 0.

[0175] Optionally, the controller in the computing system (such as Figure 4 The controller 314 shown) determines that the compression identifier of the nth element is the same as the compression identifier of the n+1th element, and the compression identifier of the nth element is the same as the compression identifier of the n-1th element.

[0176] For example, Fig.14 As shown, the multiplication processing circuit 1112 includes 8 multipliers and 8 selectors. The multiplication processing circuit 1112 multiplies the element values ​​with the same compression identifier in the 16 elements after sorting to obtain multiple calculation results. The i-th multiplier performs a multiplication calculation on the n-th element value and the value output by the i-th selector.

[0177] For example, the first multiplier obtains the element value of the second element. When the identifier 0 is 1, it means that the compression identifier of the second element is the same as the compression identifier of the third element. The first selector outputs the element value of the third element, and the first multiplier performs a multiplication calculation on the second element value and the element value of the third element output by the first selector. When the identifier 1 is 1, it means that the compression identifier of the second element is the same as the compression identifier of the first element. The first selector outputs the element value of the first element, and the first multiplier performs a multiplication calculation on the second element value and the element value of the first element output by the first selector. When the identifier 1 and the identifier 0 are both 0, it means that the compression identifier of the second element and the compression identifier of the first element and the compression identifier of the third element are all different. The first selector outputs 0, and the first multiplier performs a multiplication calculation on the second element value and the 0 output by the first selector.

[0178] Optionally, the calculation unit 1110 further includes a second cache unit 1115. The sorting circuit 1111 stores the calculation results of the element values ​​corresponding to the same compression identifier in the second cache unit 1115.

[0179] Aggregation circuit 1113 is used to accumulate multiple calculation results and values ​​in registers determined according to the same position coordinates.

[0180] In some embodiments, the aggregation circuit 1113 includes N adders. The N adders are used to accumulate the output results of the N multipliers and the value indicated by the same position coordinate in the register. For example, the first adder accumulates the calculation results of the first multiplier and the second multiplier. The second adder accumulates the calculation results of the third multiplier and the fourth multiplier. The third adder accumulates the calculation results of the fifth multiplier and the sixth multiplier. The fourth adder accumulates the calculation results of the seventh multiplier and the eighth multiplier. The fifth adder accumulates the results output by the first adder and the second adder. The sixth adder accumulates the results output by the third adder and the fourth adder. The seventh adder accumulates the results output by the fifth adder and the sixth adder. The eighth adder accumulates the results indicated by the same position coordinate in the register by the seventh adder.

[0181] In some embodiments, Fig.15 As shown, the computing device may also include multiple registers and multiple computing units. The multiple computing units process the vector inner product of the row vectors in the first matrix and the column vectors in the second matrix in parallel to realize the vector inner product of the first matrix and the second matrix and obtain the vector inner product result.

[0182] Fig.16A flowchart of a calculation method provided by the present application. The method is executed by the calculation device for calculating the matrix outer product of the above embodiment. The calculation device includes a calculation unit and a merging circuit, and the method includes: the calculation unit calculates the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices (step 1601). The merging circuit accumulates the element values ​​of the same compression identifier in the multiple intermediate result matrices to obtain the matrix outer product result (step 1602).

[0183] Fig.17 A flow chart of a calculation method provided by the present application. The method is executed by the calculation device for calculating the matrix inner product of the above embodiment. The calculation device includes a calculation unit and a register, and the method includes: the calculation unit calculates the matrix inner product of the first matrix and the second matrix according to the compression mark of the row vector in the first matrix and the compression mark of the column vector in the second matrix, and obtains multiple intermediate results (step 1701). The calculation unit accumulates the element values ​​of the same position coordinates in the multiple intermediate results to obtain the matrix inner product result (step 1702). The register stores multiple intermediate results and the matrix inner product result.

[0184] The process of calculating the matrix outer product and the matrix inner product can be described with reference to the above-mentioned related embodiments and will not be elaborated herein.

[0185] It is understandable that in order to implement the functions in the above embodiments, the device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0186] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist in a computing device as discrete components.

[0187] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in the embodiment of the present application is executed in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD). The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A computing device, characterized in that: The computing device comprises a computing unit and a merging circuit; The calculation unit is used to calculate the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The merging circuit is used to accumulate the element values ​​of the same compression identifier in the multiple intermediate result matrices to obtain the matrix outer product result.

2. The device according to claim 1, characterized in that The merging circuit is specifically used for: Accumulating the element values ​​of the same compression identifier in the plurality of intermediate result matrices to obtain a plurality of intermediate merge matrices, wherein the intermediate merge matrices are compressed sparse format matrices; The matrix outer product result is obtained by accumulating the element values ​​of the same compression identifier in the multiple intermediate merge matrices.

3. The device according to claim 2, characterized in that The merging circuit is specifically used for: The matrix outer product result is obtained by accumulating the element values ​​of the same compression identifier in multiple intermediate merge matrices obtained from the memory.

4. The device according to claim 2 or 3, characterized in that The merging circuit includes a sorting unit and an adding processing unit: The sorting unit is used to sort the elements in the multiple intermediate result matrices according to the order of compression identifiers from small to large, or to sort the elements in the multiple intermediate merged matrices, wherein the elements include element values ​​and compression identifiers; The adding processing unit is used to accumulate the element values ​​of the same compression identifier in the sorted elements.

5. The device according to claim 4, characterized in that The merging circuit also includes an arrangement unit; The arrangement unit is used to arrange the elements indicated as valid by the valid values ​​according to the valid values ​​contained in the elements in the intermediate merge matrix.

6. The device according to claim 4 or 5, characterized in that The merging circuit also includes a boundary removal unit; The boundary removal unit is used to remove elements corresponding to compression identifiers other than the minimum compression identifier in the maximum compression identifiers of the two vectors in the accumulated vector of the two vectors, wherein the two vectors are vectors other than the last two vectors in the matrix to be merged, and the matrix to be merged includes either the intermediate merge matrix or the intermediate result matrix.

7. The device according to any one of claims 1 to 6, characterized in that The computing device further includes a first cache unit; The first cache unit is used to store the intermediate result matrix, which includes an element value product matrix, a compression identification matrix and a valid value matrix.

8. The device according to any one of claims 2 to 7, characterized in that: The merging circuit further includes a second cache unit; The second cache unit is used to store the intermediate merging matrix.

9. A computing device, characterized in that: The computing device comprises a computing unit and a register; The calculation unit is used to calculate the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The calculation unit is further used to accumulate the element values ​​of the same position coordinates in the multiple intermediate results to obtain a matrix inner product result; The register is used to store the multiple intermediate results and the matrix inner product result.

10. The device according to claim 9, characterized in that The computing unit is specifically used for: Vector inner product calculation is performed on element values ​​corresponding to the same compression identifier in the row vector and the column vector to obtain the intermediate result.

11. The device according to claim 9, characterized in that The computing unit is specifically used for: The first intermediate result output by the calculation unit is added to the value indicated by the same position coordinate in the register to obtain a second intermediate result.

12. The device according to any one of claims 9 to 11, characterized in that The computing unit includes a sorting circuit, a multiplication processing circuit and an aggregation circuit; The sorting circuit is used to sort the element values ​​of the row vector and the element values ​​of the column vector according to the compression identifier of the row vector and the compression identifier of the column vector; The multiplication processing circuit is used to multiply the element values ​​with the same compression identifier in the sorted elements to obtain multiple calculation results, wherein the elements include element values ​​and compression identifiers; The aggregation circuit is used to accumulate the multiple calculation results and the value in the register determined according to the same position coordinates.

13. The device according to claim 12, characterized in that The multiplication processing circuit includes N multipliers and N selectors, one multiplier is connected to one selector, and N represents the number of row vectors and the number of column vectors of the compressed sparse format matrix; an i-th selector, used to determine a value input to the multiplier according to an element adjacent to an n-th element and a compression identifier of an element adjacent to the n-th element, wherein the element adjacent to the n-th element includes an n+1-th element and an n-1-th element, the value range of i is 1-N, and the value range of n is an odd or even number in 1-2N; The i-th multiplier is used to perform a multiplication calculation on the n-th element value and the value output by the selector.

14. The device according to claim 13, characterized in that The aggregation circuit includes N adders; The N adders are used to accumulate the output results of the N multipliers and the value indicated by the same position coordinate in the register.

15. The device according to any one of claims 9 to 14, characterized in that When the matrix inner product calculation is completed, the matrix inner product result stored in the register is transferred to the memory.

16. A calculation method, characterized in that: The method is performed by a computing device, the computing device comprising a computing unit and a merging circuit, and the method comprises: The calculation unit calculates the matrix outer product of the first matrix and the second matrix according to the compression identifier of the column vector in the first matrix and the compression identifier of the row vector in the second matrix to obtain multiple intermediate result matrices, the first matrix, the second matrix and the intermediate result matrix are all compressed sparse format matrices, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the first sparse matrix, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The merging circuit accumulates the element values ​​of the same compression identifier in the multiple intermediate result matrices to obtain a matrix outer product result.

17. A calculation method, characterized in that: The method is performed by a computing device, the computing device comprising a computing unit and a register, and the method comprises: The calculation unit calculates the matrix inner product of the first matrix and the second matrix according to the compression identifier of the row vector in the first matrix and the compression identifier of the column vector in the second matrix to obtain multiple intermediate results, the first matrix and the second matrix are both compressed sparse format matrices, the compression identifier of the row vector is used to indicate the column coordinates of the element value in the row vector in the first sparse matrix, the compression identifier of the column vector is used to indicate the row coordinates of the element value in the column vector in the second sparse matrix, the first matrix is ​​a compressed sparse format matrix of the first sparse matrix, and the second matrix is ​​a compressed sparse format matrix of the second sparse matrix; The calculation unit accumulates the element values ​​of the same position coordinates in the multiple intermediate results to obtain a matrix inner product result; The register stores the plurality of intermediate results and the matrix inner product result.

18. A chip, characterized in that: The chip includes a processor and a power supply circuit; wherein the power supply circuit is used to supply power to the processor; the processor is used to implement the function of the computing device described in any one of claims 1 to 8, or to implement the function of the computing device described in any one of claims 9 to 15.

19. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a set of computer instructions; when the processor executes the set of computer instructions, the function of the computing device described in any one of claims 1 to 8 is implemented, or the function of the computing device described in any one of claims 9 to 15 is implemented.

20. A computing system, characterized in that: The computing system comprises a processor and a computing device, and the computing device is used to implement the function of the computing device described in any one of claims 1 to 8, or to implement the function of the computing device described in any one of claims 9 to 15.

Images