A sparse matrix dense multiplication accelerator based on in-memory computing

Abstract

The embodiment of the present application provides a sparse matrix dense multiplication accelerator based on in-memory computing, which comprises: a controller, configured to determine all non-zero values of a sparse matrix, all non-zero values of a dense vector, and row index values and column index values corresponding to each non-zero value of the sparse matrix and the dense vector according to a current computing task, wherein the non-zero values are floating point values; a CAM array, configured to store the row index values and the column index values corresponding to each non-zero value of the sparse matrix, and match the column index values and the row index values of each non-zero value of the sparse matrix with each non-zero value of the dense vector according to a row index value of each non-zero value of the dense vector, to obtain a matching result; and a MAC array, configured to store each non-zero value of the sparse matrix and each non-zero value of the dense vector in association according to the matching result of the CAM array, and perform floating point multiplication calculation between each non-zero value of the sparse matrix and each non-zero value of the dense vector stored in association, to obtain a calculation result.

Description

Technical Field

[0001] This invention relates to the field of in-memory computing, and more specifically, to a dense multiplication accelerator for sparse matrices based on in-memory computing. Background Technology

[0002] Sparse matrix-dense vector multiplication (SpMV) is a fundamental linear algebra kernel and an important computational primitive. It is used in various applications, such as scientific computing and graphics processing. Due to poor data locality and irregular memory access, the performance of traditional SpMV accelerators is often limited by memory. In-memory computing (IMC) is considered a promising technique for alleviating memory bottlenecks. However, existing IMC-based SpMV accelerators have some shortcomings that make it difficult for them to adequately accelerate SpMV.

[0003] Existing sparse matrix-dense vector multiplication accelerators based on in-memory computing technology mainly include the following two types:

[0004] Scheme 1: The scheme represented by reference [1] uses a resistive random-access memory (ReRAM) to store the sparse matrix data involved in the multiplication calculation. Then, a content addressable memory (CAM) array is used to complete the index matching, and the matched sparse matrix data is read from the memory and then the floating-point multiplier close to the memory is used to complete the SpMV calculation [1].

[0005] Scheme 2: The scheme represented by reference [2] uses a ReRAM-based Multiply-Add Computation (MAC) array to store sparse matrix data. The size of the MAC array is determined according to the exponential range of the sparse matrix data. Dense vectors are input into the MAC array and SpMV operation is performed with the stored sparse matrix data.

[0006] The first approach uses a ReRAM array that does not support floating-point representation and computation; multiplication is performed using floating-point multipliers instead of a highly parallel MAC array. This requires additional data reads, failing to fully utilize the advantages of in-memory computing and resulting in poor performance and energy efficiency. Furthermore, the use of floating-point multipliers incurs additional area and power consumption. The second approach uses a ReRAM-based MAC array to perform SpMV operations, employing fixed-point multiplication. Its array uses a dense format for data storage, not supporting sparse storage, thus storing much zero-data and incurring additional memory overhead. Moreover, this approach selects the MAC array size based on the exponent range. When the exponent range of the locally sparse matrix data varies greatly, a suitable MAC array size cannot be selected. This unstoreable and uncomputable data must be processed by the GPU, significantly reducing the accelerator's performance. Therefore, existing technologies suffer from the drawbacks of not supporting in-memory floating-point computation and sparse storage formats, leading to low computational efficiency and high memory overhead in the accelerator.

[0007] References:

[0008] [1] L.Yavits and R.Ginosar, "Sparse matrix multiplication on CAM basedaccelerator," arXiv preprint arXiv:1705.09937, 2017.

[0009] [2] B.Feinberg, UKRVengalam, N.Whitehair, S.Wang, and E.Ipek, "Enabling Scientific Computing on Memristive Accelerators," in 2018ACM / IEEE 45th AnnualInternational Symposium on Computer Architecture (ISCA), 2018, pp.367-382. Summary of the Invention

[0010] Therefore, the purpose of this invention is to overcome the shortcomings of the prior art and provide a sparse matrix dense multiplication accelerator based on in-memory computation.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] In a first aspect of the invention, a sparse matrix dense multiplication accelerator based on in-memory computation is provided. The accelerator includes: a controller, configured to determine, according to the current computation task, all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row index and column index corresponding to each non-zero value of the sparse matrix and the dense vector, wherein the non-zero values ​​are floating-point values; a CAM array, configured to store the row index and column index corresponding to each non-zero value of the sparse matrix, and match the row index of each non-zero value of the dense vector with the column index and row index of the sparse matrix to be computed, to obtain a matching result; and a MAC array, configured to associate and store each non-zero value of the sparse matrix and each non-zero value of the dense vector according to the matching result of the CAM array, and perform floating-point multiplication calculations between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector, to obtain a computation result.

[0013] In some embodiments of the present invention, the matching result includes a first matching result of all non-zero values ​​in the corresponding column of the sparse matrix calculated with the non-zero values ​​of the row corresponding to the dense vector. The MAC array performs floating-point multiplication calculation in the following manner: based on the first matching result of the CAM array, a non-zero value of the sparse matrix and a non-zero value corresponding to the dense vector calculated with the value are associated and stored in each row of the MAC array; floating-point multiplication calculation is performed on the non-zero values ​​of the sparse matrix associated and stored in each row of the MAC array and the non-zero values ​​corresponding to the dense vector according to a preset rule to obtain the calculation result of each row of the MAC array.

[0014] In some embodiments of the present invention, each non-zero value of the sparse matrix and dense vector includes an exponent and a mantissa. Each row of the MAC array includes multiple FeFET transistors, and each FeFET transistor stores one bit of the mantissa of the non-zero value of the sparse matrix. The calculation result of each row is obtained as follows: the exponent of the non-zero value of the sparse matrix stored in association in each row is added to the exponent of the corresponding non-zero value of the dense vector to obtain the total exponent value; each bit of the mantissa of the non-zero value of the dense vector stored in association in the corresponding row is input bit by bit into each FeFET transistor in that row, and each FeFET transistor performs a multiplication calculation between the input binary value and its stored binary value to obtain the mantissa calculation result of that row; the calculation result of each row is obtained based on the total exponent value and the mantissa calculation result of each row.

[0015] In some embodiments of the present invention, each FeFET transistor performs a multiplication calculation between the input binary value and its stored binary value in the following manner: each FeFET transistor calculates the mantissa result based on the multiplication between the input binary value of zero and its stored binary value, and the corresponding bit of the mantissa result is zero based on the multiplication between the input binary value of one and its stored binary value.

[0016] In some embodiments of the present invention, the MAC array includes a register file consisting of multiple registers and a first crossbar switch matrix. Each row of the first crossbar switch matrix corresponds to a register. The non-zero value of the sparse matrix and the non-zero value corresponding to the dense vector calculated with the non-zero value are stored in association between each row of the first crossbar switch matrix and its corresponding register. The association storage is performed in the following manner: the mantissa of the non-zero value of the sparse matrix is ​​stored in each row of the matrix through the first crossbar switch matrix. Multiple FeFET transistors in each row of the MAC array are arranged in each row of the first crossbar switch matrix to store the binary value of the corresponding bit of the mantissa. The exponent of the non-zero value of the sparse matrix, the exponent of the non-zero value of the dense vector calculated with the non-zero value of the sparse matrix, and the mantissa are stored in the register corresponding to the row where the mantissa of the non-zero value of the sparse matrix is ​​located through the register file.

[0017] In some embodiments of the present invention, the MAC array obtains the final calculation result in the following manner: The MAC array sequentially matches the row index values ​​stored in the CAM array that are equal to the determined search row index values ​​in ascending order, obtaining a second matching result of the non-zero values ​​corresponding to the non-zero values ​​of the dense vector calculated with all non-zero values ​​of the corresponding row of the sparse matrix; based on the second matching result, the MAC array performs multiplication calculations on all non-zero values ​​in the corresponding row of the sparse matrix and the non-zero values ​​of the dense vector stored in association with the corresponding non-zero values, obtaining the multiplication result corresponding to the corresponding row of the sparse matrix; based on the exponents of all non-zero values ​​in the corresponding row of the sparse matrix and the exponents of the non-zero values ​​of the dense vector calculated with the corresponding non-zero values, the MAC array performs floating-point multiplication and addition on the multiplication result corresponding to the corresponding row of the sparse matrix, obtaining the multiplication and addition result of the corresponding row of the sparse matrix; and finally, the MAC array obtains the final calculation result based on the multiplication and addition results corresponding to all rows of the sparse matrix.

[0018] In some embodiments of the present invention, the MAC array includes a maximum value finding circuit, a delay circuit, a digital-to-analog converter, a sample-and-hold circuit, an analog-to-digital converter, and a shift-accumulator circuit. The maximum value finding circuit sums the exponents of the non-zero values ​​of the sparse matrix stored in each register with the exponents of the non-zero values ​​of the dense vector to obtain the total exponent value corresponding to each register. The multiplication and addition results of the corresponding rows of the sparse matrix are obtained as follows: the maximum value finding circuit obtains the largest total exponent value based on all non-zero values ​​in the corresponding rows of the sparse matrix and the total exponent value corresponding to each non-zero value; the difference between the total exponent value corresponding to each non-zero value in the corresponding row and the largest total exponent value is calculated; the delay circuit calculates the difference between the total exponent value of the corresponding rows of the sparse matrix and the total exponent value corresponding to each non-zero value in the corresponding row. The difference corresponding to each non-zero value in the sparse matrix is ​​used to obtain the clock cycle for the delay calculation of each non-zero value in the corresponding row of the sparse matrix. The mantissa of the corresponding non-zero value of the dense vector, which is associated with each non-zero value in the corresponding row of the sparse matrix, is input into the second cross-switch matrix through a digital-to-analog converter. Multiplication is performed with the corresponding non-zero value in the corresponding row of the sparse matrix to obtain the calculation result for the corresponding clock cycle. The calculation result for each clock cycle is acquired and saved through a sample-and-hold circuit. The calculation result for each clock cycle is converted into a digital signal through an analog-to-digital converter. The digital signal for each clock cycle is shifted and accumulated through a shift-accumulator circuit, and combined with the maximum total exponent value, the multiplication and addition result of the corresponding row of the sparse matrix is ​​obtained.

[0019] In some embodiments of the present invention, the first cross-switch matrix includes a first memory array and a first driving circuit connected to the first memory array. Each row of the first memory array includes a plurality of first memories. Each FeFET transistor in each row of the MAC array is disposed in the first memory corresponding to that row. The first memory also includes an access transistor, the drain of which is connected to the source of the FeFET transistor. The first driving circuit includes a bit line driving circuit and a first word line driving circuit connected to the source and gate of the access transistor, respectively. The first cross-switch matrix stores the mantissa of non-zero values ​​in the following manner: the first word line driving circuit connected to the gate of the access transistor in the corresponding row is turned on, and a voltage difference is formed between the gate and source of the access transistor. Then, the binary value of the corresponding bit of the mantissa of the non-zero value of the sparse matrix is ​​transmitted to the access transistor in that row by the bit line driving circuit connected to the source. The access transistor inputs the received binary value into the FeFET transistor connected to the access transistor through its drain to store one bit of the binary value of the mantissa of the non-zero value.

[0020] In some embodiments of the present invention, the first cross switch matrix includes a sensing circuit connected to a first memory array, the first driving circuit includes a first data line driving circuit, and the gate and drain of the FeFET transistor are respectively connected to the first data line driving circuit and the sensing circuit. The first cross switch matrix performs multiplication calculation in the following manner: the FeFET transistor receives the binary value of the non-zero mantissa of the dense vector associated with the non-zero value of the sparse matrix, which is transmitted by the first data line driving circuit connected to its gate; the received binary value is multiplied by the stored binary value to obtain the binary value of the corresponding bit of the mantissa calculation result, and output through the sensing circuit connected to the drain of the FeFET transistor.

[0021] In some embodiments of the present invention, the CAM array obtains the matching result in the following manner: matching the row index value of each non-zero value of the dense vector with the column index value of each non-zero value of the sparse matrix that is equal to it, to obtain a first matching result of all non-zero values ​​of the column corresponding to the sparse matrix that need to be calculated with the non-zero values ​​of the row corresponding to the dense vector, so as to associate and store each non-zero value of the sparse matrix and each non-zero value of the dense vector according to the first matching result; matching the row index value of the determined search with the row index value of each non-zero value of the sparse matrix that is equal to it, to obtain a second matching result of the non-zero values ​​corresponding to the dense vector that need to be calculated with the non-zero values ​​of the row corresponding to the sparse matrix, so as to perform floating-point multiplication calculation between each non-zero value of the sparse matrix and the non-zero value corresponding to the dense vector according to the second matching result.

[0022] In some embodiments of the present invention, the CAM array includes a second crossbar switch matrix, the second crossbar switch matrix includes a second memory array and a second driving circuit connected to the array, each row of the second memory array includes a plurality of second memories, each second memory includes two storage units, wherein the CAM array stores the row index value and column index value of each non-zero value of the sparse matrix in the following manner: in ascending order of row index value, the second driving circuit writes the row index value, the inverted value of the row index value, the column index value and the inverted value of the column index value of the non-zero value of the sparse matrix into the corresponding row of the second memory array; wherein each row of the second memory array stores a row index value, the inverted value of the row index value, the column index value and the inverted value of the column index value of a non-zero value, and the two storage units of each second memory are respectively used to store the binary value corresponding to the corresponding bit of the row index value or the column index value and the inverted value of the binary value.

[0023] In some embodiments of the present invention, the second cross-switch matrix further includes a CAM sensitive amplifier circuit connected to a second memory array. Each memory cell includes a FeFET transistor for storing the binary value of the corresponding bit of the row index value or column index value, or storing the inverted value of the binary value, and an access transistor connected to a second driving circuit. The drain of the access transistor is connected to the gate of the FeFET transistor, and the source of the FeFET transistor is connected to the CAM sensitive amplifier circuit. The second cross-switch matrix matches the column index value of the non-zero value of the sparse matrix with it in the following manner: using the row index value of each non-zero value of the dense vector as the search key value, the corresponding search key value and the inverted value of the search key value are input into the access transistor of the second memory array through the second driving circuit; the FeFET transistor in the second memory array performs an XOR operation on the search key value and its inverted value with each column index value and its inverted value stored therein, to obtain the XOR operation result; the CAM sensitive amplifier circuit determines whether the search key value is equal to the column index value stored in the CAM array based on the XOR operation result, so as to obtain the column index value of the non-zero value of the sparse matrix that is equal to the search key value.

[0024] In a second aspect of the invention, a SpMV calculation method based on the accelerator described in the first aspect of the invention is provided, comprising: S1, determining, according to the current calculation task, all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row index and column index corresponding to each non-zero value of the sparse matrix and the dense vector, wherein the non-zero values ​​are floating-point values; S2, storing the row index and column index corresponding to each non-zero value of the sparse matrix through a CAM array, and matching the column index and row index of each non-zero value of the sparse matrix to be calculated with the row index of each non-zero value of the dense vector, to obtain a matching result; S3, storing each non-zero value of the sparse matrix and each non-zero value of the dense vector in association through a MAC array according to the matching result of the CAM array, and performing floating-point multiplication calculation between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector in the associated storage, to obtain a calculation result.

[0025] Compared with the prior art, the advantages of the present invention are as follows:

[0026] The accelerator of this invention first stores only the non-zero values ​​of the sparse matrix and dense vector, along with their corresponding row and column indices, omitting zero values, thus significantly reducing memory overhead. Secondly, the MAC array within the accelerator associates and stores each non-zero value of the sparse matrix with each non-zero value of the dense vector based on the matching results of the CAM array. This aligns each non-zero value of the dense vector with the non-zero data of the sparse matrix used in the calculation, improving the computational efficiency between the sparse matrix and dense vector. Finally, during floating-point multiplication, in-memory floating-point calculations are supported, meaning floating-point calculations between the sparse matrix and dense vector are performed directly within the MAC array, further enhancing the accelerator's computational efficiency. Furthermore, this invention does not require the assistance of a GPU or other processors and can independently complete floating-point calculations, improving the accelerator's performance. Attached Figure Description

[0027] The embodiments of the present invention will be further described below with reference to the accompanying drawings, wherein:

[0028] Figure 1 This is a schematic diagram of the structure of a sparse matrix dense multiplication accelerator based on in-memory computation according to an embodiment of the present invention;

[0029] Figure 2 This is a schematic diagram of a specific sparse matrix and dense vector in a computational task according to an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram illustrating the structure of a CAM array and the storage of row and column index values ​​in a CAM array according to an embodiment of the present invention.

[0031] Figure 4 This is a schematic diagram of the connection between the second memory array and various circuits in a second crossbar switch matrix according to an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the second memory structure of the second memory array in the second cross switch matrix according to an embodiment of the present invention, and its connection with various circuits.

[0033] Figure 6 This is a schematic diagram of the structure of a MAC array according to an embodiment of the present invention;

[0034] Figure 7 This is a schematic diagram illustrating all non-zero values ​​of a sparse matrix and a dense vector in an associative storage manner in a MAC array according to an embodiment of the present invention.

[0035] Figure 8 This is a schematic diagram of the structure of the first crossbar switch matrix in a MAC array according to an embodiment of the present invention;

[0036] Figure 9 This is a schematic diagram illustrating the structural principle of a MAC array according to an embodiment of the present invention;

[0037] Figure 10 A schematic diagram illustrating the structural principle of an accelerator according to an embodiment of the present invention for storing the mantissa and exponent of all non-zero values ​​of a sparse matrix and a dense vector, as well as the row index and column index of all non-zero values ​​of a sparse matrix.

[0038] Figure 11 (a) For the purpose of an embodiment of the present invention Figure 2 A schematic diagram illustrating the principle of operations between the first row of a sparse matrix and a dense vector.

[0039] Figure 11 (b) is a schematic diagram of the process of inputting the mantissa delay of the non-zero value of a dense vector into the second cross switch matrix according to an embodiment of the present invention;

[0040] Figure 12 This is a flowchart of a method for SpMV calculation based on an accelerator according to an embodiment of the present invention;

[0041] Figure 13 The graph shows the experimental results of the performance of an accelerator according to an embodiment of the present invention compared with conventional hardware on each dataset. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the invention.

[0043] As mentioned in the background section, existing technologies have shortcomings such as not supporting in-memory floating-point calculations and not supporting sparse storage formats, resulting in accelerators having low computational efficiency and large memory area overhead.

[0044] To address the above problems, the inventors provide a sparse matrix dense multiplication accelerator based on in-memory computation. First, the accelerator uses sparse format storage, meaning it only stores the non-zero values ​​of the sparse matrix and dense vector, along with their corresponding row and column indices. Furthermore, the non-zero values ​​are floating-point numbers, and zero values ​​are not stored, significantly reducing memory overhead. Second, the accelerator uses a CAM array to store the row and column indices of the non-zero values ​​of the sparse matrix, and matches the row index of each non-zero value in the dense vector with the column index of the sparse matrix to be computed. The MAC array in the accelerator associates and stores each non-zero value of the sparse matrix and each non-zero value of the dense vector with the matching result of the CAM array. This aligns each non-zero value of the dense vector with the non-zero data of the sparse matrix used for calculation, improving the computational efficiency between the sparse matrix and the dense vector. Finally, when performing floating-point multiplication, in-memory floating-point calculation is supported. That is, floating-point multiplication between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector is directly performed within the MAC array to obtain the calculation result, thus improving the computational efficiency of the accelerator.

[0045] To better understand the present invention, each structure of the accelerator will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] According to one embodiment of the present invention, see Figure 1 This invention provides a sparse matrix dense multiplication accelerator based on in-memory computation. The accelerator includes: a controller, configured to determine, based on the current computation task, all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row and column indices corresponding to each non-zero value of the sparse matrix and the dense vector, wherein the non-zero values ​​are floating-point values; a CAM array (ContentAddressable Memory array), configured to store the row and column indices corresponding to each non-zero value of the sparse matrix, and match the row and column indices of each non-zero value of the sparse matrix to be computed with the row indices of each non-zero value of the dense vector to obtain a matching result; and a MAC array (Multiply-Add Computation array), configured to associate and store each non-zero value of the sparse matrix and each non-zero value of the dense vector based on the matching result of the CAM array, and perform floating-point multiplication calculations between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector to obtain a computation result.

[0047] According to one embodiment of the present invention, the accelerator of the present invention can be used to perform SpMV algorithm tasks in scientific computing, such as iterative solvers, graph algorithms, machine learning, and other scientific computing tasks. Specific applications include: Example 1: In the Jacobi iteration method, to solve the system of equations AX = b, the iterative form is X^(k+1) = B*X^(k) + f, where ^(k) represents the result of the k-th iteration, ^(k+1) represents the result of the (k+1)-th iteration, B is the iteration matrix, which is also a sparse matrix, and X^(k) is a dense vector. Based on the current Jacobi iteration computing task, the accelerator determines all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row and column index values ​​of each non-zero value. Within the accelerator, the sparse matrix and the dense vector are multiplied to obtain the next iteration result. Example 2: In graph processing algorithms, an adjacency matrix (i.e., a sparse matrix) can be used to represent the structure of a graph. The row and column indices represent the start and end points of an edge, respectively, and the numerical values ​​represent the weights of those edges. A dense vector represents the vertex values ​​of each start point. A new vertex value for the endpoint is calculated by summing the products of the values ​​of all edges connected to an endpoint and the vertex values ​​of their respective start points. The accelerator, based on the current graph processing task, determines all non-zero values ​​in the sparse matrix, all non-zero values ​​in the dense vector, and the row and column indices of each non-zero value. Multiplying the sparse matrix and the dense vector within the accelerator yields a new vector. Each element in this new vector represents the vertex value of the new endpoint, thus updating the vertex value of the endpoint.

[0048] According to one embodiment of the present invention, see Figure 2 , Figure 2To illustrate the specific sparse matrix and dense vector in the current computation task, the controller determines, based on the sparse matrix and dense vector in the current computation task, all non-zero values ​​a, b, c, d, e in the sparse matrix (the value corresponding to the unlabeled square in the sparse matrix is ​​0), all non-zero values ​​f, g, h, i in the dense vector, and the row index value corresponding to each non-zero value a, b, c, d, e in the sparse matrix is ​​0, 0, 1, 2, 3, and the column index value corresponding to each non-zero value is 0, 2, 1, 0, 1. That is, the row index value corresponding to the non-zero values ​​and their corresponding row index values, and the row index value corresponding to each non-zero value f, g, h, i in the dense vector are 0, 1, 2, 3, and the corresponding column index value is 0. All non-zero values ​​in sparse matrices and dense vectors, along with their corresponding row and column indices, are represented using the COO (Coordinate) format. Each element requires a triple, represented as (row index, column index, non-zero value). Thus, a sparse matrix is ​​represented as {(0, 0, a), (0, 2, b), (1, 1, c), (2, 0, d), (3, 1, e)}, and a dense vector is represented as {(0, 0, f), (1, 0, g), (2, 0, h), (3, 0, i)}. Non-zero values ​​can be either double-precision or single-precision floating-point numbers.

[0049] According to one embodiment of the present invention, see Figure 3 The CAM array includes a second crossbar switch matrix, which in turn includes a second memory array, a second driving circuit connected to the array, a memory read-sensitive amplifier circuit, and a CAM sensitive amplifier circuit. The second driving circuit includes a second data line driving circuit and a second word line driving circuit. The CAM array stores the row and column indices of each non-zero value in the sparse matrix in ascending order of row index. The second driving circuit writes the row index, its inverse, the column index, and its inverse into the corresponding row of the second memory array. The CAM sensitive amplifier circuit connected to the second memory array determines whether a search key is equal to a column index stored in the CAM array and outputs the result. The memory read-sensitive amplifier circuit also reads the row index stored in each row of the second memory array. This row index is used to instruct each element in the final vector to be output at its corresponding position according to its row index when performing floating-point multiplication on the sparse matrix and dense vector to obtain the final vector result. In this second memory array, each row stores a non-zero row index, the inverse of that row index, a column index, and the inverse of that column index. Each row of the second memory array stores the row index and column index in binary form. Figure 3In binary, the row and column indices are represented in simplified decimal notation. In binary, 1 and 0 are inverses of each other; for example, the inverted binary number 0111 is 1000. Figure 2 Taking the sparse matrix as an example, the first row of the second memory array stores the row index 0, the inverse of the row index 0, the column index 0, and the inverse of the column index 0 for the non-zero value 'a'. The stored row index 0 and the inverse of the row index 0 together represent the stored row index 0. The second row stores the row index 0, the inverse of the row index 0, the column index 2 (represented as 10 in binary), and the inverse of the column index 2 (represented as 01 in binary). The column index 2 (represented as 10 in binary) and the inverse of the column index 2 (represented as 01 in binary) together represent the stored column index 2. According to this embodiment, the row index 1 and column index 1 of the non-zero value 'c' are stored in the third row, the row index 2 and column index 0 of the non-zero value 'd' are stored in the fourth row, and the row index 3 and column index 1 of the non-zero value 'e' are stored in the fifth row.

[0050] According to one embodiment of the present invention, the CAM array can also store the row index and column index of each non-zero value of the sparse matrix in descending order of row index value. That is, the first to fifth rows of the second memory array sequentially store the row index and column index corresponding to the non-zero values ​​e, d, c, b, and a. It should be understood that storing the row index and column index of each non-zero value of the sparse matrix in ascending or descending order of row index value is merely illustrative. Alternatively, the row index and column index of a non-zero value of the sparse matrix can be randomly stored in each row until the row index and column index of all non-zero values ​​of the sparse matrix have been stored. The present invention is not limited to this.

[0051] According to one embodiment of the present invention, see Figure 4 , Figure 4 This is a schematic diagram showing the connection between the second memory array and various circuits in the second crossbar switch matrix. Each row of the second memory array in the second crossbar switch matrix includes multiple second memories. Each second memory is connected to a second word line driver circuit (WL), a second data line driver circuit (DL), a memory read sensitive amplifier circuit (also known as a sensing line, SL), and a CAM sensitive amplifier circuit (also known as a match line, ML). In the diagram, the intersections of DL and WL / ML are not connected, and the intersections of SL and WL / ML are also not connected. Each second memory includes two storage units, which are used to store the binary value corresponding to the corresponding bit of the row index or column index value and the inverted value of that binary value, respectively. See [link to documentation]. Figure 5 , Figure 5 This is a schematic diagram of the second memory structure of the second memory array in the second crossbar switch matrix and its connection with various circuits. Figure 5 The structure within the dashed box represents a second memory. Each storage cell of the second memory includes a FeFET transistor (ferroelectric field-effect transistor) for storing the binary value of the corresponding bit of the row or column index value, or storing the inverted value of that binary value, and an access transistor connected to the second driving circuit. In the diagram, the ellipse represents the access transistor, and the rectangle represents the FeFET transistor. The drain of the access transistor and the gate of the FeFET transistor are connected to form a storage cell. The source and drain of the FeFET transistor are connected to the CAM sensitive amplifier circuit ML and the memory read sensitive amplifier circuit SL, respectively. Each row and column of the second memory array includes multiple second memories. ML connected to the sources of the two FeFET transistors in the first row of the second memory array is denoted as ML1, and ML connected to the sources of the two FeFET transistors in the nth row of the second memory array is denoted as ML2. n WL connected to the gates of the two access transistors of the first row of the second memory is denoted as WL1, and WL connected to the gates of the two access transistors of the nth row of the second memory is denoted as WL2. n The DLs connected to the sources of the two access transistors of the first column of the second memory array are denoted as DL1 and DL2, respectively. The SLs connected to the drains of the two FeFET transistors in the first column of the second memory are denoted as SL1 and SL2, respectively. And connection Access transistors and connections A FeFET transistor forms a memory cell. A FeFET transistor connected to DL1 and a FeFET transistor connected to SL1 form another memory cell. The DL transistors connected to the sources of the two access transistors in the m-th column of the second memory are denoted as DL1, DL2, DL3, DL4, DL5, DL6, DL7, DL8, DL9, DL1, DL1, DL2, DL3, DL4, DL5, DL6, DL7, DL8, DL9, DL1, DL9, DL1, DL9, DL1, DL2, DL3, DL4, DL5, DL6, DL7, DL8, DL9 ... m and The SLs connected to the drains of the two FeFET transistors in the m-th column of the second memory are respectively denoted as SL. m and

[0052] According to one embodiment of the present invention, the access transistor is a metal-oxide-semiconductor field-effect transistor (MOSFET), and the FeFET transistor in the second memory can also be a ReRAM memristor. The ReRAM memristor can also be used to store the binary value of the corresponding bit of the row index value or column index value or to store the inverted value of the binary value.

[0053] According to an embodiment of the present invention, the CAM array obtains the matching result by following steps a1 and a2:

[0054] In step a1, the row index value of each non-zero value of the dense vector is matched with the column index value of each non-zero value of the sparse matrix that is equal to it, so as to obtain the first matching result of all non-zero values ​​of the corresponding column of the sparse matrix that need to be calculated with the non-zero values ​​of the corresponding row of the dense vector, and to associate and store each non-zero value of the sparse matrix and each non-zero value of the dense vector according to the first matching result.

[0055] According to one embodiment of the present invention, the column index values ​​of the non-zero values ​​of the sparse matrix are matched with the second cross-switch matrix in the CAM array according to the following steps a11, a12, and a13:

[0056] In step a11, the row index of each non-zero value in the dense vector is used as the search key value, and the corresponding search key value and its inverse value are input into the access transistor of the second memory array through the second driving circuit.

[0057] In step a12, the FeFET transistors in the second memory array are used to perform an XOR operation on the search key value and its inverse value with the index value of each column stored therein and its inverse value to obtain the XOR operation result.

[0058] In step a13, the CAM sensitive amplifier circuit determines whether the search key value is equal to the column index value stored in the CAM array based on the XOR operation result, so as to obtain the column index value of the non-zero value of the sparse matrix that is equal to the search key value.

[0059] According to one embodiment of the present invention, the following example illustrates how to match the column index values ​​of non-zero values ​​in a sparse matrix that are equal to the row index values ​​of non-zero values ​​in a dense vector:

[0060] Example 1: When the search key value is 100, the inverted value of the search key value is 011. The search key value 100 and its inverted value are input bit-by-bit into the corresponding access transistors. The first digit 1, the first digit 0, and the first digit 0 of 100 are input from DL1, DL2, and DL3 respectively into the access transistors of each row. The first digit 0, the first digit 1, and the first digit 1 of the inverted value 011 are input from... The input is fed into the access transistors of each row to perform a bitwise XOR operation with the column index value stored in each row of the second memory array. See also Figure 5 The first two storage units in the first row of the second memory store the first value of the column index (1) and its inverse (0), respectively. According to the XOR operation rule, if the first value of the search key is 1, then inputting 1 from DL1 will result in... Inputting 0 causes a NAND operation to be performed between the input search key value and the value stored in the second memory array. If the NAND result is the same, the voltage value indicated by the first digit of the search key value and the column index value is output through ML1. If the NAND result is different, the voltage value indicated by the first digit of the search key value and the column index value is output through ML1. When all bits of the search key value and its inverted value are the same as all bits of the stored column index value, it indicates that the search key value is the same as the column index value stored in the corresponding row of the second memory array.

[0061] According to one embodiment of the present invention, Figure 2 Taking a sparse matrix and a dense vector as an example, the first matching result is as follows: the non-zero value f in the 0th row of the dense vector needs to be calculated with all non-zero values ​​in the 0th column of the sparse matrix; the non-zero value g in the 1st row of the dense vector needs to be calculated with all non-zero values ​​in the 1st column of the sparse matrix; the non-zero value h in the 2nd row of the dense vector needs to be calculated with the 2nd column of the sparse matrix; there is no column index value in the sparse matrix that is equal to the row index value 3 of the non-zero value of the dense vector; and the non-zero value i in the dense vector is not calculated with the corresponding column of the sparse matrix. Finally, each non-zero value of the sparse matrix and each non-zero value of the dense vector are associated and stored according to the first matching result. That is, the controller associates the non-zero value f in the 0th row of the dense vector with the non-zero values ​​a and d in the 0th column of the sparse matrix, the non-zero value g in the 1st row of the dense vector with the non-zero values ​​c and e in the 1st column of the sparse matrix, and the non-zero value h in the 2nd row of the dense vector with the non-zero value b in the 2nd column of the sparse matrix and stores them in the MAC array according to the first matching result of the CAM array.

[0062] In step a2, the row index value of each non-zero value of the sparse matrix that is equal to the determined search row index value is matched to obtain the second matching result of the non-zero value corresponding to the dense vector calculated with all the non-zero values ​​of the corresponding row of the sparse matrix. Then, floating-point multiplication calculation between each non-zero value of the sparse matrix and the non-zero value corresponding to the dense vector is performed based on the second matching result.

[0063] According to one embodiment of the present invention, the row index value of each non-zero value of the sparse matrix that is equal to the determined search row index value is matched, and the same XOR operation method as in the above embodiment is used to match the row index value of each non-zero value of the sparse matrix that is equal to it, so as to obtain a second matching result.

[0064] Before performing floating-point operations between sparse matrices and dense vectors using a MAC array, this invention first explains the concept of floating-point values. Floating-point values ​​are represented using binary scientific notation. For example, a floating-point value n is represented in binary scientific notation as: n = (-1). s *m*2e In this invention, 2 is the base, s is the sign number, m is the mantissa, and e is the exponent. It is stored in binary format on a computer. If the non-zero values ​​of sparse matrices and dense vectors are all floating-point values, for a 32-bit single-precision floating-point number, the highest bit is the sign bit; a sign bit of 1 indicates a positive floating-point number, and a sign bit of 0 indicates a negative floating-point number. The next 8 bits of the sign bit are the exponent e, and the last 23 bits are the mantissa m. For a 64-bit double-precision floating-point number, the highest bit is the sign bit, the next 11 bits of the sign bit are the exponent e, and the last 52 bits are the mantissa m. Before inputting the floating-point number to the accelerator for floating-point calculation, the CPU adds a value to the negative floating-point number to convert it to a positive floating-point number. After the calculation is completed, the added value is subtracted from the result. This ensures that all floating-point values ​​performed by the accelerator in this invention are positive. When two positive floating-point values ​​are used in calculation, for example, floating-point numbers... and floating-point values The result of a floating-point operation between two floating-point values ​​can be obtained by multiplying the exponents of the sum of their exponents with the mantissas of the two floating-point values. Therefore, floating-point operations can be performed by knowing only the exponent and mantissa of each non-zero value in a sparse matrix and a dense vector.

[0065] According to one embodiment of the present invention, see Figure 6 , Figure 6 This is a schematic diagram of the MAC array. The MAC array includes a register file consisting of multiple registers and a first crossbar switch matrix. Each row of the first crossbar switch matrix corresponds to one register. The first crossbar switch matrix includes a first memory array, a first driving circuit connected to the first memory array, and a sensing circuit SL connected to the first memory array. ′ The first driving circuit includes a first data line driving circuit DL. ′ Bit line drive circuit BL ′ and the first word line drive circuit WL ′ .

[0066] According to one embodiment of the present invention, each non-zero value of the sparse matrix and the dense vector includes an exponent and a mantissa. The non-zero values ​​of the sparse matrix and the corresponding non-zero values ​​of the dense vector calculated with those values ​​are stored in association with each row of a first cross-switch matrix and its corresponding register as follows: the mantissa of the non-zero value of the sparse matrix is ​​stored in each row of the first cross-switch matrix. The exponent of the non-zero value of the sparse matrix, the exponent of the non-zero value of the dense vector calculated with that non-zero value are stored in the register corresponding to the row containing the mantissa of the non-zero value of the sparse matrix via a register stack. Figure 2Taking sparse matrices and dense vectors as examples, we will now assign specific values ​​to the non-zero values ​​of sparse matrices and dense vectors, as shown in Table 1 below, and illustrate the associative storage method of MAC arrays through the following examples:

[0067] Table 1: Exponents and mantissas for all non-zero values ​​in sparse matrices and dense vectors

[0068]

[0069]

[0070] See Figure 7 In the first memory array of the first crossbar switch matrix, each row stores the mantissas (4, 2, 4, 4, 3) of the non-zero values ​​a, b, c, d, and e of the sparse matrix in row-major order. The corresponding register in each row stores the exponents (1, 2, 2, 3, 3) of the non-zero values ​​a, b, c, d, and e of the sparse matrix. Since the non-zero value f needs to be calculated with the non-zero values ​​a and d in the 0th column of the sparse matrix, the exponent 3 and mantissa 3 of f are stored in the register containing the exponent of a. Simultaneously, the exponent 3 of f also needs to be stored in the register containing the exponent of a. The exponent 3 and mantissa 3 are stored in the register where the exponent of d is located. The non-zero value g in the first row of the dense vector needs to be calculated with the non-zero values ​​c and e in the first column of the sparse matrix. The exponent 2 and mantissa 3 of g are stored in the register where the exponent of c is located. At the same time, the exponent 2 and mantissa 3 of g also need to be stored in the register where the exponent of e is located. The non-zero value h in the second row of the dense vector needs to be calculated with the non-zero value b in the second column of the sparse matrix. The exponent 4 and mantissa 2 of h are stored in the register where the exponent of b is located.

[0071] According to one embodiment of the present invention, see Figure 8 , Figure 8 This is a detailed structural diagram of the first crossbar switch matrix in the MAC array. Each row and column of the first memory array of the first crossbar switch matrix includes multiple first memories. Figure 8 The dashed box in the middle represents a first memory structure of the first memory array. The first memory includes FeFET transistors and access transistors. The FeFET transistors are shown in the rectangular box, and the access transistors are shown in the elliptical box. The drain of the access transistor is connected to the source of the FeFET transistor. The source and gate of the access transistor are respectively connected to the bit line driving circuit BL. ′ and the first word line drive circuit WL ′ The gate and drain of the FeFET transistor are respectively connected to the first data line drive circuit DL. ′ and sensing circuit SL ′ The DL connected to the gate of the FeFET transistor of the first memory in the first row of the first memory array. ′ Represented as DL1′ The gate of the FeFET transistor of the first memory in the nth row is connected to DL. ′ Represented as DL n ′ SL connected to the drain of the FeFET transistor of the first memory in the first column ′ Represented as SL1 ′ SL connected to the drain of the FeFET transistor of the first memory in the second column ′ Represented as SL2 ′ WL, which is connected to the gate of the access transistor of the first row of the first memory ′ Represented as WL1 ′ WL, which is connected to the gate of the access transistor of the first memory in the nth row ′ Represented as WL n ′ BL, which is connected to the source of the access transistor of the first column of the first memory ′ Represented as BL1 ′ BL, which is connected to the source of the access transistor of the first memory in the second column ′ Represented as BL2 ′ The first cross-switch matrix stores the mantissa of non-zero values ​​as follows: The first word line driving circuit connected to the gate of the access transistor in the corresponding row is turned on, creating a voltage difference between the gate and source of the access transistor. Then, the binary value of the corresponding bit of the mantissa of the non-zero value of the sparse matrix is ​​transmitted to the access transistor in that row via the bit line driving circuit connected to the source. The access transistor inputs the received binary value into the FeFET transistor connected to it through its drain to store one bit of the binary value of the mantissa of the non-zero value.

[0072] According to one embodiment of the present invention, see [reference needed]. Figure 9 , Figure 9 This is a schematic diagram of the structural principle of a MAC array. The MAC array includes the register file and the first crossbar switch matrix described in the above embodiments. The MAC array also includes a maximum value finding circuit, a delay circuit, a digital-to-analog converter (DAC), a sample-and-hold circuit, an analog-to-digital converter (ADC), and a shift-accumulator circuit. The sample-and-hold circuit is connected to the sensing circuit. Figure 9 The digital-to-analog converter (DAC), the first word line driver circuit, and the first data line driver circuit are integrated into one module. It should be understood that the DAC can also be a separate module, and this invention does not limit this.

[0073] According to one embodiment of the present invention, the non-zero values ​​of the sparse matrix stored in each row of the MAC array can be multiplied by floating-point operations with the corresponding non-zero values ​​of the dense vector to obtain the calculation result of each row of the MAC array. If the row of the sparse matrix contains more than two non-zero values ​​that need to be calculated with the non-zero values ​​of the dense vector, then each non-zero value of that row of the sparse matrix and the corresponding non-zero value of the dense vector stored in its associated array are multiplied by floating-point operations in the corresponding row of the MAC array, and the output results of each row of the MAC array are added together to obtain the result of the floating-point calculation of the first row of the sparse matrix and the dense vector. Figure 2 The first row of the sparse matrix contains two or more non-zero values ​​a and b, where a needs to be multiplied by f and b needs to be multiplied by h. In this case, the floating-point multiplication of a and f, which are stored in the corresponding row of the MAC array, is performed, as well as the floating-point multiplication of b and h, which are stored in the corresponding row of the MAC array. The results of the floating-point multiplication of the two rows are then added together with the exponent sum of a and f and the exponent sum of b and h stored in the corresponding row. This gives the result of the floating-point calculation of the first row of the sparse matrix and the dense vector.

[0074] According to one embodiment of the present invention, the MAC array performs floating-point multiplication calculations according to the following steps b1 and b2 to obtain the calculation results for each row of the MAC array:

[0075] In step b1, based on the matching result of the CAM array, a non-zero value of the sparse matrix and a non-zero value corresponding to the dense vector calculated with that value are associated and stored in each row of the MAC array.

[0076] According to one embodiment of the present invention, the above example will be Figure 2 After assigning specific values ​​to the non-zero values ​​of the sparse matrix and dense vector, based on the matching results of the CAM array, and using the storage methods of the CAM array for the row and column indices of all non-zero values ​​of the sparse matrix and the MAC array for the mantissa and exponent of all non-zero values ​​of the sparse matrix and dense vector in the above embodiments, combined with... Figure 3 , Figure 7 and Figure 9 , to obtain Figure 10 The diagram illustrates the storage of the mantissas and exponents of all non-zero values ​​in a sparse matrix and a dense vector, as well as the row and column indices of all non-zero values ​​in the sparse matrix, within the accelerator. Figure 10The first row of the MAC array stores 'a' and 'f', which is calculated with 'a'; the second row stores 'b' and 'h', which is calculated with 'b'; the third row stores 'c' and 'g', which is calculated with 'c'; the fourth row stores 'd' and 'f', which is calculated with 'd'; and the fifth row stores 'e' and 'g', which is calculated with 'e'. In each row of the MAC array, the two non-zero values ​​involved in the calculation are aligned, and floating-point calculations are performed directly within the MAC array without the assistance of the GPU, which greatly improves the computational efficiency.

[0077] In step b2, according to preset rules, the non-zero values ​​of the sparse matrix stored in each row of the MAC array are multiplied by floating-point numbers with the non-zero values ​​of the dense vector to obtain the calculation result for each row.

[0078] According to one embodiment of the present invention, each non-zero value of the sparse matrix and the dense vector includes an exponent and a mantissa, and each row of the MAC array includes a plurality of FeFET transistors, each FeFET transistor storing a one-bit binary value of the mantissa of the non-zero value of the sparse matrix, wherein the MAC array obtains the calculation result of each row according to the following steps b21, b22 and b23:

[0079] In step b21, the exponents of the non-zero values ​​of the sparse matrix stored in each row are added to the exponents of the non-zero values ​​of the dense vector to obtain the total exponent value.

[0080] In step b22, the binary value of each bit of the mantissa of the non-zero value of the dense vector associated with the corresponding row is input into each FeFET transistor of the row. Each FeFET transistor performs a multiplication calculation between the input binary value and its stored binary value to obtain the mantissa calculation result of the row.

[0081] According to one embodiment of the present invention, each FeFET transistor performs multiplication calculation as follows: the FeFET transistor receives the binary value of the non-zero mantissa of the dense vector associated with the non-zero value of the sparse matrix, which is transmitted by the first data line driving circuit connected to its gate; the received binary value is multiplied by the stored binary value to obtain the binary value of the corresponding bit of the mantissa calculation result, which is output through the sensing circuit connected to the drain of the FeFET transistor.

[0082] According to one embodiment of the present invention, each FeFET transistor performs a multiplication calculation on the received binary value and its stored binary value as follows: each FeFET transistor calculates the mantissa result by multiplying the binary value with the stored binary value when the input is zero, and calculates the mantissa result by multiplying the binary value with the stored binary value when the input is one, with the corresponding bit being its stored binary value.

[0083] In step b23, the calculation result of each row of the MAC array is obtained based on the total exponent value and mantissa calculation result of each row. This invention adds the exponents of the non-zero values ​​of the associatively stored sparse matrix and the exponents of the non-zero values ​​corresponding to the dense vectors calculated with those values, multiplies the mantissas, and concatenates the total exponent value and the mantissa calculation result to obtain two non-zero floating-point calculation results. This allows the accelerator to store and calculate the exponents of the non-zero values ​​of the sparse matrix within any exponent range, resulting in a wide range of applications. Furthermore, the accelerator can independently complete floating-point calculations without GPU assistance, leading to high computational efficiency.

[0084] According to one embodiment of the present invention, the MAC array obtains the final calculation result between the sparse matrix and the dense vector by following the steps c1, c2, c3 and c4:

[0085] c1. Match the row index values ​​stored in the CAM array in ascending order according to the determined search row index values ​​to obtain the second matching result of the non-zero values ​​corresponding to the dense vector calculated with all non-zero values ​​of the corresponding row of the sparse matrix.

[0086] c2. Based on the second matching result, multiply all non-zero values ​​in the corresponding row of the sparse matrix with the non-zero values ​​of the dense vector stored in association with the corresponding non-zero values ​​to obtain the multiplication result corresponding to the corresponding row of the sparse matrix.

[0087] According to one embodiment of the present invention, when searching for row index values ​​stored in the CAM array, the row index value to be searched and its inverse are input into each row of the CAM array in the same manner as in the above embodiment. An XOR operation is performed with the row index values ​​stored in each row to obtain the XOR result. Based on the XOR result, a row index value identical to the searched row index value can be obtained, thereby obtaining a second matching result of the non-zero values ​​corresponding to the non-zero values ​​of the dense vector calculated with all non-zero values ​​of the corresponding row of the sparse matrix. The signal corresponding to this second matching result is received by the controller, which determines which non-zero values ​​of the associated stored dense vector and the non-zero values ​​of the sparse matrix participate in the calculation. Figure 2Taking an example, when matching sequentially according to the determined search row index values ​​0, 1, 2, 3, when the search row index value is determined to be 0, the non-zero values ​​'a' and 'b' corresponding to row index value 0 are obtained. This is achieved by multiplying the non-zero value 'a' in the 0th row of the sparse matrix with the non-zero value 'f' of the dense vector associated with the corresponding non-zero value 'a', and multiplying the non-zero value 'b' in the 0th row with the non-zero value 'h' of the dense vector associated with the corresponding non-zero value 'b'. When the search row index value is determined to be 1, the non-zero value 'c' corresponding to row index value 1 is obtained. This is achieved by multiplying the non-zero value 'c' in the 1st row of the sparse matrix with the non-zero value 'g' of the dense vector associated with the corresponding non-zero value 'c'. It should be understood that this is only for illustration, and the search can also be performed sequentially from largest to smallest or randomly. This invention is not limited to this, until all row index values ​​stored in the CAM array have been searched.

[0088] c3. Based on the exponents of all non-zero values ​​in the corresponding row of the sparse matrix and the exponents of the non-zero values ​​of the dense vector calculated by floating-point multiplication with the corresponding non-zero values, perform floating-point multiplication and addition on the corresponding multiplication results in the corresponding row of the sparse matrix to obtain the multiplication and addition results of the corresponding row of the sparse matrix.

[0089] According to one embodiment of the present invention, the maximum value finding circuit sums the exponents of the non-zero values ​​of the sparse matrix and the exponents of the non-zero values ​​of the dense vector stored in each register to obtain the total exponent value corresponding to each register, and obtains the multiplication and addition results of the corresponding rows of the sparse matrix in the following manner:

[0090] The maximum value is found by a circuit that finds the maximum total exponent value based on all non-zero values ​​in the corresponding row of the sparse matrix and the total exponent value corresponding to each non-zero value. The difference between the total exponent value corresponding to each non-zero value in the corresponding row and the maximum total exponent value is calculated. The delay circuit obtains the clock cycle for delaying the calculation of each non-zero value in the corresponding row of the sparse matrix based on the difference. The mantissa of the corresponding non-zero value of the dense vector associated with each non-zero value in the corresponding row of the sparse matrix is ​​delayed by the corresponding clock cycle and input into the second cross-switch matrix by a digital-to-analog converter. Multiplication is performed with the corresponding non-zero value in the corresponding row of the sparse matrix to obtain the calculation result for the corresponding clock cycle. The calculation result for each clock cycle is acquired and saved by a sample-and-hold circuit. The calculation result for each clock cycle is converted into a digital signal by an analog-to-digital converter. The digital signal for each clock cycle is shifted and accumulated by a shift-accumulator circuit, and combined with the maximum total exponent value, the multiplication and addition result of the corresponding row of the sparse matrix is ​​obtained.

[0091] According to one embodiment of the present invention, see Figure 11 (a) and Figure 11 (b), Figure 11 (a) is for Figure 2 A schematic diagram illustrating the principle of operations between the first row of a sparse matrix and a dense vector. Figure 11 (b) is a schematic diagram illustrating the process of inputting the non-zero mantissa delay of a dense vector into the second crossbar switch matrix corresponding to the clock cycle. For example... Figure 11 As shown in (a), for example, when searching for row index value 0 in the CAM array, the row index values ​​of the first and second rows match. Therefore, the first and second rows in the corresponding MAC array participate in the calculation. The maximum value finding circuit retrieves the largest total exponent value from the registers corresponding to these two rows, i.e., comparing the two total exponent values ​​E1 = 4 and E2 = 6, determining the largest total exponent value to be 6. Since the difference between the first row's total exponent value of 4 and the largest total exponent value is 2, the clock cycle for the delay calculation of the first row of the MAC array is 2, and the clock cycle for the delay calculation of the second row of the MAC array is 0. Therefore, if... Figure 11 As shown in (b), the mantissas f and h of the non-zero values ​​in the dense vector are input bit by bit into each FeFET transistor in that row according to the clock cycle of the delay calculation. First, the first bit 1 of the mantissa 2 (binary value 10) of h in the register corresponding to the second row is input into each FeFET transistor in the second row of the first cross-switch matrix in the first cycle. It is multiplied with the mantissa 2 (binary value 10, 1, 0, and stored in two FeFET transistors in the second row of the first cross-switch matrix) stored in the second row of the first cross-switch matrix, so that the input 1 is operated with each bit of the binary value 10, resulting in... In the same way, for the next cycle, the second bit of the mantissa 2 of the input h is 0 and operated on with each bit of the binary value 10 to get 0. The first bit of the mantissa 3 (binary value 11) of f in the register corresponding to the first row of the MAC array is input in the third cycle and multiplied with 4 (binary value 100) stored in the first row of the first crossbar switch matrix to get the corresponding result 100. The second bit of the mantissa 3 (binary value 11) of f is 1 in the fourth cycle and multiplied with 4 (binary value 100) stored in the first row of the first crossbar switch matrix to get the corresponding result 100. The calculation result of each clock cycle is obtained and saved by the sample-and-hold circuit. The analog-to-digital converter (ADC) converts the calculation result of each clock cycle into a digital signal. The digital signal of each clock cycle is shifted and accumulated by the shift-accumulator circuit to get the mantissa 11100. Combined with the maximum exponent value 6 obtained from the first and second rows of the corresponding MAC array, a multiplication and addition result with exponent 110 and mantissa 11100 is obtained. Figure 2The second row of the sparse matrix only includes non-zero values ​​c. Therefore, after the calculation of the first row of the sparse matrix is ​​completed, the floating-point operation between the non-zero values ​​c and g in the third row of the MAC array is performed. Combined with the total exponent value of 4 of the non-zero values ​​c and g, the multiplication and addition result of the second row of the sparse matrix is ​​directly obtained. The multiplication and addition result of the third row of the sparse matrix is ​​obtained in the manner described in the above embodiment.

[0092] c4. Obtain the final calculation result based on the multiplication and addition results corresponding to all rows of the sparse matrix. Figure 2 For example, if the multiplication and addition results of the three rows of a sparse matrix are x1, x2, and x3 respectively, since multiplying a 4*4 sparse matrix by a 4*1 dense vector results in a 4*1 vector, the remaining positions are automatically padded with 0s. Therefore, the final calculated result is a vector.

[0093] According to another embodiment of the present invention, the MAC array can store the exponent and mantissa of the non-zero values ​​of the sparse matrix in the registers of the register file, and store the exponent of the non-zero values ​​of the dense vector corresponding to the non-zero values ​​of the sparse matrix. The mantissa of the non-zero values ​​of the dense vector is stored in the row of the first cross-switch matrix corresponding to the mantissa of the non-zero values ​​of the dense vector through a first cross-switch matrix. During floating-point operations, the mantissa of the non-zero values ​​of the sparse matrix stored in the register is input into the first cross-switch matrix and multiplied with the mantissa of the non-zero values ​​of the dense vector stored in the first cross-switch matrix to obtain the multiplication result. The exponent of the non-zero values ​​of the dense vector is then added to the exponent of the non-zero values ​​of the sparse matrix to obtain the exponent sum. Finally, the calculation result for that row of the MAC array is obtained based on the multiplication result and the exponent sum.

[0094] According to an embodiment of the present invention, a SpMV calculation method based on the accelerator described in the above embodiment is provided, as shown in the figure, including steps S1, S2 and S3:

[0095] S1. Based on the current computation task, the controller determines all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row index and column index corresponding to each non-zero value of the sparse matrix and the dense vector, and the non-zero values ​​are floating-point values.

[0096] S2. Store the row index and column index of each non-zero value of the sparse matrix using a CAM array. Match the row index of each non-zero value of the dense vector with the column index and row index of each non-zero value of the sparse matrix to be calculated, and obtain the matching result.

[0097] S3. Based on the matching results of the CAM array, the MAC array associates and stores each non-zero value of the sparse matrix and each non-zero value of the dense vector, and performs floating-point multiplication between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector to obtain the calculation result.

[0098] To verify the effectiveness of the present invention, the inventors conducted the following experiments.

[0099] The experiment used 20 datasets, which were sparse matrices from the SuiteSparse Matrix dataset: 2cubes_sphere, ASIC_100k, bcircuit, crystalm03, epb3, finan512, G2_circuit, GaAH6, nasasrb, ns3Da, Pres_Poisson, qa8fm, ship_001, Si34H36, thermomomech_TC, torso2, Trefethen_20000, venkat25, wang3, and xenon1. SpMV computations were performed on CPUs, GPUs, and the accelerator of this invention. The performance improvement of the accelerator was compared with that of traditional hardware CPUs and GPUs. The final experimental results are shown in [link to experimental results]. Figure 13 , Figure 13 This is a schematic diagram illustrating the experimental results of the performance of the accelerator of this invention compared to traditional hardware on each dataset. Figure 13 The x-axis represents the names of the selected datasets, the last one is the average performance (Geomean), and the y-axis represents the improvement factor. Accelerator performance includes: speedup relative to CPU (speedup factor), speedup relative to GPU, energy saving relative to CPU (energy saving factor), and energy saving relative to GPU. The Geomean represents the geometric mean improvement in speedup and energy saving across the 20 datasets (average performance). Specifically, based on the average performance (Geomean), this invention achieves a speedup of 13 times compared to the CPU baseline while consuming only [amount missing]. Compared to the GPU baseline, it offers a speedup of 47.84 times while consuming only [amount missing] energy. The CPU baseline data uses the SpMV function from the Intel MKL library and runs on an Intel Xeon Platinum 8260 CPU, while the GPU baseline data uses the SpMV function from the cuSPARSE library and runs on an NVIDIA Tesla V100 GPU.

[0100] It should be noted that although the steps are described in a specific order above, it does not mean that the steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently, or even in a different order, as long as the required function can be achieved.

[0101] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0102] Computer-readable storage media can be tangible devices that hold and store instructions for use by an instruction execution device. Computer-readable storage media can be, for example, including but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof.

[0103] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A sparse matrix dense multiplication accelerator based on in-memory computation, characterized in that, The accelerator includes: The controller is used to determine, based on the current computation task, all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row index and column index corresponding to each non-zero value of the sparse matrix and the dense vector, where the non-zero values ​​are floating-point values. The CAM array is used to store the row index and column index of each non-zero value in the sparse matrix. It matches the row index of each non-zero value in the dense vector with the column index and row index of each non-zero value in the sparse matrix to be calculated, and obtains the matching result. The MAC array is used to associate and store each non-zero value of the sparse matrix and each non-zero value of the dense vector according to the matching result of the CAM array, and to perform floating-point multiplication between each non-zero value of the associated sparse matrix and the corresponding non-zero value of the dense vector to obtain the calculation result.

2. The accelerator according to claim 1, characterized in that, The matching result includes the first matching result of all non-zero values ​​in the corresponding columns of the sparse matrix calculated with the non-zero values ​​of the corresponding rows of the dense vector. The MAC array performs floating-point multiplication calculations in the following manner: Based on the first matching result of the CAM array, a non-zero value of the sparse matrix and a non-zero value corresponding to the dense vector calculated with that value are stored together in each row of the MAC array. According to preset rules, the non-zero values ​​of the sparse matrix stored in each row of the MAC array are multiplied by floating-point numbers with the non-zero values ​​of the dense vector to obtain the calculation result of each row of the MAC array.

3. The accelerator according to claim 2, characterized in that, Each non-zero value in the sparse matrix and dense vector includes an exponent and a mantissa. Each row of the MAC array includes multiple FeFET transistors, each FeFET transistor storing one bit of the mantissa of the non-zero value in the sparse matrix. The calculation result for each row is obtained as follows: The total exponent value is obtained by adding the exponents of the non-zero values ​​of the sparse matrix stored in association in each row to the exponents of the non-zero values ​​of the dense vector. The binary value of each bit of the mantissa of the non-zero value of the dense vector associated with the corresponding row is input into each FeFET transistor in that row. Each FeFET transistor performs a multiplication between the input binary value and its stored binary value to obtain the mantissa calculation result of that row. The calculation result for each row is obtained by calculating the total index value and the last digit of each row.

4. The accelerator according to claim 3, characterized in that, Each FeFET transistor performs the multiplication calculation between the input binary value and its stored binary value in the following manner: Each FeFET transistor calculates the mantissa by multiplying the binary value with its stored binary value when the input is zero, and the corresponding bit of the mantissa calculation result is zero when multiplying the binary value with its stored binary value when the input is one.

5. The accelerator according to claim 3, characterized in that, The MAC array includes a register file consisting of multiple registers and a first crossbar switch matrix. Each row of the first crossbar switch matrix corresponds to a register. The non-zero values ​​of the sparse matrix and the non-zero values ​​of the dense vectors calculated with those values ​​are stored in association between each row of the first crossbar switch matrix and its corresponding register. The association storage is performed in the following manner: The mantissa of the non-zero values ​​of the sparse matrix is ​​stored in each row of the matrix by a first cross-switch matrix, wherein multiple FeFET transistors in each row of the MAC array are configured in each row of the first cross-switch matrix to store the binary values ​​of the corresponding bits of the mantissa. The exponents of the non-zero values ​​of the sparse matrix, the exponents of the non-zero values ​​of the dense vector calculated with the non-zero values ​​of the sparse matrix, and the mantissas are stored in the register corresponding to the row containing the mantissas of the non-zero values ​​of the sparse matrix through the register file.

6. The accelerator according to claim 5, characterized in that, The MAC array obtains the final calculation result in the following manner: The search results are obtained by sequentially matching the row index values ​​stored in the CAM array with the row index values ​​that are equal to the search row index values ​​in ascending order, and obtaining the second matching result of the non-zero values ​​corresponding to the non-zero values ​​of the dense vector calculated with all the non-zero values ​​of the corresponding row of the sparse matrix. Based on the second matching result, multiply all non-zero values ​​in the corresponding row of the sparse matrix with the non-zero values ​​of the dense vector stored in association with the corresponding non-zero values ​​to obtain the multiplication result corresponding to the corresponding row of the sparse matrix. Based on the exponents of all non-zero values ​​in the corresponding row of the sparse matrix and the exponents of the non-zero values ​​of the dense vector calculated by floating-point multiplication with the corresponding non-zero values, the corresponding multiplication results in the corresponding row of the sparse matrix are multiplied by floating-point addition to obtain the multiply-add result of the corresponding row of the sparse matrix. The final calculation result is obtained by multiplying and adding the results of all rows of the sparse matrix.

7. The accelerator according to claim 6, characterized in that, The MAC array includes a maximum value finding circuit, a delay circuit, a digital-to-analog converter, a sample-and-hold circuit, an analog-to-digital converter, and a shift-accumulator circuit. The maximum value finding circuit sums the exponents of the non-zero values ​​of the sparse matrix and the non-zero values ​​of the dense vector stored in each register to obtain the total exponent value for each register. The multiplication and addition results for the corresponding rows of the sparse matrix are then obtained as follows: The maximum value is obtained by finding the maximum value circuit based on all non-zero values ​​in the corresponding row of the sparse matrix and the total exponent value corresponding to each non-zero value. The difference between the total exponent value corresponding to each non-zero value in the corresponding row and the maximum total exponent value is calculated. The clock cycle for delaying each non-zero value in the corresponding row of the sparse matrix is ​​obtained by using a delay circuit based on the difference between each non-zero value in the corresponding row of the sparse matrix. The mantissa of the corresponding non-zero value of the dense vector, which is associated with each non-zero value in the corresponding row of the sparse matrix, is delayed by the clock cycle and input into the second cross-switch matrix. The matrix is ​​then multiplied with the corresponding non-zero value in the corresponding row of the sparse matrix to obtain the calculation result for the corresponding clock cycle. The calculation results for each clock cycle are acquired and saved through a sample-and-hold circuit; The calculation results for each clock cycle are converted into digital signals using an analog-to-digital converter; The digital signal for each clock cycle is shifted and accumulated by a shift-accumulator circuit, and the result of the multiplication and addition of the corresponding row of the sparse matrix is ​​obtained by combining the maximum total exponent value.

8. The accelerator according to claim 5, characterized in that, The first cross-switch matrix includes a first memory array and a first driving circuit connected to the first memory array. Each row of the first memory array includes multiple first memories. Each FeFET transistor in each row of the MAC array is disposed in the first memory corresponding to that row. The first memory also includes access transistors, with the drain of the access transistors connected to the source of the FeFET transistors. The first driving circuit includes a bit line driving circuit and a first word line driving circuit connected to the source and gate of the access transistors, respectively. The first cross-switch matrix stores the mantissa of non-zero values ​​in the following manner: The first word line driving circuit connected to the gate of the access transistor of the corresponding row is turned on, so that a voltage difference is formed between the gate and the source of the access transistor. Then, the binary value of the corresponding bit of the mantissa of the non-zero value of the sparse matrix is ​​sent to the access transistor of that row by the bit line driving circuit connected to the source. The access transistor receives the binary value through its drain and inputs it into the FeFET transistor connected to the access transistor to store one bit of the mantissa of the non-zero value.

9. The accelerator according to claim 8, characterized in that, The first cross-switch matrix includes a sensing circuit connected to the first memory array, and the first driving circuit includes a first data line driving circuit. The gate and drain of the FeFET transistor are respectively connected to the first data line driving circuit and the sensing circuit. The first cross-switch matrix performs multiplication calculations in the following manner: The FeFET transistor receives the binary value of the non-zero mantissa of the dense vector, which is associated with the non-zero value of the sparse matrix, from the first data line driving circuit connected to its gate. The received binary value is multiplied by the stored binary value to obtain the binary value of the corresponding bit of the mantissa calculation result, which is then output through the sensing circuit connected to the drain of the FeFET transistor.

10. The accelerator according to claim 1, characterized in that, The CAM array obtains the matching results in the following manner: Based on the row index value of each non-zero value of the dense vector, match the column index value of each non-zero value of the sparse matrix that is equal to it, and obtain the first matching result of all non-zero values ​​of the corresponding column of the sparse matrix that need to be calculated with the non-zero values ​​of the corresponding row of the dense vector. Then, store each non-zero value of the sparse matrix and each non-zero value of the dense vector in association based on the first matching result. Based on the determined search row index value, match the row index value of each non-zero value of the sparse matrix that is equal to it, and obtain the second matching result of the non-zero value corresponding to the dense vector calculated with all the non-zero values ​​of the corresponding row of the sparse matrix. Then, perform floating-point multiplication calculation between each non-zero value of the sparse matrix and the non-zero value corresponding to the dense vector according to the second matching result.

11. The accelerator according to claim 10, characterized in that, The CAM array includes a second crossbar switch matrix, which includes a second memory array and a second driving circuit connected to the array. Each row of the second memory array includes multiple second memories, and each second memory includes two storage cells. The CAM array stores the row index and column index of each non-zero value of the sparse matrix in the following manner: In ascending order of row index values, the second driving circuit writes the row index value, the inverted value of the row index value, the column index value, and the inverted value of the column index value of the sparse matrix into the corresponding rows of the second memory array. In this second memory array, each row stores a non-zero row index value, the inverted value of the row index value, a column index value, and the inverted value of the column index value. The two storage units of each second memory are used to store the binary value corresponding to the corresponding bit of the row index value or column index value and the inverted value of the binary value, respectively.

12. The accelerator according to claim 11, characterized in that, The second cross-switch matrix further includes a CAM sensitive amplifier circuit connected to the second memory array. Each memory cell includes a FeFET transistor for storing the binary value of the corresponding bit of the row index value or column index value, or storing the inverted value of the binary value, and an access transistor connected to the second driving circuit. The drain of the access transistor is connected to the gate of the FeFET transistor, and the source of the FeFET transistor is connected to the CAM sensitive amplifier circuit. The second cross-switch matrix matches the column index values ​​of the non-zero values ​​of the sparse matrix that are equal to it in the following manner: The row index of each non-zero value in the dense vector is used as the search key value. The corresponding search key value and the inverted value of the search key value are input into the access transistor of the second memory array through the second driving circuit. The FeFET transistors in the second memory array are used to perform an XOR operation on the search key value and its inverse value with the index value of each column read and its inverse value to obtain the XOR operation result; The CAM sensitive amplifier circuit determines whether the search key value is equal to the column index value stored in the CAM array based on the XOR operation result, so as to obtain the column index value of the non-zero value of the sparse matrix that is equal to the search key value.

13. A method for calculating SpMV based on the accelerator according to any one of claims 1-12, characterized in that, include: S1. Based on the current computation task, the controller determines all non-zero values ​​of the sparse matrix, all non-zero values ​​of the dense vector, and the row index and column index corresponding to each non-zero value of the sparse matrix and the dense vector, and the non-zero values ​​are floating-point values. S2. Store the row index and column index of each non-zero value of the sparse matrix using a CAM array. Match the row index of each non-zero value of the dense vector with the column index and row index of each non-zero value of the sparse matrix to be calculated, and obtain the matching result. S3. Based on the matching results of the CAM array, the MAC array associates and stores each non-zero value of the sparse matrix and each non-zero value of the dense vector, and performs floating-point multiplication between each non-zero value of the sparse matrix and the corresponding non-zero value of the dense vector to obtain the calculation result.

Images