Special hardware structure for left matrix rearrangement of sparse matrix multiplication
Through the dedicated hardware structure of left matrix rearrangement of sparse matrix multiplication, local sensitive hashing technology is used to optimize the row sequence or column sequence of the left matrix multiplication matrix, which solves the problem of low memory access efficiency in the sparse matrix multiplication hardware accelerator and improves the computing performance and data multiplexing rate.
Patent Information
- Application Number
- CN202411084021.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-22
AI Technical Summary
The existing sparse matrix multiplication hardware accelerator lacks the real-time rearrangement capability of sparse matrix left matrix, resulting in low memory access efficiency and poor computing performance. Especially when using the Gustavson algorithm, it fails to fully utilize the data multiplexing potential of hardware caches. .
A special hardware structure for sparse matrix multiplication left matrix rearrangement is designed, including a signature generation module, a bucket number calculation module, a cache register group and a control module. The signature vector and bucket number are generated through local sensitive hashing technology to optimize the row sequence or column. Rearrangement of sequences to improve cached data multiplexing rate.
Through a dedicated hardware structure, the sparse matrix left matrix is rearranged in real time, which significantly improves the memory access efficiency and calculation performance of sparse matrix multiplication, reduces memory access latency, and improves the performance of the Gustavson algorithm in hardware accelerators.
Smart Images

Figure CN119003962A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of hardware accelerator design, and in particular relates to a special hardware structure for rearranging left matrices of sparse matrix multiplication. Background Art
[0002] With the development of information technology such as machine learning, neural networks, and big data, the demand for converting information into matrices for storage and calculation is increasing. Sparse matrices are an important type of matrix in information life. The proportion of its non-zero elements in the matrix is low, so its calculation and storage are different from dense matrices. Using general dense matrix multiplication calculations will cause a lot of time to be spent on multiplication of 0, so special optimization algorithms are needed.
[0003] How to improve the computational efficiency of sparse matrices is an important research direction for hardware acceleration algorithms. There are three main compressed storage methods for sparse matrix storage formats: COO, CSR, and CSC. All three formats contain three arrays. For COO, they store the row index, column index, and value of the sparse matrix respectively; for CSR, they store the row pointer, column index, and value respectively; for CSC, they store the column pointer, row index, and value respectively. The main advanced sparse matrix hardware accelerator architecture is designed for sparse matrices in CSR and CSC formats. The main implementation methods of sparse matrix multiplication are the inner product method, the outer product method, and the Gustavson algorithm. The Gustavson algorithm has a natural advantage and optimization potential in memory access efficiency, so some scholars believe that it is the most suitable sparse matrix multiplication algorithm for hardware deployment.
[0004] Gustavson algorithm multiplies the left and right matrices of sparse matrix multiplication by row, and also obtains the result by row. Specifically, take a row of the left matrix, row number r, and the vector corresponding to the column index of its non-zero elements is A r , take the row number in the right matrix as A r Multiply all rows of the values of the elements in ; the same operation is performed on the column-major format.
[0005] Since the column index distribution of the non-zero elements of the rows of the left matrix is unknown, it is possible to rearrange the rows of the left matrix so that the rows of the left matrix used for multiplication have more identical column indices. In this way, they can be read directly from the cache in the hardware, reducing memory access and latency and improving computing efficiency.
[0006] However, although existing rearrangement algorithms can rearrange the left matrix of sparse matrix multiplication and improve data reuse, they are often implemented through software methods and have a time complexity higher than the matrix multiplication calculation itself, and there is no hardware acceleration solution yet.
[0007] Therefore, it is a problem worthy of attention how to accelerate the left matrix rearrangement of sparse matrix multiplication through dedicated hardware so that the hardware structure can adapt to the existing sparse matrix hardware accelerator architecture designed based on the Gustavson algorithm and perform the left matrix rearrangement operation in real time, thereby reducing memory access and improving performance. Summary of the invention
[0008] The purpose of the present invention is to provide a special hardware structure for rearranging the left matrix of sparse matrix multiplication, which can efficiently and in real time rearrange the left matrix of sparse matrix multiplication, improve the memory access efficiency of the sparse matrix multiplication hardware accelerator, and improve the computing performance.
[0009] In order to achieve the above purpose, the technical solution adopted by the present invention is:
[0010] A special hardware structure for sparse matrix multiplication left matrix rearrangement, comprising:
[0011] Signature generation module, used to generate the signature vector SIG for each row of the sparse matrix r ;
[0012] The left matrix of sparse matrix multiplication is defined as matrix A. For the row with row number r, the column index of the row is sorted from small to large to form a column vector A. r ; The number of columns of A is N; r ≥ 0; r ∈ R;
[0013] Bucket number calculation module, including a set of configurable hash function modules, used to convert the signature vector SIG of each row of the compressed sparse matrix r The label of the bucket that each row number r should enter;
[0014] A cache register group, including a cache register group 1 and a cache register group 2. Each cache register group is used to cache the row number of a row of A, the signature vector, and the label of the bucket to be entered;
[0015] A selection module, comprising an estimation submodule 1, an estimation submodule 2 and a comparison submodule;
[0016] Estimation submodule 1 and estimation submodule 2 estimate similarity in different ways.
[0017] The comparison submodule obtains the next row of the left matrix involved in the sparse matrix multiplication based on the two similarities estimated above;
[0018] and a control module, which is used to control the update of the contents of the cache register group, control the cache register group 1 to always store the signature vector of the row of A that is participating in the multiplication operation and the label of the bucket that should be entered, control the reading and writing of the memory hash table, and control the comparison content input by the comparison submodule.
[0019] Preferably, the signature generation module includes:
[0020] A first group of configurable function modules, a group of subtractor modules, a second group of configurable function modules and a group of comparators are connected in sequence;
[0021] The first group of configurable function modules includes a configurable function f1,
[0022] The second group of configurable function modules includes K configurable functions f2, K>1; the output of the second group of configurable function modules is a column vector sig, whose size is K×1;
[0023] The K configurable functions f2 are exactly the same;
[0024] The input and output of each configurable function are elements in a sequence;
[0025] The group of subtractor modules includes K subtractors;
[0026] The first set of configurable function modules has an input of x, which is A r The output of a group of subtractor modules is a vector sub, whose size is K×1;
[0027] The value of the kth element of the vector sub is [f1(x)-k+1]mod(N);
[0028] The input of the second group of configurable function modules is the vector sub,
[0029] The value of the kth element of sig is f2{[f1(x)-k+1]mod(N)};
[0030] The group of comparators includes K unsigned comparators running in parallel and a group of register modules;
[0031] The group of register modules includes K registers; the content stored in the group of register modules is vector sig c ;
[0032] sig c Signature for the middle;
[0033] The unsigned number comparator outputs a smaller value;
[0034] The input of the kth unsigned comparator is the kth element of sig and sig c The kth element of is output as sig of the next clock cycle. c The kth element of ;
[0035] A r Each element in q r-1 round of comparison, defining the vector stored in the final set of register modules as the signature vector SIG r ;q r Indicates A r The number of elements in .
[0036] Preferably, the bucket number calculation module includes a set of configurable hash function modules, which include T hash functions;
[0037] T=K / b
[0038] The input of the tth hash function is SIG r The (t×b-b+1)th to (t×b)th elements in , 1≤t≤T;
[0039] Define the output of these T identical hash functions as column vector B r , whose size is T×1, B r The t-th element of represents the bucket number of the t-th hash table in a set of in-memory hash tables, and the value r should be stored in this bucket.
[0040] Preferably, the estimation process of the estimation submodule 1 comprises the following steps:
[0041] Count the number of times each different unsigned number appears, and compare to get the unsigned number with the largest number of occurrences in this string of unsigned numbers, defined as r1, r1∈R; define its number of occurrences as sim1;
[0042] Among them, a string of unsigned numbers is serially input to its port through the control module, and this string of unsigned numbers is divided into T groups the same as step 3, and they are also serially input in sequence.
[0043] Preferably, the estimation process of the estimation submodule 2 includes the following steps:
[0044] The signature vector stored in the cache register group 1 is recorded as column vector N1, whose size is K×1, and is compared element by element with the latest generated signature vector SIG, and the number of identical elements at the same position of the two is defined as sim;
[0045] Always store the row number of the row with the largest sim, the signature vector, and the label of the bucket it should enter into cache register group 2;
[0046] Define the row number stored in cache register group 2 as r2, and its corresponding sim value is sim2; r2∈R;
[0047] If the signature vectors of all non-zero rows of the left matrix in the sparse matrix multiplication have been generated, that is, when the latest generated signature vector SIG input into the estimation submodule is invalid, the situation is defined as r2 does not exist, otherwise r2 exists.
[0048] Preferably, the comparison submodule obtains the next row of the matrix A participating in the sparse matrix multiplication in the following manner, specifically including:
[0049] Assumption A r Previously, A was involved in a multiplication operation. r When the multiplication operation involved is completed: when both r1 and r2 exist, compare sim1 and sim2;
[0050] If sim1>sim2, r1 is selected as the next row number for multiplication operation;
[0051] If sim1≤sim2, select r2 as the next row number for multiplication operation;
[0052] When r1 exists and r2 does not exist, r1 is selected as the next row number to participate in the multiplication operation;
[0053] When r2 exists and r1 does not exist, r2 is selected as the next row number for multiplication operation;
[0054] When the first row number involved in the multiplication operation is determined, the next row number involved in the multiplication operation is selected in this way until the sparse matrix multiplication is completed.
[0055] Compared with the prior art, the advantages of the present invention are:
[0056] Through a specially designed parallel computing structure, interconnected sub-modules such as signature generation modules, bucket number calculation modules, and cache register groups are used to generate a row sequence or column sequence of the rearranged sparse matrix multiplication left matrix, so that when the Gustavson algorithm is used for sparse matrix multiplication, the degree of data reuse in the cache is improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 It is a schematic diagram of the overall structure of the dedicated hardware structure for left matrix rearrangement of sparse matrix multiplication;
[0058] Figure 2 is a schematic diagram of the dense form of the example matrix A;
[0059] Figure 3 This is a schematic diagram of the data flow in the hardware during index reordering and signature generation;
[0060] Figure 4 Schematic diagram of the dense form of the rearranged matrix. DETAILED DESCRIPTION
[0061] The sparse matrix multiplication left matrix rearrangement dedicated hardware structure of the present invention will be described in more detail below in conjunction with the schematic diagram, wherein a preferred embodiment of the present invention is represented, and it should be understood that those skilled in the art can modify the present invention described herein, and still achieve the advantageous effects of the present invention. Therefore, the following description should be understood as being widely known to those skilled in the art, and not as a limitation of the present invention.
[0062] A special hardware structure for rearranging the left matrix of sparse matrix multiplication belongs to the preprocessing link of sparse matrix multiplication hardware acceleration. It uses the properties of local sensitive hashing technology, a specially designed parallel computing structure, and uses interconnected signature generation modules, bucket number calculation modules, cache register groups and other sub-modules to generate a row sequence or column sequence of the rearranged left matrix of sparse matrix multiplication, so that when using the Gustavson algorithm for sparse matrix multiplication, the degree of data reuse in the cache is improved. The specific working principle is as follows:
[0063] 1. Index re-arrangement and signature generation
[0064] The left matrix of sparse matrix multiplication is matrix A. For the row (column) numbered r, the column (row) index of the row (column) is sorted from small to large to form a column vector A. r , whose size is q r ×1;
[0065] Among them, q r Indicates A r The number of elements in
[0066] Define a value N:
[0067] If for each r, A r The values in are row indices, then N is the number of rows in A,
[0068] If for each r, A r The value in is the column index, then N is the number of columns in A. In this embodiment, r is the row number, and A r is the column index of row number r.
[0069] In this example, a 6×6 sparse matrix A (N=6) with a smaller dimension is used as the left and right matrix sparse matrix multiplication (i.e. C=A×A) by row (i.e. in CSR format). The dense form of A is shown as follows: Figure 1 , the three arrays stored in A's CSR format are as follows:
[0070] Table 1A CSR format
[0071]
[0072]
[0073] Assume that randomly determined f1 and f2 satisfy:
[0074] f1(0)=1, f1(1)=3, f1(2)=5, f1(3)=4, f1(4)=0, f1(5)=2;
[0075] f2(0)=3,f2(1)=5,f2(2)=4,f2(3)=0,f2(4)=2,f2(5)=1
[0076] The i-th element of the "row pointer" array is the position of the first non-zero element of the i-th row of the sparse matrix in the "value" array.
[0077] For example, if the second element of the "row pointer" array is 5, then the first non-zero element of the second row of the sparse matrix is at position 5 in the "value" array and has a value of f, which is exactly the first non-zero element of the second row of the sparse matrix.
[0078] 1. Index reordering
[0079] In this example, if r=0, then A0=[0,3,4] T , after rearrangement by f1, it corresponds to [1,4,0] T .
[0080] 2. Signature generation
[0081] The first group of configurable function modules output f1 (a i ), and input K subtractors in parallel, where the output of the kth subtractor is [f1(a i )-k+1]mod(N);
[0082] The results output by the K subtractors are input in parallel to the second group of configurable function modules, and K results output in parallel are obtained in each clock cycle, namely, the vector sig.
[0083] The two sets of functions can be the same or different.
[0084] Two groups of configurable function modules independently configure parameters through pseudo-random numbers generated by counters related only to the clock.
[0085] If there is a sequence that contains several consecutive positive integers starting from 0, and each integer appears only once in the sequence, an element in the sequence is input into the configurable function module, and the output value is also an element in this sequence. After the entire sequence is input into the function in sequence, the element composition of the output sequence obtained is the same as the element composition of the input sequence.
[0086] In this example, K=4, f1(a i) outputs {1,4,0} in three clock cycles, and the calculations are done in parallel. The results are as follows:
[0087] f2[(1-0)mod6],f2[(1-1)mod6],f2[(1-2)mod6],f2[(1-3)mod6];
[0088] f2[(4-0)mod6],f2[(4-1)mod6],f2[(4-2)mod6],f2[(4-3)mod6];
[0089] f2[(0-0)mod6],f2[(0-1)mod6],f2[(0-2)mod6],f2[(0-3)mod6].
[0090] Among them, in the above calculation results, each row of calculation results corresponds to the situation when the outputs of four subtractors in one clock cycle are used as the input of f2.
[0091] Take the calculation result sig1 of the first row as an example:
[0092] Take f2[(1-2)mod6] as an example, f1(a i )=1,k=3.
[0093] You can get:
[0094] sig1=[5,3,1,2] T ,sig2=[2,0,4,5] T ,sig3=[3,1,2,0] T
[0095] Take the smaller element at the corresponding position and get the signature of A0
[0096] SIG0=[2,0,1,0] t
[0097] The specific calculation process of SIG0 is as follows:
[0098] Counting from the output sig1, in the first clock cycle, the contents stored in a set of register modules in a set of comparators in the signature generation module are vector sig c , and each element of the vector is the maximum value (each bit of the register is 1). For example, if a 32-bit unsigned comparator is used, and this group of K registers is also 32 bits, then sig c The value of each element of is 4294967295, which is ffffffff in hexadecimal.
[0099] The second group of configurable function modules outputs sig1, whose kth element is the same as sig c The kth element of is compared (but this round of comparison is not counted in the number of comparison rounds), the smaller one is taken, and then stored in this group of register modules, then there is
[0100] sig c =sig1=[5,3,1,2] T
[0101] The second clock cycle, the second group of configurable function module output, sig2 = [2,0,4,5] T , whose kth element is equal to sig c Compare the kth element of (that is, the first round of comparison), take the smaller one, and then store it in this group of register modules, then there is
[0102] sig c =[2,0,1,2] T
[0103] In the third clock cycle, the second group of configurable function modules outputs sig3, and its kth element is equal to sig c Compare the kth element of (that is, the second round of comparison), take the smaller one, and then store it in this group of register modules, then there is
[0104] sig c =[2,0,1,0] T
[0105] Since two rounds of comparison have been done, and for r = 0 there are q r -1=2, so now we have
[0106] SIG0=sig c =[2,0,1,0] T
[0107] That is the signature vector of the row corresponding to r=0.
[0108] Calculate the average time spent in clock cycles and find it to be the number of elements in A0, which is 3 cycles.
[0109] Similarly, the signatures of each row are shown in the following table. On average, 16 clock cycles are required to obtain the result, because there are 16 non-zero elements in the sparse matrix.
[0110] Table 2 Signatures of rows
[0111]
[0112] The data flow in the hardware during the calculation process is as follows Figure 3 .
[0113] 2. Generate bucket labels and fill the hash table in memory
[0114] The strip width in the locality sensitive hashing technique is b, and a group of T = K / b identical sub-computing modules are used. In this example, b = 1, and T = 4.
[0115] The sub-computation module is the hash function module. The input of the t-th sub-computation module is SIG r The (t×b-b+1)th to (t×b)th elements in , 1≤t≤T.
[0116] In this example, since b=1, T=4, SIG0=[2,0,1,0] T Taking as an example, the input of the first sub-computing module is 2, the input of the second sub-computing module is 0, the input of the third sub-computing module is 1, and the input of the fourth sub-computing module is 0.
[0117] Also for this example, if only the value of b is changed, for example b=2, then T=2, and the input of the first sub-computation module is 2 and 0, and the input of the second sub-computation module is 1 and 0.
[0118] Define the output of these T identical hash functions as column vector B r , whose size is T×1; there are a total of T hash tables in memory, B r The value of the tth element of is b r,t , then the value r should be stored in the bucket number b in the tth hash table r,t Bucket.
[0119] The filled buckets are shown in the following table, where the content filled in the table is row number r.
[0120] That is, a set of configurable hash function modules, including 4 hash functions h.
[0121] In this example, the hash function used is h=h(g)=g, that is, the hash function inputs 1 parameter and the output value is equal to the input value.
[0122] Among them, the input g of the hash function is SIG r The elements in .
[0123] For example, for SIG0 = [2,0,1,0] T , there are four identical hash functions, then the inputs of the hash functions are 2, 0, 1, 0. Since the hash function used in this example is h = h(g) = g, the outputs are also 2, 0, 1, 0, that is, B0 = [2, 0, 1, 0] T .
[0124] There is a set of 4 hash tables in memory (T=4).
[0125] Since B0=[2,0,1,0] T , so the first element of B0 is 2, which means that the value 0 (r=0) should be filled into the second bucket of hash table 1; similarly, the value 0 should also be filled into the 0th bucket of hash table 2, the 1st bucket of hash table 3, and the 0th bucket of hash table 4. If only B0 is calculated, the filled hash table is as follows:
[0126] Table 3 Hash table based on B0 calculation
[0127]
[0128]
[0129] In this example, B1, B2, B3, B4, and B5 are calculated in sequence, and the hash table is filled as follows:
[0130] Table 4 Based on B r Calculate the complete hash table
[0131] Bucket number 0 1 2 3 4 Hash Table 1 1,3 4,5 0 2 / Hash Table 2 0,3,5 2 4 / 1 Hash Table 3 4,5 0,2 / 1,3 / Hash Table 4 0,2,5 1,3 4 / /
[0132] The calculation of signature generation and bucket labeling is completed in a pipeline form, so it takes an average of 16 clock cycles until all bucket labels are generated.
[0133] Writing the calculated bucket number back to memory takes up one channel, and takes t = 4 clock cycles to fill each row number in the specified position of the hash table in memory. This can overlap with the signature generation link of the next row in time.
[0134] 3. Real-time reordering of the left matrix in sparse matrix multiplication
[0135] Assume that the sparse matrix multiplication starts from the row numbered 0, and the number of elements in the right matrix that need to participate in the calculation in this round of calculation is taken as the number of cycles required for this row to complete the multiplication operation in which it participates.
[0136] Then, the number of cycles required to complete the multiplication of each row of the left sparse matrix A and the number of non-zero elements in each row are shown in the following table. This table helps to analyze the running status of real-time rearrangement.
[0137] Table 5 The number of cycles required to complete the multiplication of each row of the left sparse matrix A and the number of non-zero elements in each row
[0138] Line Number 0 1 2 3 4 5 Number of cycles 8 8 8 8 6 9 Number of non-zero elements 3 2 3 3 2 3
[0139] Taking row number 0 as an example, Table 5 is explained as follows:
[0140] Depend on Figure 2Knowing that the column indices corresponding to row number 0 of the left matrix are 0, 3, and 4, row number 0 needs to be multiplied with the 0th, 3rd, and 4th rows in the right matrix A.
[0141] Depend on Figure 2 It can be seen that the above three rows of the right matrix A contain a total of 8 non-zero elements. Therefore, the row number of the left matrix is 0, and the number of cycles required to complete the multiplication is 8.
[0142] Assuming that the clock cycle count tc=0 when the sparse matrix multiplication operation starts, then when tc=8, the next left matrix row participating in the multiplication needs to be generated.
[0143] Because the multiplication of row number 0 requires 8 cycles, and there are 8 non-zero elements in rows 0-2 of the left matrix, the signature vector and bucket labeling of these three rows also require 8 cycles.
[0144] Therefore, during the multiplication of row number 0, there are 8 non-zero elements in rows 0-2 of the left matrix, and the signature vectors and bucket labels of these three rows have been calculated.
[0145] Compare the generated signature vectors to estimate similarity: When each group of newly generated signature vectors, defined as SIG j When ready, the estimation submodule 2 compares SIG for each k in parallel. j and SIG mul The kth element (participating in the multiplication) has the same number of elements as m. j ;
[0146] Among them, mul means multiply.
[0147] j represents the number of the latest generated signature vector.
[0148] Always get the maximum m j The corresponding row number r2 and the signature vector and bucket number are stored in cache register group 2, and the estimated similarity is
[0149] In this example, when tc=8, SIG0 is stored in cache register group 1; SIG2 is stored in cache register group 2, and sim2=2.
[0150] For example, since sparse matrix multiplication starts from the row number 0, the first generated signature vector is the signature vector SIG0 of the row number 0 in Table 2.
[0151] During the multiplication process of row number 0 (8 cycles are required), the signature vector and bucket label calculation of row number 0 are completed in the first 3 cycles (as mentioned above), the signature vector SIG1 and bucket label calculation of row number 1 are completed in the 5th cycle (there are 2 non-zero elements in row number 1), and the signature vector SIG2 and bucket label calculation of row number 2 are completed in the 8th cycle.
[0152] From the above, in the first 8 cycles, the signature vector involved in the multiplication is SIG0, and the newly generated signature vector SIG j Including SIG1 and SIG2 (when 5≤tc≤7 and tc=8 respectively).
[0153] Comparing SIG1 and SIG0, the number of identical elements is 0;
[0154] Comparing SIG2 and SIG0, the number of identical elements is 2.
[0155] Therefore, sim2=2.
[0156] Using estimation submodule 1, the similarity is estimated by counting the different elements in the T buckets where mul is located. It is found that in the bucket where r=0 is located (the second bucket of hash table 1, the 0th bucket of hash table 2, the first bucket of hash table 3, and the 0th bucket of hash table 4), there are 2 elements other than 0, and 2 appears twice, so sim1=2. This is because when tc=8, the filled T hash tables are as follows:
[0157] Table 6 Schematic diagram of T filled hash tables when tc=8
[0158] Bucket number 0 1 2 3 4 Hash Table 1 1 0 2 Hash Table 2 0 2 1 Hash Table 3 0,2 1 Hash Table 4 0,2 1
[0159] So sim1=sim2=2, r1=r2=2.
[0160] Therefore, the row number selected for the next multiplication operation is 2.
[0161] The multiplication operation of the left matrix row numbered 2 with the 0th, 4th, and 5th rows of the right matrix (three rows with a total of 8 non-zero elements) requires 8 clock cycles.
[0162] That is, the left matrix row with row number 2 completes the multiplication operation in the 16th cycle.
[0163] Therefore, when tc=16, the signature vectors and bucket labels of all rows have been calculated (as mentioned above, a total of 16 clock cycles are required), and at this time, the left matrix row with row number 2 has also completed the multiplication operation involved.
[0164] That is, SIG3, SIG4 and SIG5 are compared with SIG2 respectively, and it is obtained that sim2=1.
[0165] At this time, SIG2 is stored in register group 1, SIG5 is stored in register group 2, and sim2=1.
[0166] That is, by comparing the generated signatures to estimate the similarity, we obtain r2=5.
[0167] Using estimation submodule 1, the similarity is estimated by counting the different elements in the T buckets where mul is located. It is found that in the bucket where r=2 (the row currently participating in the multiplication operation) is located, the most common element is 5 (excluding 0 and 2, r1=5), and sim1=1 is estimated.
[0168] The reason why 0 and 2 are not considered is that the first row to be multiplied is row number 0, and the second row to be multiplied is row number 2. Here we need to determine the next row after row number 2 that participates in the multiplication operation.
[0169] As shown in Table 7, the four buckets where r=2 is located are the 3rd bucket of hash table 1, the 1st bucket of hash table 2, the 1st bucket of hash table 3, and the 0th bucket of hash table 4. Among these 4 buckets, the row number 5 appears the most (excluding 0 and 2), with a frequency of 1, so sim1=1.
[0170] Table 7 The bucket where row number 2 is located
[0171] Bucket number 0 1 2 3 4 Hash Table 1 2 Hash Table 2 2 Hash Table 3 0,2 Hash Table 4 0,2,5
[0172] In summary, sim1=sim2, and the row number of the next row to participate in the multiplication operation is 5.
[0173] When tc>16, all signature vectors have been generated, and there are no valid signature vectors generated in real time. r2 does not exist, and only r1 is considered.
[0174] Therefore, the estimation submodule 1 is used to estimate the similarity by counting the different elements in the T buckets where mul is located. Similarly, it can be concluded that the row numbers of the next rows involved in the multiplication operation are 3, 1, and 4 respectively.
[0175] The dense form of the rearranged matrix is shown in Figure 4 , respectively a origin and a reorder The sum of the number of common column indices of two adjacent rows of the left matrix A after the original and rearranged sparse matrix multiplication is represented by .
[0176] a origin =0+1+1+0+1=3,
[0177] a reorder =2+1+2+2+0=7.
[0178] When using the Gustavson algorithm for sparse matrix multiplication, the number of common column indices between two adjacent rows of the left matrix A is equal to the number of common rows of the right matrix involved in the multiplication of the two adjacent rows of the left matrix A. Sparse matrix multiplication dedicated architecture or general processor architecture usually contains caches connected to the memory. The capacity of these caches is usually not large and can only cache a large sparse matrix (for example, a sparse matrix with a dimension of 10 4 The number of non-zero elements in a row can reach 10 3 Then, if the number of common column indices between two adjacent rows of the left matrix is greater, fewer rows of the right matrix need to be updated in the cache, thereby improving data reuse and reducing memory access time.
[0179] In this example, let's assume that the cache can only store three rows of elements of A. Then, when r=0 participates in the multiplication operation, the 0th, 3rd, and 4th rows of the right matrix (also A in this example) are read into the cache.
[0180] If there is no re-arrangement, the next row involved in the multiplication operation is r=1, so the 1st and 5th rows of the right matrix (also A in this case) need to be read into the cache, which requires accessing the locations of the 1st and 5th rows of elements of A in the memory to achieve this.
[0181] If the next row involved in the multiplication operation is rearranged to be r=2, then the cache needs to store the 0th, 4th, and 5th rows of the right matrix (also A in this case). Then the cache only needs to update the stored 3rd row of A to the 5th row of A, and this can be achieved by accessing the location of the 5th row of elements in the memory, thus reducing memory access and improving data reuse.
[0182] Therefore a reorder >a origin , indicating that the degree of data reuse in sparse matrix multiplication is improved and can be optimized in real time.
[0183] The above is only a preferred embodiment of the present invention and does not limit the present invention in any way. Any technician in the relevant technical field, without departing from the scope of the technical solution of the present invention, makes any form of equivalent replacement or modification to the technical solution and technical content disclosed in the present invention, which does not depart from the content of the technical solution of the present invention and still falls within the protection scope of the present invention.
Claims
1. A special hardware structure for left matrix rearrangement of sparse matrix multiplication, characterized in that: include: Signature generation module, used to generate the signature vector SIG for each row of the sparse matrix r ; The left matrix of sparse matrix multiplication is defined as matrix A. For the row with row number r, the column index of the row is sorted from small to large to form a column vector A. r ; The number of columns of A is N; r ≥ 0; r ∈ R; Bucket number calculation module, including a set of configurable hash function modules, used to convert the signature vector SIG of each row of the compressed sparse matrix r The label of the bucket that each row number r should enter; A cache register group, including a cache register group 1 and a cache register group 2. Each cache register group is used to cache the row number of a row of A, the signature vector, and the label of the bucket to be entered; A selection module, comprising an estimation submodule 1, an estimation submodule 2 and a comparison submodule; Estimation submodule 1 and estimation submodule 2 estimate similarity in different ways. The comparison submodule obtains the next row of the left matrix involved in the sparse matrix multiplication based on the two similarities estimated above; and a control module, which is used to control the update of the contents of the cache register group, control the cache register group 1 to always store the signature vector of the row of A that is participating in the multiplication operation and the label of the bucket that should be entered, control the reading and writing of the memory hash table, and control the comparison content input by the comparison submodule.
2. The sparse matrix multiplication left matrix rearrangement dedicated hardware structure according to claim 1, characterized in that: The signature generation module comprises: A first group of configurable function modules, a group of subtractor modules, a second group of configurable function modules and a group of comparators are connected in sequence; The first group of configurable function modules includes a configurable function F1, The second group of configurable function modules includes K configurable functions F2, K>1; the output of the second group of configurable function modules is a column vector sig, whose size is K×1; The K configurable functions F2 are exactly the same; The input and output of each configurable function are elements in a sequence; The group of subtractor modules includes K subtractors; The first set of configurable function modules has an input of x, which is A r The output of a group of subtractor modules is a vector sub, whose size is K×1; The value of the kth element of the vector sub is [F1(x)-k+1]mod(N); The input of the second group of configurable function modules is the vector sub, The value of the kth element of sig is F2{[F1(x)-k+1]mod(N)}; The group of comparators includes K unsigned comparators running in parallel and a group of register modules; The group of register modules includes K registers; the content stored in the group of register modules is vector sig c ;sig c Signature for the middle; The unsigned number comparator outputs a smaller value; The input of the kth unsigned comparator is the kth element of sig and sig c The kth element of is output as sig of the next clock cycle. c The kth element of ; A r Each element in q r -1 round of comparison, defining the vector stored in the final set of register modules as the signature vector SIG r ;q r Indicates A r The number of elements in .
3. The sparse matrix multiplication left matrix rearrangement dedicated hardware structure according to claim 1, characterized in that: A bucket number calculation module, including a set of configurable hash function modules, which includes T hash functions; T=K / b The input of the tth hash function is SIG r The (t×b-b+1)th to (t×b)th elements in , 1≤t≤T; Define the output of these T identical hash functions as column vector B r , whose size is T×1, B r The t-th element of represents the bucket number of the t-th hash table in a set of in-memory hash tables, and the value r should be stored in this bucket.
4. The dedicated hardware structure for left matrix rearrangement of sparse matrix multiplication according to claim 3, characterized in that: The estimation process of the estimation submodule 1 comprises the following steps: Count the number of times each different unsigned number appears, and compare to get the unsigned number with the largest number of occurrences in this string of unsigned numbers, defined as r1, r1∈R; define its number of occurrences as sim1; Among them, a string of unsigned numbers is serially input to its port through the control module, and this string of unsigned numbers is divided into T groups the same as step 3, and they are also serially input in sequence.
5. The sparse matrix multiplication left matrix rearrangement dedicated hardware structure according to claim 3, characterized in that: The estimation process of the estimation submodule 2 comprises the following steps: The signature vector stored in the cache register group 1 is recorded as column vector N1, whose size is K×1, and is compared element by element with the latest generated signature vector SIG, and the number of identical elements at the same position of the two is defined as sim; Always store the row number of the row with the largest sim, the signature vector, and the label of the bucket it should enter into cache register group 2; Define the row number stored in cache register group 2 as r2, and its corresponding sim value is sim2; r2∈R; If the signature vectors of all non-zero rows of the left matrix in the sparse matrix multiplication have been generated, that is, when the latest generated signature vector SIG input into the estimation submodule is invalid, the situation is defined as r2 does not exist, otherwise r2 exists.
6. The dedicated hardware structure for left matrix rearrangement of sparse matrix multiplication according to claim 3, characterized in that: The comparison submodule obtains the next row of the matrix A participating in the sparse matrix multiplication by the following method, specifically including: Assumption A r Previously, A was involved in a multiplication operation. r When the multiplication operation involved is completed: when both r1 and r2 exist, compare sim1 and sim2; If sim1>sim2, select r1 as the next row number for multiplication operation; If sim1≤sim2, select r2 as the next row number for multiplication operation; When r1 exists and r2 does not exist, r1 is selected as the next row number to participate in the multiplication operation; When r2 exists and r1 does not exist, r2 is selected as the next row number for multiplication operation; When the first row number involved in the multiplication operation is determined, the next row number involved in the multiplication operation is selected in this way until the sparse matrix multiplication is completed.
