In-memory index matching method and device of sparse matrix, equipment and medium

By configuring a preset comparator circuit in the static random access memory, the index matching of sparse matrix is ​​directly performed in the memory, which solves the problem of large index matching overhead in sparse matrix calculation, and improves the computing performance and processing speed.

CN119066028APending Publication Date: 2024-12-03NAT UNIV OF DEFENSE TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411281639.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In sparse matrix calculation, the overhead of index matching is large, which limits the computing performance. Especially because the matrix is ​​sparse and the proportion of non-zero elements is low, the data overhead of the index moving from memory to processor is significant.

Method used

By configuring the preset comparator circuit in the static random access memory, calculate the non-zero element proportion of the sparse matrix, select the appropriate chunking strategy, and use the preset comparator circuit to conduct index matching, and directly perform index matching in memory , avoid data moving back and forth between memory and processor.

Benefits of technology

It significantly reduces the data movement overhead and computational complexity of index matching, improves the performance of sparse matrix operations, speeds up processing speed, adapts to matrices of different sparseness, and reduces unnecessary comparison operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066028A_ABST
    Figure CN119066028A_ABST
Patent Text Reader

Abstract

The invention discloses an in-memory index matching method and device for a sparse matrix, equipment and a medium, relates to the technical field of microprocessor design, and is applied to a static random access memory for configuring a preset comparator circuit. If yes, calculating the number ratio of non-zero elements of a target sparse matrix in the sparse matrix operation request in a storage array of the static random access memory to obtain target sparseness corresponding to rows of the target sparse matrix; comparing the target sparseness with a preset sparseness threshold value to obtain a corresponding comparison result; selecting a corresponding blocking strategy according to a comparison result to obtain matrix blocks of each target sparse matrix; and performing index matching on each matrix block to be compared of the target sparse matrix participating in the matrix operation by using a preset comparator circuit to obtain a corresponding in-memory index matching result. And index matching is directly carried out in the static random access memory, so that the consumption caused by data movement is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of microprocessor design, and in particular to a method, device, equipment and medium for in-memory index matching of a sparse matrix. Background Art

[0002] Matrix operations play a key role in the field of artificial intelligence, supporting the implementation and optimization of a variety of complex models and algorithms. They are one of the cores of realizing modern AI (Artificial Intelligence) applications. However, due to the rapid increase in the scale of artificial intelligence models, the hardware overhead of model reasoning has become unaffordable. More and more work is devoted to introducing sparsity into models such as deep learning to reduce the amount of computation and accelerate the performance of training and reasoning. Current research shows that sparsity may achieve a 10 to 100 times increase in the near future, and even higher speedups may be achieved at higher sparsity levels. Since there are a large number of zero elements in sparse matrices, in order to save storage space, a compressed format is usually used to store only non-zero elements and their corresponding indices.

[0003] When performing sparse matrix calculations, general-purpose processors first need to read the index from the memory and perform index matching calculations. Then, based on the successfully matched index, the corresponding non-zero elements are read from the memory to perform multiplication and accumulation calculations. In practical applications, due to the high sparsity of the matrix and the low proportion of non-zero elements, the overhead of index matching is often the main overhead in sparse matrix multiplication calculations. At the same time, the overhead of moving the index from the memory to the data in the processor limits the performance of the calculation. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a method, device, equipment and medium for in-memory index matching of sparse matrices, which can reduce the overhead of accessing memory during index matching during sparse matrix operations and accelerate the calculation of sparse matrices. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a method for in-memory index matching of a sparse matrix, which is applied to a static random access memory configured with a preset comparator circuit, comprising:

[0006] When receiving a sparse matrix operation request, calculating the ratio of the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory to obtain target sparsities corresponding to each row of the target sparse matrix;

[0007] Comparing the magnitude relationship between each of the target sparsities and a preset sparsity threshold to obtain a corresponding comparison result;

[0008] Selecting a corresponding block strategy according to the comparison result to obtain matrix blocks of each of the target sparse matrices;

[0009] The preset comparator circuit is used to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

[0010] Optionally, when receiving a sparse matrix operation request, calculating the proportion of the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory to obtain target sparsities corresponding to each row of the target sparse matrix, including:

[0011] When receiving a sparse matrix operation request, writing the target sparse matrix in the sparse matrix operation request into different storage locations of the storage array of the static random access memory according to a preset compression format, as a first sparse matrix and a second sparse matrix; wherein the first sparse matrix and the second sparse matrix are sparse matrices participating in the matrix operation;

[0012] The number of non-zero elements and the total number of elements in each row of non-zero elements are obtained, and the proportion of the number of non-zero elements is calculated using the number of non-zero elements and the total number of elements to obtain target sparsities corresponding to each row of the target sparse matrix.

[0013] Optionally, the selecting a corresponding block strategy according to the comparison result to obtain matrix blocks of each target sparse matrix includes:

[0014] If the comparison result is that the target sparsity is less than the preset sparsity threshold, then use the preset large-granularity block strategy to perform block division to obtain corresponding matrix blocks to be compared;

[0015] If the comparison result is that the target sparsity is greater than or equal to the preset sparsity threshold, a preset small-granularity blocking strategy is used to perform blocking to obtain corresponding matrix blocks to be compared.

[0016] Optionally, the matrix blocks to be compared of the first sparse matrix are row blocks to be compared, and the matrix blocks to be compared of the second sparse matrix are column blocks to be compared.

[0017] Optionally, when receiving a sparse matrix operation request, writing the target sparse matrix in the sparse matrix operation request into different storage locations of the storage array of the static random access memory according to a preset compression format as a first sparse matrix and a second sparse matrix, including:

[0018] When receiving a sparse matrix operation request, determining an operation type representing a target sparse matrix in the sparse matrix operation request;

[0019] When the operation type is a matrix multiplication operation type, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory according to a preset compressed sparse row format and a preset compressed sparse column format, as a first sparse matrix and a second sparse matrix.

[0020] Optionally, the using the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result includes:

[0021] Amplifying the operands of each matrix block to be compared of the first sparse matrix and the second sparse matrix by a sensitive amplifier using a bit line calculation operation of an in-memory calculation in the static random access memory, thereby obtaining an amplified matrix block of the first sparse matrix and an amplified matrix block of the second sparse matrix;

[0022] Index matching is performed on the enlarged matrix blocks of the first sparse matrix and the enlarged matrix blocks of the second sparse matrix through a preset comparator circuit to obtain corresponding in-memory index matching results.

[0023] Optionally, the using the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result includes:

[0024] Using the preset comparator circuit to perform index matching on the block indexes of each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding block index matching result;

[0025] If the block index matches successfully, the intra-block index matching step is performed.

[0026] In a second aspect, the present application discloses an in-memory index matching device for a sparse matrix, which is applied to a static random access memory configured with a preset comparator circuit, comprising:

[0027] A sparsity calculation module, for, when receiving a sparse matrix operation request, calculating the ratio of the number of non-zero elements of the target sparse matrix in the storage array of the static random access memory in the sparse matrix operation request, so as to obtain each target sparsity corresponding to each row of the target sparse matrix;

[0028] A comparison module, used for comparing the magnitude relationship between each of the target sparsities and a preset sparsity threshold to obtain a corresponding comparison result;

[0029] A matrix partitioning module, used for selecting a corresponding partitioning strategy according to the comparison result to obtain matrix partitioning of each target sparse matrix;

[0030] The index matching module is used to use the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

[0031] In a third aspect, the present application discloses an electronic device, comprising:

[0032] Memory, used to store computer programs;

[0033] A processor is used to execute the computer program to implement the steps of the in-memory index matching method of the sparse matrix disclosed above.

[0034] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the in-memory index matching method of the sparse matrix disclosed above are implemented.

[0035] It can be seen that the present application discloses an in-memory index matching method for a sparse matrix, which is applied to a static random access memory configured with a preset comparator circuit, including: when receiving a sparse matrix operation request, the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory is calculated to obtain each target sparsity corresponding to each row of the target sparse matrix; compare the size relationship between each target sparsity and a preset sparsity threshold to obtain a corresponding comparison result; select a corresponding block strategy according to the comparison result to obtain a matrix block of each target sparse matrix; use the preset comparator circuit to perform index matching on each matrix block to be compared of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result. It can be seen that index matching is performed directly in the static random access memory, which avoids the back and forth movement of a large amount of data between the memory and the processor, and significantly reduces the time and energy consumption caused by data movement. It can flexibly deal with matrices of different sparsity and select appropriate blocking strategies according to specific circumstances. It has stronger adaptability and versatility. It can select blocking strategies according to sparsity, and can process matrix rows of different sparsity in a more targeted manner, reduce unnecessary comparison operations, and thus improve the overall computing efficiency. It performs index matching in the memory by presetting the comparator circuit, avoiding the complex index matching calculation process of the general processor, reducing the complexity and time cost of the calculation, and can significantly improve the performance of the entire system when processing sparse matrix operations, speed up the processing speed, and provide more efficient support for related applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A flow chart of an in-memory index matching method for a sparse matrix disclosed in this application;

[0038] Figure 2 A flowchart of a specific in-memory index matching method for a sparse matrix disclosed in this application;

[0039] Figure 3 A CSR and CSC storage format result diagram of a sparse matrix disclosed in this application;

[0040] Figure 4 A schematic diagram of index matching in a matrix inner product multiplication disclosed in this application;

[0041] Figure 5 A SRAM memory index comparison design diagram disclosed in this application;

[0042] Figure 6 A schematic diagram of block index matching disclosed in this application;

[0043] Figure 7 This is a schematic diagram of the structure of an in-memory index matching device for a sparse matrix disclosed in the present application;

[0044] Figure 8 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] Matrix operations play a key role in the field of artificial intelligence, supporting the implementation and optimization of a variety of complex models and algorithms, and are one of the cores of realizing modern AI applications. However, due to the rapid increase in the scale of artificial intelligence models, the hardware overhead of model reasoning has become unaffordable. More and more work is devoted to introducing sparsity into models such as deep learning to reduce the amount of computation and accelerate the performance of training and reasoning. Current research shows that sparsity may achieve a 10 to 100 times increase in the near future, and even higher speedups may be achieved at higher sparsity levels. Since there are a large number of zero elements in sparse matrices, in order to save storage space, a compressed format is usually used to store only non-zero elements and their corresponding indices.

[0047] When performing sparse matrix calculations, general-purpose processors first need to read the index from the memory and perform index matching calculations. Then, based on the successfully matched index, the corresponding non-zero elements are read from the memory to perform multiplication and accumulation calculations. In practical applications, due to the high sparsity of the matrix and the low proportion of non-zero elements, the overhead of index matching is often the main overhead in sparse matrix multiplication calculations. At the same time, the overhead of moving the index from the memory to the data in the processor limits the performance of the calculation.

[0048] To this end, the present invention provides an in-memory index matching scheme for sparse matrices, which can reduce the data movement overhead of the matching process, reduce the memory access overhead during index matching, and accelerate the calculation of sparse matrices.

[0049] Reference Figure 1 As shown, an embodiment of the present invention discloses a method for in-memory index matching of a sparse matrix, which is applied to a static random access memory configured with a preset comparator circuit, comprising:

[0050] Step S11: When receiving a sparse matrix operation request, the ratio of the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory is calculated to obtain target sparsities corresponding to each row of the target sparse matrix.

[0051] In this embodiment, when a sparse matrix operation request is received, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory according to a preset compression format, as a first sparse matrix and a second sparse matrix; wherein the first sparse matrix and the second sparse matrix are sparse matrices participating in the matrix operation. It can be understood that the target sparse matrices involved in the sparse matrix operation request are written into different positions of the storage array of the static random access memory SRAM (Static Random-Access Memory) according to the specified compression format. It should be noted that when writing to the SRAM in a preset compression format, because the number of elements in the sparse matrix is ​​too large, in order to reduce the computational overhead of subsequent matrix operations, generally only the non-zero elements of each sparse matrix are written, that is, only the non-zero elements with actual values ​​are paid attention to and saved. At the same time, in order to accurately restore the structure and position information of the sparse matrix in subsequent operations and processing, the indexes corresponding to the non-zero elements are saved. Therefore, the first sparse matrix and the second sparse matrix obtained each carry corresponding index information and the first sparse matrix and the second sparse matrix both follow specific and clear data structure rules to facilitate unified processing and operation in different computing environments and algorithms.

[0052] In this embodiment, the number of non-zero elements and the total number of elements of each row of non-zero elements are obtained, and the number of non-zero elements and the total number of elements are used to calculate the number of non-zero elements to obtain the target sparsity corresponding to each row of the target sparse matrix. It is understandable that according to the number of non-zero elements and the total number of elements in each row represented by the row pointers of the first sparse matrix and the second sparse matrix, the number of non-zero elements accounts for the total number of elements to obtain the target sparsity of each row. The higher the number of non-zero elements, the higher the sparsity, and vice versa.

[0053] Step S12: comparing the target sparsity with a preset sparsity threshold to obtain a corresponding comparison result.

[0054] In this embodiment, a sparsity threshold is set in advance, and the threshold is used to distinguish which block processing strategy is suitable for each row of the target sparse matrix. Therefore, after the preset sparsity threshold is set according to the index matching case of the sparse matrix, the size relationship of each determined target sparsity is compared with the preset sparsity threshold to obtain respective comparison results.

[0055] Step S13: selecting a corresponding block partitioning strategy according to the comparison result to obtain matrix partitioning of each target sparse matrix.

[0056] In this embodiment, if the comparison result is that the target sparsity is less than the preset sparsity threshold, the preset large-granularity block strategy is used for block division to obtain the corresponding matrix block to be compared; if the comparison result is that the target sparsity is greater than or equal to the preset sparsity threshold, the preset small-granularity block strategy is used for block division to obtain the corresponding matrix block to be compared. It can be seen that when the target sparsity is less than the preset sparsity threshold, it indicates that the number of non-zero elements in the current row of the corresponding target sparse matrix is ​​small, and the block granularity can select a large-granularity block strategy. By performing block division of rows of different target sparsities at different granularities to obtain the corresponding matrix block to be compared, the number of comparisons can be reduced and the comparison overhead can be reduced. When the target sparsity is greater than or equal to the preset sparsity threshold, it indicates that the number of non-zero elements in the current row of the corresponding target sparse matrix is ​​large, and the block granularity can select a small-granularity block strategy to obtain the corresponding matrix block to be compared. Avoid missing non-zero elements. It should be noted that when the matrix blocks to be compared are obtained, the block indexes after the blocks and the original index information (index information when compressed in a preset compression format) are stored in the SRAM for subsequent block comparison and index matching.

[0057] For example, for rows with higher sparsity (fewer non-zero elements), larger granularity blocks can be selected, that is, each block contains more rows; while for rows with lower sparsity (relatively more non-zero elements), smaller granularity blocks may be selected, each block contains fewer rows.

[0058] Specifically, large-granularity blocking means merging multiple rows into a larger block for processing, which can reduce the number of blocks and reduce the complexity of processing. It is especially suitable for rows with very few non-zero elements, which can reduce the number of comparisons and related overhead. Small-granularity blocking divides rows into smaller blocks, which is suitable for rows with relatively more non-zero elements. It may process the calculations or operations of these rows more carefully, but the number of blocks may be larger.

[0059] By selecting row blocks of different granularities according to sparsity, storage space and computing resources can be used more effectively, and the processing efficiency of sparse matrices can be improved. This method can flexibly deal with rows with different sparsity characteristics and better balance storage and computing overhead. For example, when processing operations such as sparse matrix multiplication, using a block strategy of appropriate granularity according to the sparsity of the row can reduce unnecessary computing and storage operations, thereby improving the performance of the overall operation.

[0060] In this embodiment, the matrix blocks to be compared of the first sparse matrix are row blocks to be compared, and the matrix blocks to be compared of the second sparse matrix are column blocks to be compared. It can be understood that, because the first sparse matrix and the second sparse matrix are compressed and written into the SRAM according to their respective preset compression formats, after processing each row of each sparse matrix according to the block strategy, the matrix blocks obtained are row blocks to be compared and column blocks to be compared.

[0061] Step S14: using the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

[0062] In this embodiment, the preset comparator circuit is used to perform index matching on the block indexes of each matrix block to be compared of the target sparse matrix participating in the matrix operation to obtain the corresponding block index matching result; if the block index matching is successful, the intra-block index matching step is performed. It can be understood that since the corresponding positions in the SRAM respectively store the block indexes of the matrix blocks to be compared of the first sparse matrix and the second sparse matrix participating in the operation. Among them, when performing index matching, it is not necessary to send the index data to be compared located in the memory to the general processor according to the data read instruction as in the prior art, and perform index data matching in the general processor. The index matching of the present invention directly uses the comparison circuit of the SRAM to perform index matching. Specifically, it is divided into two steps: one is block index matching, and the other is intra-block index matching after block index matching. It should be noted that the index matching of the present invention is all in the SRAM, and there is no need to transfer the index data.

[0063] It can be seen that the present application discloses an in-memory index matching method for a sparse matrix, which is applied to a static random access memory configured with a preset comparator circuit, including: when receiving a sparse matrix operation request, the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory is calculated to obtain each target sparsity corresponding to each row of the target sparse matrix; compare the size relationship between each target sparsity and a preset sparsity threshold to obtain a corresponding comparison result; select a corresponding block strategy according to the comparison result to obtain a matrix block of each target sparse matrix; use the preset comparator circuit to perform index matching on each matrix block to be compared of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result. It can be seen that index matching is performed directly in the static random access memory, which avoids the back and forth movement of a large amount of data between the memory and the processor, and significantly reduces the time and energy consumption caused by data movement. It can flexibly deal with matrices of different sparsity and select appropriate blocking strategies according to specific circumstances. It has stronger adaptability and versatility. It can select blocking strategies according to sparsity, and can process matrix rows of different sparsity in a more targeted manner, reduce unnecessary comparison operations, and thus improve the overall computing efficiency. It performs index matching in the memory by presetting the comparator circuit, avoiding the complex index matching calculation process of the general processor, reducing the complexity and time cost of the calculation, and can significantly improve the performance of the entire system when processing sparse matrix operations, speed up the processing speed, and provide more efficient support for related applications.

[0064] Reference Figure 2 As shown, the embodiment of the present invention discloses a specific in-memory index matching method for sparse matrices. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically:

[0065] Step S21: when receiving a sparse matrix operation request, determining the operation type representing the target sparse matrix in the sparse matrix operation request.

[0066] In this embodiment, when a sparse matrix operation request is received, the operation type representing the target sparse matrix in the sparse matrix operation request is first determined, where the operation type can be a matrix addition operation type, a matrix multiplication operation type, etc., without specific limitation.

[0067] Step S22: When the operation type is a matrix multiplication operation type, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory according to a preset compressed sparse row format and a preset compressed sparse column format, as a first sparse matrix and a second sparse matrix.

[0068] In this embodiment, when the operation type is a matrix multiplication operation type, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory SRAM according to a preset compressed sparse row format and a preset compressed sparse column format, as a first sparse matrix and a second sparse matrix. Figure 3 As shown, when the first sparse matrix is ​​a sparse matrix A, the sparse matrix is ​​compressed and stored according to the preset compressed row format CSR (Compressed Sparse Row), and the storage format is: the row pointer (Row_ptr), column index (Column_index), and non-zero element value (Value) of the sparse matrix are stored. The row pointer points to the storage location of the first non-zero element in each row. For example, the position of the first non-zero element in the first row, Row_ptr, is 0, the position of the first non-zero element in the second row, Row_ptr, is 2, and so on. The column index indicates the column corresponding to each non-zero element. For example, the column number Column_index corresponding to the first non-zero element (0,0) is 0, and the column number Column_index corresponding to the second non-zero element A(0,5) is 5. Finally, the value of the non-zero element Value of the first non-zero element is 1, and the value of the non-zero element Value of the second non-zero element is 3. When the second sparse matrix is ​​also a sparse matrix A, the compressed sparse column format CSC (Compressed Sparse Row) is preset. The CSC format is similar to the CSR format. The CSC format is column-first storage and consists of a column pointer (Col_ptr), a row index (Row_index) and a non-zero element value. Sparse compression coding only stores the values ​​of non-zero elements that have practical significance, which can greatly compress storage space and improve system performance.

[0069] It can be seen that when encoding and storing the first sparse matrix, the matrix is ​​traversed row-first, the values ​​of non-zero elements are saved in the Value array, and the corresponding column index is saved in the Column_index array. In the Row_ptr array, the i-th element records the index of the first non-zero element in the i-th row in the Value array, and the value of the last element is the total number of non-zero elements.

[0070] The encoding process of the second sparse matrix is ​​similar to the encoding process of the first sparse matrix, which will not be described in detail.

[0071] Step S23: Obtain the number of non-zero elements and the total number of elements in each row of non-zero elements, and use the number of non-zero elements and the total number of elements to calculate the proportion of the number of non-zero elements to obtain each target sparsity corresponding to each row of the target sparse matrix.

[0072] In this embodiment, when the first sparse matrix is ​​decoded to perform sparsity calculation, the element value x in the Value array (assuming it is marked as index_x) is traversed, and the column index of x in the original matrix (recorded as col_index) is taken out from the corresponding position in the Column_index array, and then index_x is searched in the Row_ptr array. If index_x is in Row_ptr, its subscript in Row_ptr is the row index of x in the original matrix (recorded as Row_index_x); if index_x is not in Row_ptr, the previous Row value is used (because the encoding is row by row, and Row_ptr only saves the first non-zero element of each row, so the current x is in the same row as the previous x). When the number of non-zero elements and the total number of elements in each row can be obtained according to the decoding information, the target sparsity calculation is then performed. For the calculation process of the target sparsity, please refer to the aforementioned disclosed embodiment content, which will not be repeated here. During the calculation process, in order to reduce the overhead of index matching, the sparsity of each row can be known based on the row pointer, such as Figure 3 The target sparsity of the first row in is 2 / 8. In order to make the block large and have as many zero element blocks as possible, the present invention has adopted the block strategy of different granularities. When the row sparsity is high, the block mode with larger granularity is selected, and when the row sparsity is low, the block mode with smaller granularity is selected. According to the sparsity of each row, the index after the block and the original index are stored in the SRAM at the same time, and this part of the operation is completed by the storage controller. The coding process of the second sparse matrix is ​​similar to the coding process of the first sparse matrix, which will not be repeated here.

[0073] Reference Figure 4 As shown in , when the operation type is matrix multiplication operation type, the inner product process of row multiplication is performed. Assume that Figure 3 The inner product multiplication of the matrix A in the first sparse matrix A is performed, and the first sparse matrix A is multiplied by the second sparse matrix A to obtain the matrix C. The first sparse matrix A is stored in CSR format, and the second sparse matrix A is stored in CSC format. Each row of the first A matrix is ​​multiplied by each column of the second A matrix to obtain the element at the corresponding position of the matrix C. Figure 4 The process described in the figure is to multiply the 0th row of the first sparse matrix A by the 0th column of the second sparse matrix A to obtain C(0,0). Since the matrix is ​​stored in a compressed manner, it is necessary to first find the elements at the corresponding position before performing the multiplication and accumulation operation, that is, index matching. In the actual execution process, it is necessary to traverse the elements of the 0th column of the second matrix A to find out whether there is a corresponding index in the first matrix A. For example, the index 0 and the index 0 in the figure match successfully, and then the matrix multiplication operation can be performed.

[0074] Step S24: comparing the target sparsity with a preset sparsity threshold to obtain a corresponding comparison result.

[0075] Step S25: selecting a corresponding block partitioning strategy according to the comparison result to obtain matrix partitioning of each target sparse matrix.

[0076] For more detailed processing procedures in steps S24 and S25, please refer to the aforementioned disclosed embodiments, which will not be described in detail here.

[0077] Step S26: Using the bit line calculation operation of the in-memory calculation in the static random access memory, the operands of each matrix block to be compared of the first sparse matrix and the second sparse matrix are amplified through a sensitive amplifier to obtain an amplified matrix block of the first sparse matrix and an amplified matrix block of the second sparse matrix.

[0078] In this embodiment, Figure 5 As shown, the block index (A matrix index) of the first sparse matrix and the block index (B matrix index) of the second sparse matrix are stored in different storage arrays (SRAM Array) in the SRAM respectively. Next, the A matrix index and the B matrix index at the corresponding position will perform index matching operations. The block index matching steps are: after the operands to be compared are amplified by the sensitive amplifier (SA) through the bit line calculation operation calculated in the SRAM memory, the block index comparison is performed in the comparator (Comparator) at the bottom. If the block indexes of the corresponding positions of the blocks can be matched, a finer-grained index matching will be performed to find the last non-zero element at the corresponding position. After that, other components will read the matched non-zero elements into the processor for corresponding matrix type calculations. In this way, large-scale comparison operations with line granularity can be realized, and the above comparison operations reduce the overhead of data movement.

[0079] Among them, the bitline calculation operation refers to the related calculation operation performed on the bitline of the memory cell in the static random access memory (SRAM) array. The SRAM array consists of multiple memory cells arranged in rows and columns. All memory cells in a row share a wordline, and all memory cells in a column share a pair of bitlines. In in-memory calculations, the bitline calculation operation is usually used to implement functions such as logical operations or data comparison. For example, in the process of multiplying two sparse matrices, by activating a specific wordline, the memory cell row to be involved in the operation can be selected. Then the voltage change on the bitline is used to represent the value of the data in the memory cell, and the bitline voltage is sensed and amplified with the help of components such as a sensitive amplifier, so as to obtain the logical operation result or comparison result of the memory cell storage bit. The specific bitline calculation operation method will vary depending on the design of the SRAM cell and the function implemented. For example, in some cases, bit line calculations may be performed based on specific logical forms (such as Sum-of-Products, SOP) to achieve various logical operations; or like the new 11TSRAM unit, it may work in different modes (such as normal read and write SRAM mode, logic mode for in-memory logic operations, BCAM mode (Binary Content Addressable Memory) and TCAM mode (Ternary Content Addressable Memory), etc.), and support row and column operations to meet different computing and data processing needs. This in-memory computing method can avoid frequent data movement between memory and processor, thereby improving computing power and reducing power consumption. It enables efficient data processing and logic operations within the memory array, which is of great significance to improving system performance and reducing data movement overhead.

[0080] Step S27: performing index matching on the enlarged matrix blocks of the first sparse matrix and the enlarged matrix blocks of the second sparse matrix through a preset comparator circuit to obtain corresponding in-memory index matching results.

[0081] In this embodiment, refer to Figure 6 As shown, Figure 6For matrix block index matching, the proportion of non-zero elements in sparse matrices used in actual applications is very low. If the search is performed one by one, when the second row of the first sparse matrix A is multiplied by the seventh column of the second sparse matrix A as shown in the figure, it is necessary to search for each element of the seventh column of the second sparse matrix A to find whether the second row of the first sparse matrix A has a corresponding value. This search operation requires 6 times. After the block is divided, the second row of the first sparse matrix A has only one non-zero row block A1 (2, 0), and the seventh column of the second sparse matrix A has only one non-zero column block A2 (1, 7). In this way, in the actual execution process, only one search operation is needed to make a judgment, which improves the efficiency of index matching.

[0082] It can be seen that by adopting the method of performing index matching in SRAM memory, the index matching operation that originally needs to be performed in the processor core is processed in SRAM memory, which can reduce the overhead of data movement between the processor and the memory, and more efficient data processing can be performed in the memory. In addition, the present invention designs an efficient index matching method to improve the efficiency of index matching.

[0083] Reference Figure 7 As shown, the present invention also discloses a sparse matrix in-memory index matching device, which is applied to a static random access memory configured with a preset comparator circuit, comprising:

[0084] The sparsity calculation module 11 is used for calculating the ratio of the number of non-zero elements of the target sparse matrix in the storage array of the static random access memory when receiving a sparse matrix operation request, so as to obtain each target sparsity corresponding to each row of the target sparse matrix;

[0085] A comparison module 12 is used to compare the size relationship between each of the target sparsities and a preset sparsity threshold to obtain a corresponding comparison result;

[0086] A matrix partitioning module 13, configured to select a corresponding partitioning strategy according to the comparison result to obtain matrix partitioning of each target sparse matrix;

[0087] The index matching module 14 is used to use the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

[0088] It can be seen that the present application discloses that when receiving a sparse matrix operation request, the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory is calculated to obtain each target sparsity corresponding to each row of the target sparse matrix; compare the size relationship between each target sparsity and the preset sparsity threshold to obtain a corresponding comparison result; select a corresponding block strategy according to the comparison result to obtain the matrix block of each target sparse matrix; use the preset comparator circuit to perform index matching on each matrix block to be compared of the target sparse matrix participating in the matrix operation to obtain the corresponding in-memory index matching result. It can be seen that index matching is performed directly in the static random access memory, which avoids the back and forth movement of a large amount of data between the memory and the processor, and significantly reduces the time and energy consumption caused by data movement. It can flexibly deal with matrices of different sparsity and select appropriate blocking strategies according to specific circumstances. It has stronger adaptability and versatility. It can select blocking strategies according to sparsity, and can process matrix rows of different sparsity in a more targeted manner, reduce unnecessary comparison operations, and thus improve the overall computing efficiency. It performs index matching in the memory by presetting the comparator circuit, avoiding the complex index matching calculation process of the general processor, reducing the complexity and time cost of the calculation, and can significantly improve the performance of the entire system when processing sparse matrix operations, speed up the processing speed, and provide more efficient support for related applications.

[0089] Furthermore, the present application also discloses an electronic device. Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0090] Figure 8 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the in-memory index matching method of the sparse matrix disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0091] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0092] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0093] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0094] Among them, the operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the processor 21 on the massive data 223 in the memory 22, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the in-memory index matching method of the sparse matrix executed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. In addition to data transmitted from an external device received by the electronic device, the data 223 can also include data collected by its own input and output interface 25.

[0095] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the in-memory index matching method of the sparse matrix disclosed above is implemented. For the specific steps of the method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, and no further description will be given here.

[0096] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0097] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly with hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory RAM (Random Access Memory), memory, read-only memory ROM (Read Only Memory), electrically programmable EPROM (Electrically Programmable Read Only Memory), electrically erasable programmable EEPROM (ElectricErasable Programmable Read Only Memory), register, hard disk, removable disk, CD-ROM (CompactDisc-Read Only Memory), or any other form of storage medium known in the technical field.

[0098] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0099] The scheme provided by the present invention is introduced in detail above. Specific examples are used in this article to illustrate the principle and implementation mode of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for in-memory index matching of a sparse matrix, characterized in that: A static random access memory for configuring a preset comparator circuit, comprising: When receiving a sparse matrix operation request, calculating the ratio of the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory to obtain target sparsities corresponding to each row of the target sparse matrix; Comparing the magnitude relationship between each of the target sparsities and a preset sparsity threshold to obtain a corresponding comparison result; Selecting a corresponding block strategy according to the comparison result to obtain matrix blocks of each of the target sparse matrices; The preset comparator circuit is used to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

2. The in-memory index matching method of a sparse matrix according to claim 1, characterized in that: When receiving a sparse matrix operation request, calculating the ratio of the number of non-zero elements of the target sparse matrix in the sparse matrix operation request in the storage array of the static random access memory to obtain target sparsities corresponding to each row of the target sparse matrix, including: When receiving a sparse matrix operation request, writing the target sparse matrix in the sparse matrix operation request into different storage locations of the storage array of the static random access memory according to a preset compression format, as a first sparse matrix and a second sparse matrix; wherein the first sparse matrix and the second sparse matrix are sparse matrices participating in the matrix operation; The number of non-zero elements and the total number of elements in each row of non-zero elements are obtained, and the proportion of the number of non-zero elements is calculated using the number of non-zero elements and the total number of elements to obtain target sparsities corresponding to each row of the target sparse matrix.

3. The in-memory index matching method of a sparse matrix according to claim 2, characterized in that: The selecting a corresponding block strategy according to the comparison result to obtain matrix blocks of each target sparse matrix includes: If the comparison result is that the target sparsity is less than the preset sparsity threshold, then use the preset large-granularity block strategy to perform block division to obtain corresponding matrix blocks to be compared; If the comparison result is that the target sparsity is greater than or equal to the preset sparsity threshold, a preset small-granularity blocking strategy is used to perform blocking to obtain corresponding matrix blocks to be compared.

4. The in-memory index matching method of a sparse matrix according to claim 3, characterized in that: The matrix blocks to be compared of the first sparse matrix are row blocks to be compared, and the matrix blocks to be compared of the second sparse matrix are column blocks to be compared.

5. The in-memory index matching method of a sparse matrix according to claim 2, characterized in that: When receiving a sparse matrix operation request, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory according to a preset compression format as a first sparse matrix and a second sparse matrix, including: When receiving a sparse matrix operation request, determining an operation type representing a target sparse matrix in the sparse matrix operation request; When the operation type is a matrix multiplication operation type, the target sparse matrix in the sparse matrix operation request is written into different storage locations of the storage array of the static random access memory according to a preset compressed sparse row format and a preset compressed sparse column format, as a first sparse matrix and a second sparse matrix.

6. The in-memory index matching method of a sparse matrix according to claim 5, characterized in that: The using the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result includes: Amplifying the operands of each matrix block to be compared of the first sparse matrix and the second sparse matrix by a sensitive amplifier using a bit line calculation operation of an in-memory calculation in the static random access memory, thereby obtaining an amplified matrix block of the first sparse matrix and an amplified matrix block of the second sparse matrix; Index matching is performed on the enlarged matrix blocks of the first sparse matrix and the enlarged matrix blocks of the second sparse matrix through a preset comparator circuit to obtain corresponding in-memory index matching results.

7. The in-memory index matching method of a sparse matrix according to any one of claims 1 to 6, characterized in that: The using the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation to obtain a corresponding in-memory index matching result includes: Using the preset comparator circuit to perform index matching on the block indexes of each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding block index matching result; If the block index matches successfully, the intra-block index matching step is performed.

8. A sparse matrix in-memory index matching device, characterized in that: A static random access memory for configuring a preset comparator circuit, comprising: A sparsity calculation module, for, when receiving a sparse matrix operation request, calculating the ratio of the number of non-zero elements of the target sparse matrix in the storage array of the static random access memory in the sparse matrix operation request, so as to obtain each target sparsity corresponding to each row of the target sparse matrix; A comparison module, used for comparing the magnitude relationship between each of the target sparsities and a preset sparsity threshold to obtain a corresponding comparison result; A matrix partitioning module, used for selecting a corresponding partitioning strategy according to the comparison result to obtain matrix partitioning of each target sparse matrix; The index matching module is used to use the preset comparator circuit to perform index matching on each to-be-compared matrix block of the target sparse matrix participating in the matrix operation, so as to obtain a corresponding in-memory index matching result.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the in-memory index matching method for sparse matrices as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the in-memory index matching method for sparse matrices as described in any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Storage optimization method and device for hybrid expert model, equipment, medium and product

    CN119808861A

  • Data processing method and device for sparse matrix multiplication calculation, equipment and medium

    CN119848404A