Hash-based sparse matrix vector multiplication optimization method and device

Abstract

The application provides a hash-based sparse matrix vector multiplication optimization method, which is characterized by the following steps: dividing a sparse matrix to be multiplied into a plurality of sparse matrix blocks according to the hardware structure of a neural network hardware accelerator, performing linear hash mapping on the plurality of sparse matrix blocks to obtain a to-be-divided matrix; dividing the to-be-divided matrix into a plurality of sub-matrix blocks according to the size of the to-be-divided matrix and the hardware structure, and dividing parallel execution parts and competitive execution parts in the sub-matrix blocks; the neural network hardware accelerator performs competitive execution of a calculation task between the sub-matrix blocks and parallel execution of the calculation task in the sub-matrix blocks to obtain a plurality of sub-matrix calculation results, and restores the original order of writing through a hash table; and the plurality of sub-matrix calculation results are combined according to the original order to obtain a final result of matrix vector multiplication.

Description

Technical Field

[0001] This invention relates to the fields of computational data processing and parallel program task allocation, and particularly to a hash-based sparse matrix-vector multiplication optimization method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] In sparse matrix-vector multiplication, most computational resources are wasted because sparse matrices contain a large number of zero elements. Furthermore, due to the large size of sparse matrices, the corresponding vectors also require a large amount of storage, making it difficult to store them completely on the high-bandwidth storage of some edge devices. During the computation process, low-bandwidth storage needs to be accessed frequently, resulting in a large amount of memory access overhead.

[0003] Existing technology 1 proposes the CSR5 storage format based on the assumption that each thread has the same time overhead when processing the same number of non-zero elements. This format fills non-zero elements in the sparse matrix row by row into fixed-size matrix blocks, ensuring that each parallel unit processes the same number of non-zero elements with the same computational cost. Furthermore, it introduces... Figure 1 The method uses bit-flag arrays to assist in inter-thread parallelism. While it achieves good results on small-scale sparse matrices, it does not take into account the large-scale nature of sparse matrices in real-world applications. When the input is a large-scale sparse matrix, the elements processed by different threads within the same thread bundle at the same time may be located in storage with different bandwidths. This can lead to different memory access times, thus violating the above assumptions. Consequently, the actual running speed of each thread within the same thread bundle is limited by the thread accessing the low-bandwidth storage.

[0004] Existing technology 2 addresses the memory access problem when processing large-scale sparse matrices and proposes the Regu2D storage format. This format employs a two-dimensional partitioning method, further dividing the sparse matrix-vector multiplication into two steps: independent matrix-vector multiplication of each sub-matrix block, and combining the resulting intermediate vectors. Figure 2 As shown. This method employs dynamic programming, calculating rows with similar numbers of non-zero elements within the same thread bundle, and introduces zero-padding to ensure each thread bundle has the same computational task. While this method achieves good computational speed, the time cost of the dynamic programming process is significant, and the introduction of zero-padding results in an unfixed storage length. This makes it difficult to accelerate the format conversion process in the preprocessing step through parallel processing.

[0005] First, the aforementioned prior art 1, while ensuring parallel load balancing, neglects the consideration of memory access. Due to the characteristics of computation, threads access vectors randomly, and high-bandwidth storage is insufficient to store complete vectors. This leads to frequent access to low-bandwidth storage, resulting in significant memory access time overhead. In contrast, this invention uses a two-dimensional partitioning method to ensure the locality of vector access within each sub-matrix block. Therefore, only vector fragments need to be stored on the high-bandwidth GPU shared memory, reducing the time overhead caused by memory access. Second, the aforementioned prior art 2 uses a two-dimensional partitioning method to ensure parallel load balancing within sub-matrix blocks, but it does not consider that the load may differ between sub-matrix blocks. This is due to the distribution of non-zero elements in sparse matrices, and the current two-dimensional partitioning method does not effectively achieve load balancing between thread bundles. Third, both prior art 1 and prior art 2 suffer from high preprocessing costs, necessitating the design of a lightweight, parallelizable preprocessing method. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of uneven parallel load between blocks and excessive preprocessing cost in the prior art, and to propose a hash-based optimization method for sparse matrix-vector multiplication.

[0007] Specifically, addressing the shortcomings of existing technologies, such as Figure 7 As shown, this invention proposes a hash-based optimization method for sparse matrix-vector multiplication, which includes:

[0008] The initial step involves dividing the sparse matrix to be multiplied by the matrix vectors into multiple sparse matrix blocks based on the hardware structure of the neural network hardware accelerator. Then, a linear hash mapping is performed on these multiple sparse matrix blocks to obtain the matrix to be divided.

[0009] The partitioning step involves dividing the matrix to be partitioned according to its size and the hardware structure, resulting in a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein each of the parallel execution part and the competitive execution part contains multiple sub-matrix blocks.

[0010] In the execution steps, the neural network hardware accelerator performs computation tasks through competition between submatrix blocks and parallel computation tasks within blocks to obtain multiple submatrix computation results, and restores the original writing order by looking up the hash table;

[0011] The merging step combines the calculation results of the multiple submatrices according to the original order to obtain the final result of matrix-vector multiplication.

[0012] The hash-based sparse matrix-vector multiplication optimization method, wherein the initial step includes:

[0013] Based on the hardware architecture, a two-dimensional partitioning model for the input sparse matrix is ​​constructed. Linear hash mapping is performed on the sparse matrix blocks to convert the sparse matrix storage format into the HBP format of the matrix to be partitioned. Specifically:

[0014] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing steps mainly consist of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the position of the first non-zero element in each row of the currently processed sparse matrix block through binary search, and sequentially traverses until the position of the last element in the corresponding row within the range of the current sparse matrix block.

[0015] The linear hash mapping within a sparse matrix block takes the number of non-zero elements in each row as input. Leveraging the property of linear hash transformation to cluster similar elements, it maps the row numbers within the sparse matrix block to a hash table of the same size as before, ensuring that rows with the same or similar number of non-zero elements are mapped to adjacent positions. The result of the hash transformation is stored in a fixed-length hash table, the length of which is equal to the sum of the number of rows in each sparse matrix block after the two-dimensional partitioning, and its size is the total number of rows in the sparse matrix, num_of_row, multiplied by the number of sparse matrix blocks in the column direction, block_col_num. The initial value of the hash table is -1, indicating that the current position is empty. The index of the hash table is the actual execution order, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0016]

[0017] In the formula, `output_hash` represents the position of the corresponding row in the output. Calculations are performed sequentially based on this position during matrix-vector multiplication, and its range is greater than or equal to 0 and less than the number of rows in the block, `end_row`. `block_row_nnz` represents the number of non-zero elements in that row within the sparse matrix block, obtained during the data start / end position search step. `HBP_M` and `HBP_N` represent the row and column sizes of the two-dimensional partition, respectively. `ρ` represents the overall sparsity of the input matrix, and `ω` represents the size of a set of thread bundles in the single-instruction multithreaded mode of the neural network accelerator. The value of `HBP_M` must be an integer multiple of `ω`. `i` is the position offset, which is changed to find a nearby free position when a hash mapping collision occurs. `end_row` represents the number of rows contained in the sparse matrix block. The last row of the sparse matrix block may contain fewer rows than `HBP_M`, requiring a judgment. The specific value of `end_row` is shown in the following formula:

[0018]

[0019] In the formula, num_of_row is the total number of rows of the input matrix, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block being processed, and the value of block_m is greater than or equal to 0 and less than block_row_num.

[0020] The HBP storage format includes non-zero element column numbers (cols), non-zero element values ​​(data), non-zero element indexes (add_sign) for accessing non-zero elements within a block, zero row index (zero_row), the location of the first non-zero element in each block (begin_nnz), and a hash table (output_hash). The lengths of the non-zero element column numbers (cols) and non-zero element values ​​(data) in the HBP storage format, as well as the length of the non-zero element index (add_sign), are equal to the number of non-zero elements. The lengths of the zero row index (zero_row) and the hash table (output_hash) are equal to the number of rows multiplied by the number of column blocks. The location of the first non-zero element in each block (begin_nnz) is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and retrieval.

[0021] The aforementioned hash-based sparse matrix-vector multiplication optimization method includes the following partitioning step:

[0022] The division into parallel execution and competitive execution parts is calculated based on the size of the sparse matrix and the size of the sparse matrix blocks. The process of dividing the parallel execution part follows the principle that each thread is allocated the same number of block computation tasks, and the blocks to be computed in the same thread should be located in the same column as much as possible. The parts that are not divided into parallel execution parts are classified as competitive execution parts. In the competitive execution part, each sub-matrix block is competitively executed by the thread bundle that finishes its assigned task first during program execution.

[0023] The aforementioned hash-based sparse matrix-vector multiplication optimization method includes the following execution steps:

[0024] Competitive execution of computation tasks between submatrix blocks refers to a process where each group of ω threads accesses and processes several blocks of computation tasks within the group. Threads within and between groups execute in parallel. Once a group of threads has completed its parallel execution, it selects a submatrix block from the competitive execution section that has not yet been executed to perform the computation task.

[0025] The competitive execution section uses a ticket number lock to avoid multiple accesses to the same submatrix block. The ticket number lock is an integer variable, which is modified with the volatile keyword during parallel computation to indicate that no compilation optimization is performed on this variable. Each access is read from storage. This variable stores the number of submatrix blocks that are currently not being computed. The initial value is equal to the number of submatrix blocks contained in the competitive execution section. After each group of threads completes the computation of the current task, it atomically accesses this variable to obtain the next computation task. If the return value is negative, it means that there are no unexecuted computation tasks in the competitive execution section. The content that needs to be randomly accessed multiple times during the computation of each two-dimensional submatrix block, as well as the current execution block sequence number obtained by the ticket number lock;

[0026] The index of the hash table is the actual execution order of the threads, and the stored value is the order of each row before the hash transformation. Therefore, after each thread of the neural network accelerator has calculated a row of the submatrix block, it needs to access the corresponding position in the hash table and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0027] like Figure 8 As shown, this invention also proposes a hash-based sparse matrix-vector multiplication optimization device, which includes:

[0028] The initial module, based on the hardware structure of the neural network hardware accelerator, divides the sparse matrix to be multiplied by the matrix vector to obtain multiple sparse matrix blocks, and performs linear hash mapping on these multiple sparse matrix blocks to obtain the matrix to be divided.

[0029] The partitioning module divides the matrix to be partitioned according to its size and the hardware structure, resulting in a parallel execution part and a competitive execution part, each of which contains multiple sub-matrix blocks; that is, the sub-matrix blocks are divided into the parallel execution part and the competitive execution part.

[0030] The execution module, a neural network hardware accelerator, performs computational tasks through competition between submatrix blocks and parallel computational tasks within blocks, obtaining multiple submatrix computation results, and restoring the original writing order by looking up a hash table.

[0031] The merging module combines the calculation results of the multiple submatrices according to the original order to obtain the final result of matrix-vector multiplication.

[0032] The hash-based sparse matrix-vector multiplication optimization device, wherein the initial module includes:

[0033] Based on the hardware architecture, a two-dimensional partitioning model for the input sparse matrix is ​​constructed. Linear hash mapping is performed on the sparse matrix blocks to convert the sparse matrix storage format into the HBP format of the matrix to be partitioned. Specifically:

[0034] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the position of the first non-zero element in each row of the currently processed sparse matrix block through binary search, and sequentially traverses until the position of the last element in the corresponding row within the range of the current sparse matrix block.

[0035] The linear hash mapping within a sparse matrix block takes the number of non-zero elements in each row as input. Leveraging the property of linear hash transformation to cluster similar elements, it maps the row numbers within the sparse matrix block to a hash table of the same size as before, ensuring that rows with the same or similar number of non-zero elements are mapped to adjacent positions. The result of the hash transformation is stored in a fixed-length hash table, the length of which is equal to the sum of the number of rows in each sparse matrix block after the two-dimensional partitioning, and its size is the total number of rows in the sparse matrix, num_of_row, multiplied by the number of sparse matrix blocks in the column direction, block_col_num. The initial value of the hash table is -1, indicating that the current position is empty. The index of the hash table is the actual execution order, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0036]

[0037] In the formula, `output_hash` represents the position of the corresponding row in the output. Calculations are performed sequentially based on this position during matrix-vector multiplication, and its range is greater than or equal to 0 and less than the number of rows in the block, `end_row`. `block_row_nnz` represents the number of non-zero elements in that row within the sparse matrix block, obtained from the data start / end position search module. `HBP_M` and `HBP_N` represent the row and column sizes of the two-dimensional partition, respectively. `ρ` represents the overall sparsity of the input matrix, and `ω` represents the size of a set of thread bundles in the single-instruction multithreaded mode of the neural network accelerator. The value of `HBP_M` must be an integer multiple of `ω`. `i` is the position offset; when a hash mapping collision occurs, `i` is changed to find a nearby free position. `end_row` represents the number of rows contained in the sparse matrix block. The last row of the sparse matrix block may contain fewer rows than `HBP_M`, requiring a judgment. The specific value of `end_row` is shown in the following formula:

[0038]

[0039] In the formula, num_of_row is the total number of rows of the input matrix, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block being processed, and the value of block_m is greater than or equal to 0 and less than block_row_num.

[0040] The HBP storage format includes non-zero element column numbers (cols), non-zero element values ​​(data), non-zero element indexes (add_sign) for accessing non-zero elements within a block, zero row index (zero_row), the location of the first non-zero element in each block (begin_nnz), and a hash table (output_hash). The lengths of the non-zero element column numbers (cols) and non-zero element values ​​(data), as well as the length of the non-zero element index (add_sign), are equal to the number of non-zero elements. The lengths of the zero row index (zero_row) and the hash table (output_hash) are equal to the number of rows multiplied by the number of column blocks. The location of the first non-zero element in each block (begin_nnz) is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and retrieval.

[0041] This execution module includes:

[0042] Competitive execution of computation tasks between submatrix blocks refers to a process where each group of ω threads accesses and processes several blocks of computation tasks within the group. Threads within and between groups execute in parallel. Once a group of threads has completed its parallel execution, it selects a submatrix block from the competitive execution section that has not yet been executed to perform the computation task.

[0043] The competitive execution section uses a ticket number lock to avoid multiple accesses to the same submatrix block. The ticket number lock is an integer variable, which is modified with the volatile keyword during parallel computation to indicate that no compilation optimization is performed on this variable. Each access is read from storage. This variable stores the number of submatrix blocks that are currently not being computed. The initial value is equal to the number of submatrix blocks contained in the competitive execution section. After each group of threads completes the computation of the current task, it atomically accesses this variable to obtain the next computation task. If the return value is negative, it means that there are no unexecuted computation tasks in the competitive execution section. The content that needs to be randomly accessed multiple times during the computation of each two-dimensional submatrix block, as well as the current execution block sequence number obtained by the ticket number lock;

[0044] The index of the hash table is the actual execution order of the threads, and the stored value is the order of each row before the hash transformation. Therefore, after each thread of the neural network accelerator has calculated a row of the submatrix block, it needs to access the corresponding position in the hash table and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0045] The hash-based sparse matrix-vector multiplication optimization device, wherein the partitioning module includes:

[0046] The division into parallel execution and competitive execution parts is calculated based on the size of the sparse matrix and the size of the sparse matrix blocks. The process of dividing the parallel execution part follows the principle that each thread is allocated the same number of block computation tasks, and the blocks to be computed in the same thread should be located in the same column as much as possible. The parts that are not divided into parallel execution parts are classified as competitive execution parts. In the competitive execution part, each sub-matrix block is competitively executed by the thread bundle that finishes its assigned task first during program execution.

[0047] The present invention also proposes an electronic device, including the aforementioned hash-based sparse matrix-vector multiplication optimization device, which may be connected to an information display device for displaying the final result of the matrix-vector multiplication using user-set display parameters, attributes, or through an artificial intelligence model.

[0048] The present invention also proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the hash-based sparse matrix-vector multiplication optimization method.

[0049] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the steps of the hash-based sparse matrix-vector multiplication optimization method.

[0050] As can be seen from the above solutions, the advantages of the present invention are:

[0051] This invention retains the performance advantages of the two-dimensional partitioning method in sparse matrix-vector multiplication, further solves the problem of excessive preprocessing cost in the two-dimensional partitioning sparse matrix method, and adopts the work-stealing method to achieve parallel load balancing between sub-matrix blocks, further improving the speed of matrix-vector multiplication. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the existing technology in CSR5 format;

[0053] Figure 2 This is a schematic diagram of sparse matrix-vector multiplication in existing two-dimensional partitioning methods.

[0054] Figure 3 This is a flowchart illustrating the hash-based sparse matrix-vector multiplication optimization method of the present invention.

[0055] Figure 4 This is a schematic diagram of the process of converting CSR format to HBP format in this invention;

[0056] Figure 5 This is a schematic diagram of dividing the sparse matrix into parallel execution part and competitive execution part in step S2 of the present invention;

[0057] Figure 6 This is a schematic diagram showing the range covered by a single execution unit in each parallel step of the present invention;

[0058] Figure 7 This is a flowchart of the method of the present invention;

[0059] Figure 8 This is a block diagram of the device of the present invention;

[0060] Figure 9 This is a schematic diagram of the structure of the first electronic device of the present invention;

[0061] Figure 10 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0062] Figure 11 This is a schematic diagram of the structure of the second electronic device of the present invention.

[0063] Figure label:

[0064] A - First electronic device;

[0065] B-Sparse Matrix Vector Multiplication Optimization Method Apparatus;

[0066] C-Data acquisition equipment;

[0067] D-Information display device;

[0068] 1000 - Second electronic device;

[0069] Ⅰ-Computational Unit;

[0070] II-ROM;

[0071] III-RAM;

[0072] N-bus;

[0073] V-Interface;

[0074] VI - Input Unit;

[0075] VII - Output Unit;

[0076] VIII - Storage medium;

[0077] IX - Communication Unit. Detailed Implementation

[0078] It should be noted that, in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0079] Without further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0080] The processor described in this invention is the control center of an electronic device. It can be a single processor or a collective term for multiple processing elements. For example, it can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of this invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0081] Alternatively, the processor can perform various functions of the electronic device by running or executing software programs stored in memory and by calling data stored in memory.

[0082] In a specific implementation, as one example, the processor may include one or more CPUs. Each of these processors may be a single-core processor or a multi-core processor. Here, "processor" can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions). Electronic devices may include servers, desktop computers, laptops, smartphones, tablets, embedded computers, etc., where the embedded computer includes vehicles and robots, etc.

[0083] The memory is used to store the software program that executes the solution of the present invention, and the execution is controlled by the processor. For specific implementation methods, please refer to the above method embodiments, which will not be repeated here.

[0084] It should be noted that the structure of the electronic device shown in the accompanying drawings of this invention does not constitute a limitation thereof. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0085] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0086] It should also be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0087] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0088] It should also be understood that, in various embodiments of the present invention, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0090] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0091] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0092] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] While researching sparse matrix-vector multiplication, the inventors discovered that the aforementioned shortcomings in existing technologies stemmed from insufficient consideration of parallel load balancing and memory access overhead. For example, Existing technology 1 only considered parallel load balancing without taking into account the potentially varying memory access time costs of different threads in the case of large-scale sparse matrices. Existing technology 2 uses a two-dimensional partitioning method, further dividing the sparse matrix-vector multiplication into two steps: independent matrix-vector multiplication of each sub-matrix block and the combination and merging of the resulting intermediate vectors. This helps reduce memory access overhead, and the use of dynamic programming and zero-padding mitigates the impact of uneven parallel load on computation speed. However, this method does not consider the time cost of the preprocessing step. The dynamic programming itself is quite complex, and zero-padding results in an unpredictable storage length, making it difficult to perform the preprocessing step in parallel. This requires an extremely large amount of time for large-scale sparse matrices. Two-dimensional partitioning can effectively solve the overhead problem of vector memory access. Therefore, this invention uses two-dimensional partitioning as the basic method. Regarding the problem of excessive preprocessing costs, the inventors discovered through research that this deficiency can be resolved through linear hashing. In hash transformation, collisions are often avoided, meaning multiple elements are mapped to the same location after hash transformation. However, the characteristic of clustering similar elements can also be utilized. This invention designs a hash function that uses the number of non-zero elements in each row as input, clustering rows with similar numbers of elements and dispersing rows with significantly different numbers. Since the time cost of hash transformation is relatively small, it can effectively reduce preprocessing costs. In contrast, existing technology 2 only considers parallel load balancing within sub-matrix blocks, but does not consider that the load may be different between sub-matrix blocks. Therefore, this invention does not use balancing of non-zero elements between matrix blocks, but instead divides the execution into parallel execution parts and competitive execution parts. The parallel execution part is pre-allocated to each thread for execution, and the competitive execution part is executed by the thread that has completed its assigned task first. This method achieves balance in actual execution time. Specifically, to achieve the above technical effects, this invention proposes the following key technical points:

[0094] Key Point 1: This invention proposes the HBP storage format. The HBP storage format comprises six parts: column indices (cols) for non-zero elements, value (data) for non-zero elements, index (add_sign) for accessing non-zero elements within a block, zero row index (zero_row), the location (begin_nnz) where the first non-zero element in each block is stored, and a hash table (outpu_hash). The HBP format ensures parallel computation between and within sub-matrix blocks in matrix-vector multiplication.

[0095] Key point 2: Lightweight preprocessing steps based on hashing; Hash transformation can aggregate rows of non-zero elements, thereby avoiding thread divergence, and hash transformation has a low time cost.

[0096] Key point 3: Competition among submatrix blocks to execute computational tasks; This invention divides the execution into parallel execution part and competitive execution part. The parallel execution part is pre-allocated to each thread for execution, while the competitive execution part is obtained by the thread that has completed its assigned task first. This method balances the actual execution time.

[0097] To make the above-mentioned features and effects of the present invention clearer and easier to understand, specific embodiments are described below in conjunction with the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are merely illustrative. The scope of protection of the present invention is not limited to the disclosed embodiments, but is defined by the appended claims.

[0098] Combination Figure 3 The implementation process of this invention is described below. The hash-based sparse matrix-vector multiplication optimization method of this invention includes the following steps:

[0099] S1. Based on the hardware structure, construct a two-dimensional partitioning model for the input sparse matrix, and perform linear hash mapping on the sub-matrix blocks after the two-dimensional partitioning; the hardware structure here refers to the maximum shared memory supported by each block of the GPU. This invention needs to copy the accessed vector fragments to the shared memory to avoid multiple accesses to the low-bandwidth global memory, thereby improving speed.

[0100] In step S1, the specific process includes: importing the data to be calculated, calculating the two-dimensional partition size, performing intra-block traversal statistics, linear hash mapping, and converting the storage format. Specifically, the data to be calculated includes a sparse matrix stored in CSR format and a vector. The default input format for importing the data is the CSR compressed storage format commonly used for sparse matrices. The method of this invention can be used in the step of calculating image features multiplied by weights in a neural network to improve the classification efficiency of image feature extraction and image classification. This invention first reads the sparse matrix to be calculated stored in CSR format and obtains the length of the matrix in the row and column directions and the number of non-zero elements. The storage required for the HBP format is calculated and initialized. The storage length of the HBP storage format can be determined solely by the size of the matrix and the number of non-zero elements. Compared to the classic storage format CSR, it achieves a speed improvement in computation by introducing a small number of arrays at the cost of increased storage. Furthermore, the HBP storage format does not introduce additional zero-element padding. The storage size occupied by each sub-matrix block after two-dimensional partitioning can be obtained before the storage format conversion step. This provides the conditions for parallel execution of format conversion on a sub-matrix block basis. The size of the sub-matrix block is determined by the hardware structure, i.e., the maximum shared memory supported by each block of the GPU. In SpMV computation, the vector fragments required by each matrix block are stored on shared memory, and are therefore limited by its size. The HBP storage format comprises six parts: column indices (cols) for non-zero elements, value (data) for non-zero elements, index (add_sign) for non-zero elements accessed within a block, zero row index (zero_row), the location (begin_nnz) where the first non-zero element in each block is stored, and a hash table (output_hash). In the HBP storage format, the lengths of the column indices (cols) and values ​​(data) for non-zero elements, and the length of the index (add_sign) for non-zero elements, are equal to the number of non-zero elements. The lengths of the zero row index (zero_row) and the hash table (output_hash) are equal to the number of rows multiplied by the number of column blocks. The location (begin_nnz) where the first non-zero element in each block is stored is equal to the total number of submatrix blocks. Compared to the classic CSR storage format, the HBP storage format only reorders the cols and data arrays without introducing additional zero-padding. It discards the ptr array and uses begin and add_sign to achieve parallel computation between blocks. Each submatrix block, after being divided in two dimensions, serves as the basic unit for data storage and retrieval, ensuring that the data required for sparse matrix-vector multiplication calculations in each submatrix block is centrally distributed in storage location.

[0101] The non-zero element index `add_sign` represents the position from the current non-zero element to the next non-zero element in the same row; you can think of it as representing the relative position of the non-zero element in the sparse matrix. The zero row index `zero_row` records all rows of the sparse matrix, not just the zero row. For rows with all zero values, we denote them as -1; for other rows, we record the number of zero rows preceding them. This is to allow each row to find the position of the first element of SpMV, thus enabling parallel computation.

[0102] This invention obtains a hash table from a sparse matrix in CSR compressed storage format through the following steps: Step 1. Hash table length and initialization: The purpose of hashing is to reorder the rows within a matrix block to solve the load balancing problem. Therefore, the length of the hash table is equal to the total number of rows after the two-dimensional partitioning, specifically equal to the number of rows in the original sparse matrix multiplied by the number of matrix blocks in the column direction. The initial value of all elements in the hash table is -1; Step 2. Hash table in HBP storage format: In step S1, the generation of the hash table is performed in parallel among matrix blocks. For any matrix block, we first count the number of non-zero elements in each row of the block, then substitute the number of non-zero elements into the hash function to obtain the mapped position, and write the row number before hash transformation at that position. This yields the hash table.

[0103] The calculation of the size of the two-dimensional partition includes calculating the number of blocks in the row and column directions of the two-dimensional partition matrix, and calculating the sparsity of the sparse matrix. The sparsity ρ is calculated as follows:

[0104]

[0105] In the formula, m is the number of rows in the sparse matrix, n is the number of columns in the sparse matrix, and nnz is the number of non-zero elements in the sparse matrix. These values ​​are usually stored directly in the sparse matrix file and can be obtained directly. The sparsity ρ reflects the density of non-zero elements in the sparse matrix, and its value ranges from 0 to 1. This value can be used to determine a suitable linear hash mapping to map rows with similar numbers of non-zero elements to locations with similar physical storage.

[0106] The intra-block traversal and statistics, as well as subsequent preprocessing steps, are executed in parallel. Each sub-matrix block after the two-dimensional partition is processed by only one thread. In the intra-block traversal and statistics step, each sub-matrix block first obtains the start and end positions of the row in the entire sparse matrix by accessing the CSR storage format. Then, it uses binary search to find the start position of the row in the current sub-matrix block. After that, it sequentially accesses the sub-matrix block from the start position until the column coordinates or row coordinates exceed the range of the current sub-matrix block or the row coordinates exceed the range of the current row. The number of non-zero elements in the current row is recorded at the corresponding position of num_per_row.

[0107] The linear hash mapping step performs a linear hash transformation on each row in the current sub-matrix block. Based on the number of non-zero elements in each row obtained in the previous step, the position after the hash transformation is calculated. In this step, computation is performed in parallel between sub-matrix blocks, and sequentially for each row within a sub-matrix block. The corresponding position in `output_hash` is accessed based on the result. The sequence number in `output_hash` implicitly represents the actual execution order of the threads, and the stored value is the position of each row before the hash transformation. The specific hash transformation used is shown in the following formula:

[0108]

[0109] In the formula, output_hash is the position of the corresponding row of the output. In matrix-vector multiplication, the calculation is performed according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block, end_row. block_row_nnz is the number of non-zero elements in the row of the sub-matrix block. This value is obtained in the data start and end position search step. HBP_M and HBP_N are the row and column sizes of the two-dimensional partition, respectively. The values ​​are related to the hardware platform on which it is running. ρ is the sparsity of the input matrix as a whole. Its range is between 0 and 1. ω is the size of a warp in single instruction multithreading (SIMT) mode. Taking the Nvidia GeForce MX250 GPU as an example, the thread bundle size ω is 32, and the maximum shared storage supported by each block is 48KB. Assuming that each block contains only one thread bundle, and each thread bundle occupies 48KB of shared storage, the reusable part in matrix-vector multiplication is the vector fragment, which is stored using the double type, with each element occupying 8B. Therefore, each thread bundle can store a vector fragment with a maximum length of 6K in the shared storage on average. Considering that the shared storage also needs to store the sub-matrix block number to be executed obtained from the ticket lock in the subsequent step S3, let each thread bundle store a vector fragment with a length of 4096, that is, the two-dimensional partition size HBP_N in the column direction is 4096. A value too small for HBP_M will result in a limited number of rows in the submatrix block, making it easier for rows with significantly different numbers of non-zero elements to be assigned to the same thread for execution. This can lead to thread divergence in step S3, affecting computation speed. A value too large will increase the computational load per thread, consuming more time during hash table lookup and impacting the speed of step S1. Furthermore, based on the computational characteristics of step S4, the value of HBP_M needs to be an integer multiple of ω. Considering these conditions, the value of HBP_M is set to 512. `i` is the position offset; when a hash mapping collision occurs, `i` is changed to find a nearby free position. `end_row` is the number of rows in the submatrix block. The last row block in a sparse matrix may contain fewer rows than HBP_M, requiring a check. The specific value of `end_row` is shown in the following formula:

[0110]

[0111] In the formula, num_of_row is the total number of rows in the input matrix, block_row_num is the number of sub-matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sub-matrix block being processed, and the value of block_m is greater than or equal to 0 and less than block_row_num.

[0112] The storage format conversion step is designed based on the memory access characteristics of thread bundles performing sparse matrix-vector multiplication in parallel on the GPU. During sparse matrix-vector multiplication, each non-zero element in the sparse matrix is ​​accessed only once, lacking temporal locality, making it unsuitable for storage using the GPU's high-bandwidth shared memory. On the GPU, when a group of thread bundles accesses contiguous memory addresses simultaneously, memory access merging can be used to reduce the number of accesses and improve the speed of sparse matrix-vector multiplication. The storage format conversion step uses the hash table obtained from the linear hash mapping step as the access index. Each time, it performs storage format conversion on rows equal to the thread bundle size ω. The number of non-zero rows j is calculated from the number of non-zero elements in each row of the block obtained from the block traversal statistics step. Zero rows are indexed as zero_row. If all elements in the row are zero, it is recorded as -1; if there is a non-zero element, the number of zero rows preceding that row is recorded, with a value ranging from -1 to ω. If there is at least one non-zero row in the current ω rows, then select the first non-zero element that has not yet been converted in each non-zero row, write the column number cols and the non-zero element value data stored in the original CSR format to the new position, and then record the number of the remaining non-empty rows in the non-zero element index add_sign. This value represents the bias from the current element to the next element in the same row. Then num_per_row is decremented by 1 to indicate that the element has completed the format conversion. If the value corresponding to the row in num_per_row is zero at this time, it means that the current element is the last non-zero element in the row within the submatrix block. Modify add_sign to -1 to indicate that the row is terminated and decrement the number of non-zero rows j by 1. When j is less than 0, it means that the current ω rows have completed the format conversion.

[0113] S2. Based on the size of the input sparse matrix and the hardware architecture of the running platform, divide the process into parallel execution and competitive execution parts. It's important to note that the hardware metrics focused here differ from those in step S1. Step S1 focuses on the size of high-bandwidth storage, such as the size of shared memory per block on a GPU. Here, step S2 focuses on the maximum number of blocks and the warp size, which represent computational power. For example, on a Nvidia GPU, these two numbers can be directly read, but their values ​​may differ on Nvidia GPUs with varying computational capabilities.

[0114] For step S2, the specific process includes: obtaining the required platform parameters, dividing the parallel execution part, and dividing the competitive execution part.

[0115] The maximum number of parallel threads P and the thread bundle size ω required by the hardware platform in step S2 may have different values ​​for different hardware platforms. Therefore, these parameters need to be obtained before performing parallel computation of sparse matrix-vector multiplication. Parameter P is equal to the maximum number of blocks multiplied by the thread bundle size ω. Step S2, which divides the parallel execution part and the competitive execution part, only divides the number of sub-matrix blocks, implicitly using a column-matrix-first indexing order. It does not involve parallel computation of reading elements in the matrix, so it can be executed simultaneously with step S1 on a serial computing platform. The hardware platform refers to a platform with parallel computing capabilities, such as a GPU hardware platform, which is the same platform as step S1. It should be noted that step S2 only requires some parameters of the GPU hardware platform, not computation on the GPU platform. Step S2 only divides the number of matrix blocks contained in each part. This part requires very little computing resources, so it can run on a CPU platform and run simultaneously with step S1. Moreover, this part requires very little time and can be hidden within the time of S1.

[0116] The parallel execution part allocates an equal number of sub-matrix blocks to each thread bundle. Under this premise, sub-matrix blocks located in the same column are allocated to a thread bundle whenever possible. Sub-matrix blocks in the same column require access to the same vector range during matrix-vector multiplication. Since the program will randomly access vector segments within this range multiple times during computation, the required vector segments are stored in shared memory with higher bandwidth to improve memory access speed. The parallel execution part first calculates the number of thread bundles obtainable by each column of sub-matrix blocks after the two-dimensional partition, and then obtains the number of parallel execution thread blocks allocated to each thread. The calculation of the number of thread bundles obtainable by each column of sub-matrix blocks after the two-dimensional partition is shown in the following formula:

[0117]

[0118] In the formula, ceil() represents rounding up, and the number of thread bundles obtained in each sub-matrix block is an integer greater than 0. After completing the step of dividing the parallel execution part, the remaining undivided part is the competitive execution part. Depending on the different divisions of the parallel execution part, the division results of the competitive execution part can be divided into three cases:

[0119] The first scenario is that the ratio of the maximum number of parallel threads P to the thread bundle size ω is exactly an integer multiple of the number of column-direction submatrix blocks block_col_num. In this case, each column submatrix block obtains the same number of thread blocks, the parallel execution part is located above the sparse matrix, and the competitive execution part is located below the sparse matrix.

[0120] The second scenario is when the ratio of the maximum number of parallel threads P to the thread bundle size ω is less than the number of submatrix blocks in the column direction (block_col_num). In this case, the submatrix blocks on the right are not assigned to the thread bundle, the parallel execution part is located on the left side of the sparse matrix, and the competitive execution part is located on the right side of the sparse matrix.

[0121] The third type: The ratio of the maximum number of parallel threads P to the thread bundle size ω is greater than the number of submatrix blocks in the column direction block_col_num and is not an integer multiple. In this case, each submatrix block is allocated a thread bundle, but the number of thread bundles allocated is not equal. The parallel execution part is located in the upper left part of the sparse matrix, while the competitive execution part is located in the lower right part of the sparse matrix.

[0122] The above three cases are calculated to obtain the number of sub-matrix blocks contained in the competitive execution part. This value and the number of parallel execution thread blocks allocated to each thread are passed as parameters into step S3.

[0123] S3. Computation tasks are executed in competition between sub-matrix blocks after two-dimensional partitioning, and computation tasks are executed in parallel within blocks. The original writing order is restored by looking up the hash table before the merging step.

[0124] For step S3, the specific process includes: reading the required data, performing matrix-vector multiplication calculations on the parallel execution part, and performing matrix-vector multiplication calculations on the competitive execution part. Step S3 is the core step of sparse matrix-vector multiplication calculation, which is executed on a device with parallel computing capabilities, such as a GPU.

[0125] In the required data reading step, each thread first obtains its own thread number and its sequence number in the current thread bundle. Based on this, the pre-partitioned parallel computing portion to be computed can be obtained. The pre-partitioned parallel computing portion corresponding to each thread bundle consists of several sub-matrix blocks located in the same column. Sub-matrix blocks in the same column access the same vector fragment during matrix-vector multiplication, meaning that the fragment will be repeatedly accessed during matrix-vector multiplication in the same column of sub-matrix blocks, exhibiting temporal locality. Therefore, it can be stored in high-bandwidth storage to improve memory access speed. In this invention, these vector fragments are stored in the shared storage of each thread bundle. A group of thread bundles checks whether there are non-zero blocks in the sub-matrix blocks to be executed. If so, the corresponding vector fragment is copied and stored in the shared storage. When executing the pre-partitioned parallel computing portion subsequently, it only needs to be read from the shared storage, which effectively reduces memory access time for matrix-vector multiplication, a computation that requires multiple random accesses.

[0126] In the parallel execution part of the matrix-vector multiplication calculation step, the number of parallel execution thread blocks allocated to each thread obtained from step S2, combined with the thread bundle number, can determine the position of the executed sub-matrix block. When the sub-matrix block is located at the bottom of the original sparse matrix, there may be a situation where the sub-matrix block is not full. In this case, the actual number of rows contained in the sub-matrix block needs to be used to replace the row size HBP_M of the two-dimensional partition for calculation.

[0127] In step S3, the sub-matrix blocks after two-dimensional partitioning are used as the basic unit of computation. That is, each sub-matrix block is executed by only one set of thread bundles, and both the parallel computation part and the competitive execution part use the sub-matrix blocks as the basic unit of storage. This makes the sparse matrix-vector multiplication code with sub-matrix blocks as the unit highly reusable.

[0128] In each thread, the thread has a unique starting position on the non-zero element value data. The vector vect corresponding to the non-zero element column number cols is multiplied to complete one calculation, and the result is written to the register. Then, according to the non-zero element index add_sign within the block, the thread jumps to the position above and below data. The above calculation is repeated until the value of add_sign is equal to -1, indicating that the current element is the last element of the row in the sub-matrix block. The result in the register is written to the position before hash transformation in the intermediate result vector mid_output. This position index is obtained by accessing output_hash. Each row in the sub-matrix block uniquely corresponds to a position in output_hash, and this position stores the position of the row before hash transformation.

[0129] After the thread bundle completes the parallel computation portion, the first thread within the bundle queries the number of remaining uncompleted submatrix blocks, Res. This query process is mutually exclusive and implemented using atomic instructions. The number of remaining uncompleted submatrix blocks is an integer variable, and the initial value of Res is equal to the number of submatrix blocks included in the competitive execution portion described in step S2; that is, this value does not include the uncompleted portion of the parallel computation. If the number of remaining uncompleted submatrix blocks is not zero, it indicates that there are still uncomputed submatrix blocks in the current competitive execution portion. The current thread atomically subtracts n from Res to obtain n submatrix blocks from the competitive execution portion, where n is an integer greater than 0, and its maximum value will not exceed the number of submatrix blocks included in the competitive execution portion described in step S2. Sparse matrix-vector multiplication is then performed on the obtained submatrix blocks, with the specific steps being the same as the sparse matrix-vector multiplication calculation in the parallel execution portion described above.

[0130] S4. Combine the calculation results of each submatrix to obtain the final result of matrix-vector multiplication;

[0131] In step S4, the final result is obtained by merging the intermediate result vectors output in step S3. Since the positions where the intermediate result vectors are stored in each row in step S3 are the original positions before the hash transformation, the final result can be obtained simply by accumulating the elements in the same row. Each thread processes the element accumulation task for one row. The number of accumulations required for each row is equal to the number of submatrix blocks (block_col_num) in the column direction of the original sparse matrix. Except for the last row of submatrix blocks, all other rows are accumulated with a step size of HBP_M. The same number of accumulations ensures that each thread has an equal computational load, avoiding thread divergence. Furthermore, the fact that each thread accumulates with the same step size ensures that the address space accessed by each thread is always contiguous, thus allowing for memory access merging to reduce the number of memory accesses and improve computational efficiency.

[0132] Combination Figure 4 This describes the process of converting CSR format to HBP format.

[0133] Figure 4 The format conversion process for a single submatrix block in step S101 is explained. The conversion process for each submatrix block is executed by only one thread, while a single thread may execute the format conversion tasks for multiple submatrix blocks.

[0134] Step S111 is executed only once in each submatrix block. First, the position of the current submatrix block in the entire matrix after the two-dimensional partition is determined, thereby determining the starting position of the zero row index zero_row of the current submatrix block after the transformation and the number of rows contained in the current submatrix block. Except for the submatrix block located in the last row, the number of rows of the other submatrix blocks is HBP_M.

[0135] Step S112 is executed for each row within the block. The CSR format provides the position of the first non-zero element in each row of the current sub-matrix block. First, a binary search is used to obtain the first non-zero element of the current row. Then, the last non-zero element in the row within the sub-matrix block is found sequentially. Based on this, the number of non-zero elements in the row is obtained. The total number of non-zero elements can be obtained by summing the values ​​of each row, thereby determining the position of the last non-zero element in each sub-matrix block. This position is saved as the starting position of the next sub-matrix block in step S103.

[0136] Step S113 is performed on each row in the block. Based on the number of non-zero elements in each row of the block obtained in step S112, a new position can be obtained in the hash table through the transformation of the linear hash function, and the original row number is written at the position.

[0137] Step S114 executes each ω line within a block as a group, determining the corresponding zero-row index `zero_row` based on the number of zero rows preceding the current line in that group. If the current line is a zero row, it is recorded as -1. The zero-row index is used in step S103 to determine the position of the first non-zero element accessed in each line, in conjunction with the storage position `begin_nnz` of the first non-zero element in each block. Subsequently, according to the hash-transformed positions, the non-zero element column number `cols` and the non-zero element value `data` of the new storage format are alternately written to each line. The purpose of alternating writing is to enable the threads within the thread bundle in step S103 to merge memory access instructions when accessing the sparse matrix, thereby improving speed.

[0138] Combination Figure 5 This explains that in step S2, the sparse matrix is ​​divided into a parallel execution part and a competitive execution part.

[0139] Examples of how the sparse matrix is ​​divided into parallel execution and competitive execution parts in this invention are shown below. Figure 5 The division of parallel execution parts follows these principles:

[0140] In the process of partitioning the parallel execution portion, it is assumed that thread bundles are used as the basic unit of computation and sub-matrix blocks are used as the basic unit of storage. That is, at any given time, the data accessed by each thread within a group of thread bundles is located within the same sub-matrix block. Because a group of thread bundles requires the same resources during task allocation and storage / retrieval, it is not necessary to focus on each thread within it during the partitioning process.

[0141] During the process of dividing the parallel execution part, the same number of thread bundles should be allocated to each column of sub-matrix blocks as much as possible. If this is not possible, priority should be given to allocating more thread bundles to the columns located on the left side of the sparse matrix.

[0142] During the process of dividing the parallel execution part, each column of the submatrix is ​​divided equally according to the number of thread bundles on the left side of the sparse matrix, so that each thread bundle is responsible for the calculation of the same number of submatrix blocks.

[0143] After the parallel execution part is completed, the remaining undivided sub-matrix blocks are the competitive execution part. Each thread bundle executes the same number of sub-matrix blocks in the parallel execution part. Because the distribution of non-zero elements among the sub-matrix blocks is uneven, even if each thread bundle executes the same number of sub-matrix blocks, the computation time of each thread bundle may still be different. Therefore, it is necessary to divide the competitive execution part to balance the computation time among the thread bundles. Figure 3 This diagram illustrates all possible divisions into parallel and contention-based execution sections. The blue dashed boxes represent the parallel execution sections, and the red dashed boxes represent the contention-based execution sections. Assume the current device has 8 thread bundles. Figure 5 (a) Each column is assigned an equal number of thread bundles, at which point the remaining competitive execution is concentrated below the original sparse matrix. Figure 5(b) Each column is assigned a different number of thread bundles. In this case, columns on the left side of the sparse matrix are prioritized for allocation of more thread bundles. Therefore, the contention for execution is concentrated in the lower right corner of the original sparse matrix; this is the typical case during the allocation process. Figure 5 (c) Each column is assigned a different number of thread bundles. At this time, the column on the left side of the sparse matrix is ​​assigned 1 thread bundle, and the competitive execution part is concentrated on the right side of the original sparse matrix. This is because the length of the matrix in the row direction is greater than the total number of thread bundles.

[0144] Combination Figure 6 This section explains the scope covered by a single execution unit in each parallel step. On the GPU platform, this method uses a thread as the smallest execution unit. In the diagram, t represents a thread, ω represents a thread bundle, the gray area represents the sub-matrix block after two-dimensional partitioning, and the white solid line represents the data distribution that a single thread needs to compute. Figure 6 (a) In step S101, the hash reordering of each row of the submatrix block depends on the information of each row within the submatrix block. Therefore, each thread processes one submatrix block, achieving parallelism of step S101 between submatrix blocks. Figure 6 (b) In the parallel execution phase of step S103, each thread bundle computes the same number of sub-matrix blocks. Sub-matrix blocks computed within the same thread bundle are located in the same columns. The white solid line represents the data distribution that a single thread needs to compute, not the storage structure. The actual storage structure is that rows are stored alternately, thus reducing the number of memory accesses by each thread within a thread bundle and improving speed. In the contention execution phase of step S103, sub-matrix blocks computed within the same thread bundle may not be located in the same columns. The parallel computation execution process within the same thread bundle is the same as in the parallel execution phase of step S103. Step S103 achieves parallelism between different sub-matrix blocks and parallelism between different rows of the same sub-matrix block. Figure 6 (b) In step S104, a merging operation is performed on the intermediate vector mid_output. After the two-dimensional partition, each sub-matrix block corresponds to a vector fragment of length HBP_M stored in mid_output. This facilitates both the writing in step S103 and the merging operation in step S104. The gray part represents the total sparse matrix. Each thread accumulates a whole row of data as the final result. The two-dimensional partition size HBP_M in the column dimension is an integer multiple of the thread bundle size ω, thus ensuring that the distance from the current calculation position to the next calculation position of each thread within each thread bundle is always HBP_M.

[0145] The following are system embodiments corresponding to the above method embodiments. This embodiment can be implemented in conjunction with the above embodiments. The relevant technical details mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0146] like Figure 8 As shown, this invention also proposes a hash-based sparse matrix-vector multiplication optimization device, which includes:

[0147] The initial module, based on the hardware structure of the neural network hardware accelerator, divides the sparse matrix to be multiplied by the matrix vector to obtain multiple sparse matrix blocks, and performs linear hash mapping on these multiple sparse matrix blocks to obtain the matrix to be divided.

[0148] The partitioning module divides the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure, to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each contain multiple sub-matrix blocks;

[0149] The execution module, a neural network hardware accelerator, performs computational tasks through competition between submatrix blocks and parallel computational tasks within blocks, obtaining multiple submatrix computation results, and restoring the original writing order by looking up a hash table.

[0150] The merging module combines the calculation results of the multiple submatrices according to the original order to obtain the final result of matrix-vector multiplication.

[0151] The hash-based sparse matrix-vector multiplication optimization device, wherein the initial module includes:

[0152] Based on the hardware architecture, a two-dimensional partitioning model for the input sparse matrix is ​​constructed. Linear hash mapping is performed on the sparse matrix blocks to convert the sparse matrix storage format into the HBP format of the matrix to be partitioned. Specifically:

[0153] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the position of the first non-zero element in each row of the currently processed sparse matrix block through binary search, and sequentially traverses until the position of the last element in the corresponding row within the range of the current sparse matrix block.

[0154] The linear hash mapping within a sparse matrix block takes the number of non-zero elements in each row as input. Utilizing the property of linear hash transformation to cluster similar elements, it maps the row numbers within the sparse matrix block to a hash table of the same size as before, ensuring that rows with the same or similar number of non-zero elements are mapped to adjacent positions. The result of the hash transformation is stored in a fixed-length hash table, the length of which is equal to the sum of the number of rows contained in each sparse matrix block after the two-dimensional partitioning, and its size is the total number of rows in the sparse matrix, num_of_row, multiplied by the number of sparse matrix blocks in the column direction, bloc_col_num. The hash table is initially set to -1 to indicate that the current position is empty. The index of the hash table is the actual execution order, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0155]

[0156] In the formula, `output_hash` represents the position of the corresponding row in the output. Calculations are performed sequentially based on this position during matrix-vector multiplication, and its range is greater than or equal to 0 and less than the number of rows in the block, `end_row`. `block_row_nnz` represents the number of non-zero elements in that row within the sparse matrix block, obtained from the data start / end position search module. `HBP_M` and `HBP_N` represent the row and column sizes of the two-dimensional partition, respectively. `ρ` represents the overall sparsity of the input matrix, and `ω` represents the size of a set of thread bundles in the single-instruction multithreaded mode of the neural network accelerator. The value of `HBP_M` must be an integer multiple of `ω`. `i` is the position offset; when a hash mapping collision occurs, `i` is changed to find a nearby free position. `end_row` represents the number of rows contained in the sparse matrix block. The last row of the sparse matrix block may contain fewer rows than `HBP_M`, requiring a judgment. The specific value of `end_row` is shown in the following formula:

[0157]

[0158] In the formula, num_of_row is the total number of rows of the input matrix, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block being processed, and the value of block_m is greater than or equal to 0 and less than block_row_num.

[0159] The HBP storage format includes non-zero element column numbers (cols), non-zero element values ​​(data), non-zero element indexes (add_sign) for accessing non-zero elements within a block, zero row index (zero_row), the location of the first non-zero element in each block (begin_nnz), and a hash table (output_hash). The lengths of the non-zero element column numbers (cols) and non-zero element values ​​(data), as well as the length of the non-zero element index (add_sign), are equal to the number of non-zero elements. The lengths of the zero row index (zero_row) and the hash table (output_hash) are equal to the number of rows multiplied by the number of column blocks. The location of the first non-zero element in each block (begin_nnz) is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and retrieval.

[0160] This execution module includes:

[0161] Competitive execution of computation tasks between submatrix blocks refers to a process where each group of ω threads accesses and processes several blocks of computation tasks within the group. Threads within and between groups execute in parallel. Once a group of threads has completed its parallel execution, it selects a submatrix block from the competitive execution section that has not yet been executed to perform the computation task.

[0162] The competitive execution section uses a ticket number lock to avoid multiple accesses to the same submatrix block. The ticket number lock is an integer variable, which is modified with the volatile keyword during parallel computation to indicate that no compilation optimization is performed on this variable. Each access is read from storage. This variable stores the number of submatrix blocks that are currently not being computed. The initial value is equal to the number of submatrix blocks contained in the competitive execution section. After each group of threads completes the computation of the current task, it atomically accesses this variable to obtain the next computation task. If the return value is negative, it means that there are no unexecuted computation tasks in the competitive execution section. The content that needs to be randomly accessed multiple times during the computation of each two-dimensional submatrix block, as well as the current execution block sequence number obtained by the ticket number lock;

[0163] The index of the hash table is the actual execution order of the threads, and the stored value is the order of each row before the hash transformation. Therefore, after each thread of the neural network accelerator has calculated a row of the submatrix block, it needs to access the corresponding position in the hash table and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0164] The hash-based sparse matrix-vector multiplication optimization device, wherein the partitioning module includes:

[0165] The division into parallel execution and competitive execution parts is calculated based on the size of the sparse matrix and the size of the sparse matrix blocks. The process of dividing the parallel execution part follows the principle that each thread is allocated the same number of block computation tasks, and the blocks to be computed in the same thread should be located in the same column as much as possible. The parts that are not divided into parallel execution parts are classified as competitive execution parts. In the competitive execution part, each sub-matrix block is competitively executed by the thread bundle that finishes its assigned task first during program execution.

[0166] like Figure 9 As shown, in another embodiment of the present invention, a first electronic device A is also proposed, including the aforementioned hash-based sparse matrix-vector multiplication optimization device.

[0167] like Figure 10 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to acquire the sparse matrix to be multiplied by the matrix vector, such as the image feature vector extracted by the convolutional neural network. The information display device D is used to display the final result of the matrix vector multiplication obtained by the present invention, or to further display the image category obtained by the fully connected layer based on the final result.

[0168] The information display device D can process and organize the data output by the first electronic device A based on an information display mechanism to improve the readability of the data. This information display mechanism can be manually preset, for example, visualizing the data output by the first electronic device A. It can present the user with specified key information, such as the final result or image category, based on user-defined display parameters and / or attributes. Display parameters could be, for example, the data range, and display attributes could be, for example, the font, color, or scrolling. This allows the user to understand this information more quickly without having to access secondary pages or scroll through pages, saving user effort. Alternatively, this information display mechanism can be an artificial intelligence (AI) display model, which can learn the user's key information interests based on previous usage habits, such as viewing time, number of clicks, and number of edits, and then automatically present the user with rich and necessary key information.

[0169] The present invention also provides a computer program product, which includes a computer program that can be stored on a readable storage medium. When the computer program is executed by a processor, the computer is able to execute the sparse matrix-vector multiplication optimization method provided by the above methods.

[0170] In another embodiment, the present invention also proposes a storage medium VIII for storing a computer program that performs the sparse matrix-vector multiplication optimization method. It should be understood that the storage medium in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0171] Figure 11 A schematic block diagram of a second electronic device 1000 that can be used to implement embodiments of the present invention is shown. The second electronic device 1000 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein. The second electronic device 1000 may be the same as or different from the first electronic device A.

[0172] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from storage medium VIII into random access memory (RAM) III. The RAM III may also store various programs and data required for the operation of the device 1000. The computing unit I, ROM II, and RAM III are interconnected via bus IV. An input / output (I / O) interface V is also connected to bus IV.

[0173] Multiple components in the second electronic device 1000 are connected to I / O interface V, including: input unit VI, such as a keyboard, mouse, etc.; output unit VII, such as various types of displays, speakers, etc.; storage medium VIII, such as a disk, optical disk, etc.; and communication unit IX, such as a network card, modem, wireless transceiver, etc. Communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0174] The computing unit I can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage medium VIII. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM II and / or communication unit IX. When the computer program is loaded into RAM III and executed by computing unit I, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, computing unit I can be configured to perform methods by any other suitable means (e.g., by means of firmware).

[0175] Although embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and embodiments. They can be applied to various fields suitable for the present invention. For those skilled in the art, other modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and illustrations shown and described herein.

Claims

1. A hash-based optimization method for sparse matrix-vector multiplication, characterized in that, include: The initial step involves dividing the sparse matrix to be multiplied by the matrix vectors into multiple sparse matrix blocks based on the hardware structure of the neural network hardware accelerator. Then, a linear hash mapping is performed on these multiple sparse matrix blocks to obtain the matrix to be divided. The partitioning step involves dividing the matrix to be partitioned according to its size and the hardware structure, resulting in a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein each of the parallel execution part and the competitive execution part contains multiple sub-matrix blocks. In the execution steps, the neural network hardware accelerator performs computation tasks through competition between submatrix blocks and parallel computation tasks within blocks to obtain multiple submatrix computation results, and restores the original writing order by looking up the hash table; The merging step combines the calculation results of the multiple sub-matrices according to the original order to obtain the final result of matrix-vector multiplication; The partitioning steps include: The division into parallel execution and competitive execution parts is calculated based on the size of the sparse matrix and the size of the sparse matrix blocks. The process of dividing the parallel execution part follows the principle that each thread is allocated the same number of block computation tasks, and the blocks to be computed in the same thread should be located in the same column as much as possible. The parts that are not divided into parallel execution parts are classified as competitive execution parts. In the competitive execution part, each sub-matrix block is competitively executed by the thread bundle that finishes its assigned task first during program execution.

2. The hash-based sparse matrix-vector multiplication optimization method as described in claim 1, characterized in that, The initial steps include: Based on the hardware architecture, a two-dimensional partitioning model for the input sparse matrix is ​​constructed. Linear hash mapping is performed on the sparse matrix blocks to convert the sparse matrix storage format into the HBP format of the matrix to be partitioned. Specifically: The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing steps mainly consist of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the position of the first non-zero element in each row of the currently processed sparse matrix block through binary search, and sequentially traverses until the position of the last element in the corresponding row within the range of the current sparse matrix block. The linear hash mapping within a sparse matrix block takes the number of non-zero elements in each row as input. Leveraging the property of linear hash transformation to cluster similar elements, it maps the row numbers within the sparse matrix block to a hash table of the same size as before, ensuring that rows with the same or similar number of non-zero elements are mapped to adjacent positions. The result of the hash transformation is stored in a fixed-length hash table, the length of which is equal to the sum of the number of rows contained in each sparse matrix block after the two-dimensional partitioning, and the size of which is the total number of rows in the sparse matrix. Multiplying the sparse matrix by the number of sparse matrix blocks in the column direction The hash table is initialized with -1 to indicate that the current position is empty; the index of the hash table is the actual execution order, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula: In the formula, The position of the corresponding row in the output is used to determine the order of calculations in matrix-vector multiplication. The range of this position is greater than or equal to 0 and less than the number of rows in the block. , This represents the number of non-zero elements in that row within the sparse matrix block. This value is obtained during the data start and end position search step. and These represent the row and column sizes for the two-dimensional partition. The sparsity of the input matrix as a whole. This represents the size of a set of thread bundles in the single-instruction multithreaded mode of the neural network accelerator. The value must be Integer multiples of, This is the position offset, which changes when a collision occurs in the hash map. To find nearby vacant locations, This represents the number of rows contained in a sparse matrix block. The last row block in a sparse matrix may contain fewer than [number of rows]. An assessment is required. The specific values ​​are shown in the following formula: In the formula, The total number of rows in the input matrix. Let be the number of sparse matrix blocks in the row direction after the input matrix is ​​partitioned in two dimensions. This represents the row number corresponding to the currently processed sparse matrix block. The range of values ​​is greater than or equal to 0 and less than 0. ; HBP storage format includes column numbers of non-zero elements. non-zero element values Accessing non-zero element indices within a block Zero-row index The location where the first non-zero element of each block is stored. and hash tables HBP storage format uses non-zero element column numbers non-zero element values Length and non-zero element index The length of each row is equal to the number of non-zero elements, and the zero-row index is used. and hash table The length is equal to the number of rows multiplied by the number of columns, and the location where the first non-zero element of each block is stored. It equals the total number of sparse matrix blocks; each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and retrieval.

3. The hash-based sparse matrix-vector multiplication optimization method as described in claim 2, characterized in that, The execution steps include: Competition between submatrix blocks for executing computational tasks refers to each Each thread is grouped into a set, and the set of threads accesses the storage and processes a number of blocks of computation tasks. Threads within a set and between sets execute in parallel. After a set of threads has completed the parallel execution portion, it selects a sub-matrix block that has not yet been executed from the competitive execution portion to perform the computation task. The competitive execution section uses a ticket number lock to avoid multiple accesses to the same submatrix block. The ticket number lock is an integer variable, which is modified with the volatile keyword during parallel computation to indicate that no compilation optimization is performed on this variable. Each access is read from storage. This variable stores the number of submatrix blocks that are currently not being computed. The initial value is equal to the number of submatrix blocks contained in the competitive execution section. After each group of threads completes the computation of the current task, it atomically accesses this variable to obtain the next computation task. If the return value is negative, it means that there are no unexecuted computation tasks in the competitive execution section. The content that needs to be randomly accessed multiple times during the computation of each two-dimensional submatrix block, as well as the current execution block sequence number obtained by the ticket number lock; The index of the hash table represents the actual execution order of the threads, while the stored values ​​represent the order of each row before the hash transformation. Therefore, after each thread in the neural network accelerator has computed a row of a submatrix block, it needs to access the corresponding position in the hash table and write the register result into the intermediate result vector. The position before the hash transformation.

4. A hash-based sparse matrix-vector multiplication optimization device, characterized in that, include: The initial module, based on the hardware structure of the neural network hardware accelerator, divides the sparse matrix to be multiplied by the matrix vector to obtain multiple sparse matrix blocks, and performs linear hash mapping on these multiple sparse matrix blocks to obtain the matrix to be divided. The partitioning module divides the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure, to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each contain multiple sub-matrix blocks; The execution module, a neural network hardware accelerator, performs computational tasks through competition between submatrix blocks and parallel computational tasks within blocks, obtaining multiple submatrix computation results, and restoring the original writing order by looking up a hash table. The merging module merges the calculation results of the multiple submatrices according to the original order to obtain the final result of matrix-vector multiplication; This partitioning module includes: The division into parallel execution and competitive execution parts is calculated based on the size of the sparse matrix and the size of the sparse matrix blocks. The process of dividing the parallel execution part follows the principle that each thread is allocated the same number of block computation tasks, and the blocks to be computed in the same thread should be located in the same column as much as possible. The parts that are not divided into parallel execution parts are classified as competitive execution parts. In the competitive execution part, each sub-matrix block is competitively executed by the thread bundle that finishes its assigned task first during program execution.

5. The hash-based sparse matrix-vector multiplication optimization device as described in claim 4, characterized in that, This initial module includes: Based on the hardware architecture, a two-dimensional partitioning model for the input sparse matrix is ​​constructed. Linear hash mapping is performed on the sparse matrix blocks to convert the sparse matrix storage format into the HBP format of the matrix to be partitioned. Specifically: The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the position of the first non-zero element in each row of the currently processed sparse matrix block through binary search, and sequentially traverses until the position of the last element in the corresponding row within the range of the current sparse matrix block. The linear hash mapping within a sparse matrix block takes the number of non-zero elements in each row as input. Leveraging the property of linear hash transformation to cluster similar elements, it maps the row numbers within the sparse matrix block to a hash table of the same size as before, ensuring that rows with the same or similar number of non-zero elements are mapped to adjacent positions. The result of the hash transformation is stored in a fixed-length hash table, the length of which is equal to the sum of the number of rows contained in each sparse matrix block after the two-dimensional partitioning, and the size of which is the total number of rows in the sparse matrix. Multiplying the sparse matrix by the number of sparse matrix blocks in the column direction The hash table is initialized with -1 to indicate that the current position is empty; the index of the hash table is the actual execution order, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula: In the formula, The position of the corresponding row in the output is used to determine the order of calculations in matrix-vector multiplication. The range of this position is greater than or equal to 0 and less than the number of rows in the block. , This represents the number of non-zero elements in that row within the sparse matrix block. This value is obtained from the data start and end position search module. and These represent the row and column sizes for the two-dimensional partition. The sparsity of the input matrix as a whole. This represents the size of a set of thread bundles in the single-instruction multithreaded mode of the neural network accelerator. The value must be Integer multiples of, This is the position offset, which changes when a collision occurs in the hash map. To find nearby vacant locations, This represents the number of rows contained in a sparse matrix block. The last row block in a sparse matrix may contain fewer than [number of rows]. An assessment is required. The specific values ​​are shown in the following formula: In the formula, The total number of rows in the input matrix. Let be the number of sparse matrix blocks in the row direction after the input matrix is ​​partitioned in two dimensions. This represents the row number corresponding to the currently processed sparse matrix block. The range of values ​​is greater than or equal to 0 and less than 0. ; HBP storage format includes column numbers of non-zero elements. non-zero element values Accessing non-zero element indices within a block Zero-row index The location where the first non-zero element of each block is stored. and hash tables HBP storage format uses non-zero element column numbers non-zero element values Length and non-zero element index The length of each row is equal to the number of non-zero elements, and the zero-row index is used. and hash table The length is equal to the number of rows multiplied by the number of columns, and the location where the first non-zero element of each block is stored. This is equal to the total number of sparse matrix blocks; each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and retrieval. This execution module includes: Competition between submatrix blocks for executing computational tasks refers to each Each thread is grouped into a set, and the set of threads accesses the storage and processes a number of blocks of computation tasks. Threads within a set and between sets execute in parallel. After a set of threads has completed the parallel execution portion, it selects a sub-matrix block that has not yet been executed from the competitive execution portion to perform the computation task. The competitive execution section uses a ticket number lock to avoid multiple accesses to the same submatrix block. The ticket number lock is an integer variable, which is modified with the volatile keyword during parallel computation to indicate that no compilation optimization is performed on this variable. Each access is read from storage. This variable stores the number of submatrix blocks that are currently not being computed. The initial value is equal to the number of submatrix blocks contained in the competitive execution section. After each group of threads completes the computation of the current task, it atomically accesses this variable to obtain the next computation task. If the return value is negative, it means that there are no unexecuted computation tasks in the competitive execution section. The content that needs to be randomly accessed multiple times during the computation of each two-dimensional submatrix block, as well as the current execution block sequence number obtained by the ticket number lock; The index of the hash table represents the actual execution order of the threads, while the stored values ​​represent the order of each row before the hash transformation. Therefore, after each thread in the neural network accelerator has computed a row of a submatrix block, it needs to access the corresponding position in the hash table and write the register result into the intermediate result vector. The position before the hash transformation.

6. An electronic device, characterized in that, Includes a hash-based sparse matrix-vector multiplication optimization device as described in claim 4 or 5, wherein the electronic device is connected to an information display device, which is used to display the final result of the matrix-vector multiplication using user-set display parameters, attributes, or through an artificial intelligence model.

7. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the hash-based sparse matrix-vector multiplication optimization method according to any one of claims 1-3.

8. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the hash-based sparse matrix-vector multiplication optimization method as described in any of claims 1-3.

Images