Hash-based sparse matrix vector multiplication optimization method and device

Abstract

The invention provides a Hash-based sparse matrix vector multiplication optimization method, which is characterized by comprising the following steps of: dividing a sparse matrix to be subjected to matrix vector multiplication according to a hardware structure of a neural network hardware accelerator to obtain a plurality of sparse matrix blocks, and performing linear Hash mapping on the plurality of sparse matrix blocks to obtain a matrix to be divided; according to the scale of the to-be-divided matrix and the hardware structure, dividing the to-be-divided matrix to obtain a plurality of sub-matrix blocks, and dividing parallel execution parts and competition execution parts in the sub-matrix blocks; the neural network hardware accelerator competes among the sub-matrix blocks to execute a calculation task, the calculation task is executed in parallel in the blocks to obtain a plurality of sub-matrix calculation results, and the original written sequence is restored by searching the hash table; and merging the plurality of sub-matrix calculation results according to the original sequence to obtain a final result of matrix vector multiplication.

Description

Technical Field

[0001] The present invention relates to the fields of computational data processing and parallel program task allocation technology, and in particular to a hash-based sparse matrix-vector multiplication optimization method, device, electronic device, computer-readable storage medium and computer program product. Background Art

[0002] When calculating sparse matrix-vector multiplication, most of the computing resources are wasted due to the large number of zero elements in the sparse matrix. In addition, due to the large size of the sparse matrix, the corresponding vector also requires a large amount of storage, which makes it difficult to store it completely on the high-bandwidth storage of some edge devices. During the calculation process, low-bandwidth storage needs to be frequently accessed, resulting in a large amount of memory access overhead.

[0003] Prior art 1 proposes the CSR5 storage format based on the assumption that each thread has the same time overhead when processing the same number of non-zero elements, fills the non-zero elements in the sparse matrix row by row into a matrix block of a fixed size, so that each parallel unit processes the same number of non-zero elements and has the same amount of calculation, and introduces the following Figure 1 This method has achieved good results on small-scale sparse matrices, but it does not take into account the large-scale characteristics of sparse matrices in practical applications. When the input is a large-scale sparse matrix, the elements processed by each thread in the same thread bundle at the same time may be located on storage with different bandwidths, which will result in different memory access times and thus destroy the above assumptions, resulting in the actual running speed of each thread in the same thread bundle being limited by the thread accessing the low-bandwidth storage.

[0004] Prior art 2 considers the memory access problem when processing large-scale sparse matrices, and accordingly proposes the Regu2D storage format, which uses a two-dimensional partitioning method to further divide the sparse matrix-vector multiplication into two steps: each sub-matrix block performs matrix-vector multiplication independently, and the generated intermediate vector group is combined, such as Figure 2 As shown in the figure. This method uses dynamic programming to calculate rows with similar numbers of non-zero elements in the same thread bundle, and introduces zero padding to ensure that each thread bundle has the same computing task. This method achieves good results in computing speed, but the time cost of the dynamic programming process is large, and the storage length is not fixed due to the introduction of zero padding, which makes it difficult to accelerate it in parallel during the format conversion process in the preprocessing step.

[0005] First, the above-mentioned prior art 1 ignores the consideration of storage access while ensuring parallel load balancing. Due to the characteristics of the calculation, the threads randomly access the vector, and the high-bandwidth storage is not enough to save the complete vector, which leads to the need for frequent access to the low-bandwidth storage, thus causing a large amount of time overhead for memory access. The present invention uses a two-dimensional partitioning method to ensure the locality of vector access by each sub-matrix block, so it is only necessary to store vector fragments on the high-bandwidth GPU shared storage to reduce the time overhead caused by memory access. Second, the above-mentioned prior art 2 adopts a two-dimensional partitioning method to ensure parallel load balancing within the sub-matrix block, but does not take into account that the loads between each sub-matrix block may not be the same, which is caused by the distribution of non-zero elements of the sparse matrix, and the current two-dimensional partitioning method does not achieve good load balancing between thread bundles. Third, both prior art 1 and prior art 2 have the problem of high preprocessing cost of the method, and it is necessary to design a lightweight and parallel preprocessing method. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of uneven parallel load between blocks and excessive preprocessing cost in the above-mentioned prior art, and proposes a hash-based sparse matrix-vector multiplication optimization method.

[0007] Specifically, in view of the shortcomings of existing technologies, such as Figure 7 As shown, the present invention proposes a hash-based sparse matrix vector multiplication optimization method, which includes:

[0008] In the initial step, according to the hardware structure of the neural network hardware accelerator, the sparse matrix to be subjected to matrix-vector multiplication is divided to obtain a plurality of sparse matrix blocks, and linear hash mapping is performed on the plurality of sparse matrix blocks to obtain the matrix to be divided;

[0009] a partitioning step, partitioning the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure, to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each include a plurality of sub-matrix blocks;

[0010] Execution step: the neural network hardware accelerator performs computing tasks on the sub-matrix blocks competitively, performs computing tasks in parallel within the blocks, obtains multiple sub-matrix computing results, and restores the original writing order by searching the hash table;

[0011] The merging step combines the calculation results of the multiple sub-matrices in order according to the original order to obtain the final result of the matrix-vector multiplication.

[0012] The hash-based sparse matrix-vector multiplication optimization method, wherein the initial step includes:

[0013] According to the hardware structure, a two-dimensional partitioning model of the input sparse matrix is ​​constructed, and linear hash mapping is performed on the sparse matrix blocks, and the sparse matrix storage format is converted into the matrix to be partitioned in HBP format. Specifically:

[0014] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing steps mainly consist of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the first non-zero element position of each row in the sparse matrix block currently being processed through binary search, and sequentially traverses until the last element position of the corresponding row in the range of the current sparse matrix block;

[0015] The linear hash mapping within the sparse matrix block takes the number of non-zero elements in each row as input, and uses the characteristic of linear hash transformation to aggregate similar elements to map the row numbers of each row in the sparse matrix block to a hash table of the same size as before, and enables rows with the same or similar number of non-zero elements to be mapped to adjacent positions; the result of the hash transformation is stored in a fixed-length hash table, whose length is equal to the sum of the number of rows contained in each sparse matrix block after two-dimensional partitioning, and the size is the total number of sparse matrix rows num_of_row multiplied by the number of sparse matrix blocks in the column direction of the sparse matrix block_col_num. The initial value of the hash table is recorded as -1, indicating that the current position is empty; the index of the hash table is the order of actual execution, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0016]

[0017] Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed sequentially according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements in the row of the sparse matrix block. This value is obtained in the data start and end position search step. HBP_M and HBP_N are the row and column sizes of the two-dimensional partition respectively. ρ is the overall sparsity of the input matrix. ω is the size of a group of thread bundles in the single instruction multi-threading mode of the neural network accelerator. The value of HBP_M must be an integer multiple of ω. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the sparse matrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula:

[0018]

[0019] Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num;

[0020] The HBP storage format includes the non-zero element column number cols, the non-zero element value data, the non-zero element index add_sign for accessing the non-zero element within the block, the zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and the hash table output_hash. The length of the HBP storage format on the non-zero element column number cols, the non-zero element value data, and the length of the non-zero element index add_sign are both equal to the number of non-zero elements. The length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks. The location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and reading.

[0021] The hash-based sparse matrix vector multiplication optimization method, wherein the partitioning step comprises:

[0022] The division of the parallel execution part and the competitive execution part is calculated according to the scale of the sparse matrix and the size of the sparse matrix block. The division process of the parallel execution part follows the principle that each thread is assigned the same number of block calculation tasks, and the blocks to be calculated in the same thread should be located in the same column as much as possible; the part that is not divided into parallel execution is classified as the competitive execution part. Each sub-matrix block in the competitive execution part is competitively executed by the thread bundle that completes the assigned task first during the program running process.

[0023] The hash-based sparse matrix vector multiplication optimization method, wherein the execution step includes:

[0024] The computing task of competing among sub-matrix blocks refers to that each thread is a group, and the computing task of accessing storage and processing several blocks is performed in units of groups, and the threads within the group and between the groups are executed in parallel; when a group of threads finishes executing the parallel execution part, the sub-matrix blocks that have not been executed are selected from the competitive execution part to execute the computing task;

[0025] The competitive execution part uses a ticket lock to avoid multiple accesses to the same sub-matrix block; the ticket lock uses an int type variable, which is modified with the volatile keyword during the parallel computing process to indicate that the variable will not be compiled and optimized. Each access is read from the storage. The variable stores the number of sub-matrix blocks that have not been calculated, and the initial value is equal to the number of sub-matrix blocks included in the competitive execution part; each group of threads atomically accesses the variable to obtain the next computing task after calculating the current task. If the return value is a negative number, it means that the competitive execution part has no unexecuted computing tasks; the content of each sub-matrix block after two-dimensional division that needs multiple random accesses during the calculation process, and the current execution block sequence number obtained by the ticket lock;

[0026] The index of the hash table is the actual execution order of the thread, and the stored value is the order of each row before the hash transformation. Therefore, each thread of the neural network accelerator needs to access the corresponding position of the hash table after calculating a row of the sub-matrix block, and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0027] like Figure 8 As shown, the present invention also proposes a hash-based sparse matrix vector multiplication optimization device, which includes:

[0028] The initial module divides the sparse matrix to be multiplied by the matrix-vector according to the hardware structure of the neural network hardware accelerator to obtain a plurality of sparse matrix blocks, and performs linear hash mapping on the plurality of sparse matrix blocks to obtain the matrix to be divided;

[0029] A partitioning module, which partitions the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each include a plurality of sub-matrix blocks; that is, the sub-matrix blocks are partitioned into the parallel execution part and the competitive execution part;

[0030] The execution module, the neural network hardware accelerator performs computing tasks through competition between the sub-matrix blocks, performs computing tasks in parallel within the blocks, obtains multiple sub-matrix calculation results, and restores the original order of writing by searching the hash table;

[0031] The merging module merges the calculation results of the multiple sub-matrices according to the original order to obtain the final result of the matrix-vector multiplication.

[0032] The hash-based sparse matrix vector multiplication optimization device, wherein the initial module includes:

[0033] According to the hardware structure, a two-dimensional partitioning model of the input sparse matrix is ​​constructed, and linear hash mapping is performed on the sparse matrix blocks, and the sparse matrix storage format is converted into the matrix to be partitioned in HBP format. Specifically:

[0034] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the first non-zero element position of each row in the sparse matrix block currently being processed through binary search, and sequentially traverses until the last element position of the corresponding row in the range of the current sparse matrix block;

[0035] The linear hash mapping within the sparse matrix block takes the number of non-zero elements in each row as input, and uses the characteristic of linear hash transformation to aggregate similar elements to map the row numbers of each row in the sparse matrix block to a hash table of the same size as before, and enables rows with the same or similar number of non-zero elements to be mapped to adjacent positions; the result of the hash transformation is stored in a fixed-length hash table, whose length is equal to the sum of the number of rows contained in each sparse matrix block after two-dimensional partitioning, and the size is the total number of sparse matrix rows num_of_row multiplied by the number of sparse matrix blocks in the column direction of the sparse matrix block_col_num. The initial value of the hash table is recorded as -1, indicating that the current position is empty; the index of the hash table is the order of actual execution, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0036]

[0037] Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed in sequence according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements in the row of the sparse matrix block. This value is obtained in the data start and end position search module. HBP_M and HBP_N are the row and column sizes of the two-dimensional division, respectively. ρ is the overall sparsity of the input matrix. ω is the size of a group of thread bundles in the single instruction multi-threading mode of the neural network accelerator. The value of HBP_M must be an integer multiple of ω. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the sparse matrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula:

[0038]

[0039] Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num;

[0040] The HBP storage format includes the non-zero element column number cols, the non-zero element value data, the non-zero element index add_sign for accessing the non-zero element in the block, the zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and the hash table output_hash. The length of the HBP storage format on the non-zero element column number cols, the non-zero element value data, and the length of the non-zero element index add_sign are both equal to the number of non-zero elements. The length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks. The location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning is used as the basic unit for data storage and reading.

[0041] The execution module includes:

[0042] The computing task of competing among sub-matrix blocks refers to that each thread is a group, and the computing task of accessing storage and processing several blocks is performed in units of groups, and the threads within the group and between the groups are executed in parallel; when a group of threads finishes executing the parallel execution part, the sub-matrix blocks that have not been executed are selected from the competitive execution part to execute the computing task;

[0043] The competitive execution part uses a ticket lock to avoid multiple accesses to the same sub-matrix block; the ticket lock uses an int type variable, which is modified with the volatile keyword during the parallel computing process to indicate that the variable will not be compiled and optimized. Each access is read from the storage. The variable stores the number of sub-matrix blocks that have not been calculated, and the initial value is equal to the number of sub-matrix blocks included in the competitive execution part; each group of threads atomically accesses the variable to obtain the next computing task after calculating the current task. If the return value is a negative number, it means that the competitive execution part has no unexecuted computing tasks; the content of each sub-matrix block after two-dimensional division that needs multiple random accesses during the calculation process, and the current execution block sequence number obtained by the ticket lock;

[0044] The index of the hash table is the actual execution order of the thread, and the stored value is the order of each row before the hash transformation. Therefore, each thread of the neural network accelerator needs to access the corresponding position of the hash table after calculating a row of the sub-matrix block, and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0045] The hash-based sparse matrix vector multiplication optimization device, wherein the partitioning module includes:

[0046] The division of the parallel execution part and the competitive execution part is calculated according to the scale of the sparse matrix and the size of the sparse matrix block. The division process of the parallel execution part follows the principle that each thread is assigned the same number of block calculation tasks, and the blocks to be calculated in the same thread should be located in the same column as much as possible; the part that is not divided into parallel execution is classified as the competitive execution part. Each sub-matrix block in the competitive execution part is competitively executed by the thread bundle that completes the assigned task first during the program running process.

[0047] The present invention also proposes an electronic device, which includes the hash-based sparse matrix-vector multiplication optimization device. The electronic device may be connected to an information display device, which is used to display the final result of the matrix-vector multiplication according to display parameters and attributes set by the user or through an artificial intelligence model.

[0048] The present invention also proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the hash-based sparse matrix-vector multiplication optimization method are implemented.

[0049] The present invention also proposes a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the hash-based sparse matrix-vector multiplication optimization method are implemented.

[0050] It can be seen from the above scheme that the advantages of the present invention are:

[0051] While retaining the performance advantage of the two-dimensional partitioning method in sparse matrix-matrix-vector multiplication, the present invention further solves the problem of excessive preprocessing cost of the two-dimensional partitioning sparse matrix method, and adopts the work-stealing method to achieve parallel load balancing between sub-matrix blocks, further improving the speed of matrix-vector multiplication. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the CSR5 format of the prior art;

[0053] Figure 2 A schematic diagram of sparse matrix-vector multiplication of a two-dimensional partitioning method in the prior art;

[0054] Figure 3 A schematic diagram of the flow of the hash-based sparse matrix vector multiplication optimization method of the present invention;

[0055] Figure 4 It is a schematic diagram of the process of converting the CSR format into the HBP format in the present invention;

[0056] Figure 5 It is a schematic diagram of dividing the sparse matrix into a parallel execution part and a competitive execution part in step S2 of the present invention;

[0057] Figure 6 A schematic diagram of the range covered by a single execution unit in each parallel step of the present invention;

[0058] Figure 7 is a flow chart of the method of the present invention;

[0059] Figure 8 It is a module diagram of the device of the present invention;

[0060] Fig. 9 This is a schematic diagram of the structure of a first electronic device of the present invention;

[0061] Fig.10 This is a schematic diagram of the application environment structure of the first electronic device of the present invention;

[0062] Fig.11 It is a schematic diagram of the structure of a second electronic device of the present invention.

[0063] Reference numerals:

[0064] A-First electronic device;

[0065] B-Sparse matrix vector multiplication optimization method device;

[0066] C-data acquisition equipment;

[0067] D-information display device;

[0068] 1000 - second electronic device;

[0069] Ⅰ-computational unit;

[0070] II-ROM;

[0071] III-RAM;

[0072] IV-bus;

[0073] V-interface;

[0074] VI - input unit;

[0075] VII-output unit;

[0076] VIII- Storage medium;

[0077] Ⅸ-Communication unit. DETAILED DESCRIPTION

[0078] It should be noted that, in this application, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0079] Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0080] The processor described in the present invention is the control center of the electronic device, which can be a processor or a general term for multiple processing elements. For example, it can be one or more central processing units (CPUs), or application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs).

[0081] Optionally, the processor can perform various functions of the electronic device by running or executing a software program stored in the memory, and calling data stored in the memory.

[0082] In a specific implementation, as an embodiment, the processor may include one or more CPUs. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions). Electronic devices may include: servers, desktop computers, laptops, smart phones, tablet computers, embedded computers, etc., wherein the embedded computers include vehicles and robots, etc.

[0083] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment, which will not be repeated here.

[0084] It should be noted that the structure of the electronic device shown in the drawings of the present invention does not constitute a limitation thereto, and the actual knowledge structure recognition device may include more or fewer components than shown in the drawings, or a combination of certain components, or a different arrangement of components.

[0085] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0086] It should also be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0087] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0088] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0089] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0092] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0093] When the inventors conducted research on sparse matrix-vector multiplication, they found that the above-mentioned defects in the prior art were caused by insufficient consideration of parallel load balancing and memory access overhead. For example, prior art 1 only considers parallel load balancing, but does not consider that the time overhead of memory access of each thread may be different in the case of large-scale sparse matrices. Prior art 2 uses a two-dimensional partitioning method to further divide the sparse matrix-vector multiplication into two steps: each sub-matrix block performs matrix-vector multiplication independently, and the generated intermediate vector group merges. This is conducive to reducing memory access overhead, and the use of dynamic programming and zero filling reduces the impact of uneven parallel load on calculation speed. However, this method does not take into account the time cost of the preprocessing step. The dynamic programming itself in this method is relatively complex, and the zero filling causes the storage length to be unfixed, which makes it difficult to perform the preprocessing step in a parallel manner, which takes a very large amount of time for large-scale sparse matrices. Two-dimensional partitioning can effectively solve the overhead problem of vector memory access. Therefore, the present invention uses two-dimensional partitioning as the basic method. As for the problem of excessive preprocessing cost, the inventor has found through research that the solution to this defect can be achieved through a linear hashing method. In hash transformation, conflicts often need to be avoided, that is, multiple elements are mapped to the same position after hash transformation, but the characteristic of aggregating similar elements can also be utilized. The present invention designs a hash function, takes the number of non-zero elements in each row as input, aggregates rows with similar numbers of elements, and disperses rows with large differences in number. The time cost of hash transformation is small, so the preprocessing cost can be effectively reduced. However, the prior art 2 only considers the parallel load balancing within the sub-matrix block, but does not consider that the loads between each sub-matrix block may not be the same. For this, the present invention does not adopt the method of balancing the non-zero elements between each matrix block, but divides the parallel execution part and the competitive execution part. The parallel execution part is allocated to each thread for execution in advance, and the competitive execution part is obtained by the thread that first executes the assigned task. The execution, this method realizes the balance of the actual execution time. Specifically, in order to achieve the above technical effects, the present invention proposes the following key technical points:

[0094] Key point 1, the present invention proposes the HBP storage format; the HBP storage format includes six parts: non-zero element column number cols, non-zero element value data, non-zero element index add_sign for accessing the block, zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and hash table outpu_hash. The HBP format can ensure parallel computing between sub-matrix blocks and within sub-matrix blocks in matrix-vector multiplication;

[0095] Key point 2: Lightweight preprocessing steps based on hashing; hash transformation can achieve the aggregation of rows of non-zero elements, thereby avoiding thread divergence, and hash transformation has a low time cost;

[0096] Key point 3, the sub-matrix blocks compete to execute the computing tasks; the present invention divides the parallel execution part and the competitive execution part, the parallel execution part is allocated to each thread for execution in advance, and the competitive execution part is acquired by the thread that has completed the assigned task first, and the method balances the actual execution time;

[0097] In order to make the above features and effects of the present invention more clearly and understandably described, embodiments are given below and described in detail with reference to the accompanying drawings. This specification discloses one or more embodiments that include the features of the present invention. The disclosed embodiments are only for illustration. The scope of protection of the present invention is not limited to the disclosed embodiments, and the present invention is defined by the attached claims.

[0098] Combination Figure 3 The implementation process of the present invention is described. The hash-based sparse matrix vector multiplication optimization method of the present invention comprises the following steps:

[0099] S1. Construct a two-dimensional partition model of the input sparse matrix according to the hardware structure, and perform linear hash mapping on the sub-matrix blocks after the two-dimensional partition; the hardware structure here refers to the maximum shared memory supported by each block of the GPU. The present invention needs to copy the accessed vector fragments to the shared memory to avoid multiple accesses to the global memory with lower bandwidth, thereby improving the speed.

[0100] For step S1, the specific process includes: importing the data to be calculated, calculating the scale of two-dimensional partitioning, traversing statistics within the block, linear hash mapping, and storage format conversion. Specifically, the data to be calculated includes a sparse matrix and a vector stored in CSR format. The default input format for importing the data to be calculated is the CSR compression storage format commonly used for sparse matrices. The method of the present invention can be used to calculate the step of multiplying image features by weights in a neural network to improve the classification efficiency of image feature extraction and image classification. The present invention first reads the sparse matrix to be calculated stored in CSR format, and obtains the length of the matrix in the row and column directions and the number of non-zero elements. Calculate the storage required by the HBP format and initialize it. The storage length of the HBP storage format can be determined only according to the size of the matrix and the number of non-zero elements. Compared with the classic storage format CSR, a small number of arrays are introduced to achieve an improvement in computing speed at the cost of increased storage. The HBP storage format does not introduce additional zero element padding. The storage size occupied by each sub-matrix block after two-dimensional division can be obtained before the storage format conversion step, which provides conditions for parallel execution of format conversion in units of sub-matrix blocks. The size of the sub-matrix block is determined by the hardware structure, that is, the maximum shared memory supported by each block of the GPU. In the SpMV calculation, the vector fragments required for each matrix block are stored in the shared memory and are therefore limited by its size. The HBP storage format includes six parts: non-zero element column number cols, non-zero element value data, non-zero element index add_sign for accessing non-zero elements within a block, zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and hash table outpu_hash. The length of the HBP storage format on the non-zero element column number cols, non-zero element value data, and the length of the non-zero element index add_sign are equal to the number of non-zero elements, the length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks, and the location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sub-matrix blocks. Compared with the classic storage format CSR, the HBP storage format only reorders the cols and data arrays without introducing additional zero padding, discards the ptr array, and uses begin and add_sign to achieve parallel computing between blocks. Each sub-matrix block after two-dimensional division is used as the basic unit for data storage and reading, ensuring that the data required for each sub-matrix block in the sparse matrix-vector multiplication calculation is centrally distributed in the storage location.

[0101] Among them, the non-zero element index add_sign is the position of the current non-zero element to the next non-zero element in the same row. You can think of it as the relative position of the non-zero element in the sparse matrix. The zero row index zero_row records all the rows of the sparse matrix, not just the zero row. For the rows with all zero values, we record it as -1, and for other rows, we record the number of zero rows before it. This is to find the position of the first element of SpMV for each row, so as to perform parallel calculations.

[0102] The present invention obtains the hash table according to the sparse matrix in the CSR compression storage format through the following steps: Step 1. Hash table length and initialization: The purpose of hashing is to reorder the rows in the matrix block to solve the load balancing problem. Therefore, the length of the hash table is equal to the total number of rows after two-dimensional division, and the specific value is equal to the number of rows of the original sparse matrix multiplied by the number of matrix blocks in the column direction. The initial value of all elements in the hash table is -1; Step 2. HBP storage format hash table: In step S1, the generation of the hash table is parallel between matrix blocks. For any matrix block, we first count the number of non-zero elements in each row of the block, and then substitute the number of non-zero elements into the hash function to obtain the position after mapping, and write the row number before hash transformation at this position, so as to obtain the hash table.

[0103] The two-dimensional partition scale calculation includes calculating the number of blocks in the row and column directions of the two-dimensional partition matrix and calculating the sparsity of the sparse matrix. The calculation of the sparsity ρ is shown in the following formula:

[0104]

[0105] Where m is the number of sparse matrix rows, n is the number of sparse matrix columns, and nnz is the number of non-zero elements in the sparse matrix. The above values ​​are usually stored directly in the sparse matrix file and can be directly obtained. The sparsity ρ reflects the density of non-zero elements in the sparse matrix, ranging from 0 to 1. This value can be used to determine the appropriate linear hash mapping to map rows with similar numbers of non-zero elements to locations with similar physical storage.

[0106] The intra-block traversal statistics and subsequent preprocessing steps are executed in parallel, and each two-dimensionally divided sub-matrix block is processed by only one thread. In the intra-block traversal statistics step, each sub-matrix block first obtains the starting position and ending position of the row in the entire sparse matrix by accessing the CSR storage format, then binary searches the starting position of the row in the current sub-matrix block, and then sequentially accesses from the starting position until the column coordinate exceeds the range of the current sub-matrix block or the row coordinate exceeds the range of the current row, and records the number of non-zero elements in the current row at the corresponding position of num_per_row.

[0107] The linear hash mapping step performs a linear hash transformation on each row in the current sub-matrix block, and calculates the position after the hash transformation based on the number of non-zero elements in each row obtained in the previous step. In this step, each sub-matrix block is calculated in parallel, and each row in the sub-matrix block is calculated serially. The corresponding position of output_hash is accessed based on the obtained result. The serial number in output_hash implies the actual execution order of the thread, and the stored value is the position of each row before the hash transformation. The specific hash transformation used is shown in the following formula:

[0108]

[0109] Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed sequentially according to this position, and its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements of this row in the submatrix block. This value is obtained in the data start and end position search step. HBP_M and HBP_N are the row and column sizes of the two-dimensional partition, respectively. The value is related to the hardware platform on which it runs. ρ is the sparsity of the entire input matrix, which ranges from 0 to 1. ω is the size of a group of warps in the single instruction multiple thread (SIMT) mode. Taking Nvidia GeForce MX250 GPU as an example, the thread bundle ω size is 32, and the maximum shared storage supported by each block is 48KB. Assuming that each block contains only one thread bundle, each thread bundle occupies 48KB of shared storage. The reusable part in matrix-vector multiplication is the vector fragment, which is stored in double type and each element occupies 8B. Therefore, each thread bundle can store vector fragments with a maximum length of 6K in the shared storage on average. Considering that the shared storage also needs to store the sub-matrix block number to be executed obtained from the ticket number lock in the subsequent step S3, each thread bundle is allowed to store a vector fragment with a length of 4096, that is, the two-dimensional partition size HBP_N in the column direction is 4096. If the value of HBP_M is too small, the number of rows contained in the submatrix block will be limited. Rows with a large difference in the number of non-zero elements are more likely to be divided into the same thread bundle for execution, which will cause thread divergence in step S3 and affect the calculation speed. If the value is too large, the amount of calculation for each thread will increase, and more time will be spent in the process of searching the hash table, thus affecting the speed of step S1. At the same time, according to the calculation characteristics of step S4, the value of HBP_M needs to be an integer multiple of ω. Combining the above conditions, the value of HBP_M is 512. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the submatrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula:

[0110]

[0111] Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sub-matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sub-matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num.

[0112] The storage format conversion step is designed based on the memory access characteristics of the thread warp on the GPU during the parallel execution of the sparse matrix-vector multiplication calculation. During the sparse matrix-vector multiplication calculation, each non-zero element in the sparse matrix will only be accessed once, and there is no temporal locality, so it is not suitable to use the shared memory with large bandwidth on the GPU for storage. When accessing the video memory on the GPU, a group of thread warps access continuous storage addresses at the same time, and the memory access merging can be used to reduce the number of memory accesses and improve the speed of sparse matrix-vector multiplication calculation. The storage format conversion step uses the hash table obtained in the linear hash mapping step as the access index, and each time takes a row equal to the thread warp size ω for storage format conversion. The number of non-zero rows j is calculated by the number of non-zero elements in each row in the block obtained by the block traversal statistics step. For the zero row index zero_row, if all the elements in the row are zero, it is recorded as -1. If there are non-zero elements, the number of zero rows before the row is recorded, and its value range is -1 to ω. If there is at least one non-zero row in the current ω rows, then for each non-zero row, select the first non-zero element that has not been converted to the format, write the column number cols and the non-zero element value data stored in the original CSR format to the new position, and then record the number of non-empty rows currently remaining in the non-zero element index add_sign. This value represents the offset from the current element to the next element in the same row. Then num_per_row minus 1 indicates that the element has completed the format conversion. If the value corresponding to the row in num_per_row is zero at this time, it means that the current element is the last non-zero element of the row in the sub-matrix block. Modify add_sign to -1 to indicate that the row is terminated and reduce the number of non-zero rows j by 1. When j is less than 0, it means that the current ω rows have completed the format conversion.

[0113] S2. Divide the parallel execution part and the competitive execution part according to the size of the input sparse matrix and the hardware structure of the operating platform; it should be noted that the hardware indicators concerned here are different from those in step S1. Step S1 focuses on the size of high-bandwidth storage, such as the shared memory size of each block on the GPU. In step S2 here, the maximum number of blocks and the warp size are concerned, which are computing capabilities. For example, these two numbers can be read directly on Nivida GPUs, and the values ​​of these two values ​​may not be the same on Nivida GPUs with different computing capabilities.

[0114] For step S2, the specific process includes: obtaining the required operating platform parameters, dividing the parallel execution part, and dividing the competitive execution part.

[0115] The maximum number of parallel threads P and the thread bundle ω size parameters supported by the hardware platform required in step S2 may take different values ​​for different hardware platforms. Therefore, the above parameters need to be obtained before executing the parallel calculation of sparse matrix vector multiplication. The parameter P is equal to the maximum number of blocks multiplied by the thread bundle ω size. Step S2 divides the parallel execution part and the competitive execution part only to divide the number of sub-matrix blocks, which implies the column-first index order and does not involve the reading of elements in the matrix and other contents that need to be calculated in parallel. Therefore, it can be executed simultaneously with step S1 on a serial calculation platform. The hardware platform refers to a platform with parallel computing capabilities, such as a GPU hardware platform, which is the same as step S1. It should be noted that step S2 only requires some parameters of the GPU hardware platform, rather than calculating on the GPU platform. Step S2 only divides the number of matrix blocks contained in each part. The computing resources required for this part are very few, so it can be run on the CPU platform and run simultaneously with step S1, and the time required for this part is very short, which can be hidden in the S1 time.

[0116] The step of dividing the parallel execution part allocates an equal number of sub-matrix blocks to each thread bundle. Under this premise, sub-matrix blocks located on the same column are allocated to a thread bundle as much as possible. The vector range required to be accessed by the sub-matrix blocks on the same column during the matrix-vector multiplication process is the same. During the calculation process, the program will randomly access the vector fragments in this range multiple times, so the required vector fragments are stored in a shared storage with a larger bandwidth to improve the memory access speed. The step of dividing the parallel execution part first calculates the number of thread bundles that can be obtained by the sub-matrix blocks in each column after the two-dimensional division, and then obtains the number of thread blocks for parallel execution allocated to each thread. Among them, the calculation of the number of thread bundles that can be obtained by the sub-matrix blocks in each column after the two-dimensional division is shown in the following formula:

[0117]

[0118] In the formula, ceil() means rounding up, and the number of thread warps obtained by each column of the sub-matrix block is an integer greater than 0. After completing the steps of dividing the parallel execution part, the remaining undivided part is the competitive execution part. According to the different divisions of the parallel execution part, the division results of the competitive execution part can be divided into three cases:

[0119] The first type: the ratio of the maximum number of parallel threads P to the thread bundle size ω is exactly an integer multiple of the number of column-wise submatrix blocks block_col_num. In this case, each column submatrix block obtains the same number of thread blocks, the parallel execution part is located above the sparse matrix, and the competitive execution part is located below the sparse matrix.

[0120] The second type: the ratio of the maximum number of parallel threads P to the thread warp size ω is less than the number of column-wise submatrix blocks block_col_num. In this case, the submatrix blocks on the right are not assigned to the thread warp, the parallel execution part is on the left side of the sparse matrix, and the competitive execution part is on the right side of the sparse matrix.

[0121] The third type: The ratio of the maximum number of parallel threads P to the thread bundle size ω is greater than the number of column-wise submatrix blocks block_col_num and is not an integer multiple. In this case, each column submatrix block is assigned to a thread bundle, but the number of assigned thread bundles is unequal. The parallel execution part is located in the upper left part of the sparse matrix, while the competitive execution part is located in the lower right part of the sparse matrix.

[0122] After calculation, the above three situations can obtain the number of sub-matrix blocks included in the competitive execution part. This value and the number of parallel execution thread blocks allocated to each thread are passed into step S3 as parameters.

[0123] S3. Competitively perform computing tasks on the sub-matrix blocks after two-dimensional division, and perform computing tasks in parallel within the blocks, and restore the original order of writing by searching the hash table before the merging step;

[0124] For step S3, the specific process includes: reading the required data, performing matrix-vector multiplication calculation on the parallel execution part, and performing matrix-vector multiplication calculation on the competitive execution part. Step S3 is the core step of sparse matrix-vector multiplication calculation, which is performed on the device end with parallel computing capability represented by GPU.

[0125] In the required data reading step, each thread first obtains its own thread number and the serial number in the current thread bundle, and the pre-divided parallel computing part to be calculated can be obtained based on this. The pre-divided parallel computing part corresponding to each thread bundle is a number of sub-matrix blocks located in the same column. The sub-matrix blocks in the same column have the same vector fragments accessed in the execution of matrix-vector multiplication, that is, the fragment will be repeatedly accessed in the execution of matrix-vector multiplication in the sub-matrix blocks in the same column, which has time locality, so it can be stored in a storage with high bandwidth to improve the memory access speed. In the present invention, these vector fragments are stored in the shared storage of each thread bundle. A group of thread bundles obtain whether there is a non-zero block in the sub-matrix block to be executed. If there is, the corresponding vector fragment is copied and stored in the shared storage. When the pre-divided parallel computing part is subsequently executed, it only needs to be read from the shared storage, which effectively reduces the memory access time for calculations such as matrix-vector multiplication that require multiple random accesses.

[0126] In the step of executing matrix-vector multiplication in the parallel execution part, the number of parallel execution thread blocks allocated to each thread obtained in step S2 combined with the number of thread bundles can be used to determine the position of the executed sub-matrix block. When the sub-matrix block is located at the bottom of the original sparse matrix, the sub-matrix block may not be full. At this time, the actual number of rows contained in the sub-matrix block needs to be used instead of the row size HBP_M of the two-dimensional division for calculation.

[0127] In step S3, the sub-matrix blocks after two-dimensional division are used as the basic units of calculation, that is, each sub-matrix block is executed by only one set of thread warps, and both the parallel computing part and the competitive execution part use the sub-matrix blocks as the basic units of storage, which makes the sparse matrix-vector multiplication code based on sub-matrix blocks highly reusable.

[0128] In each thread, the thread has a unique starting position on the non-zero element value data. The data and the vector vect at the corresponding position of the non-zero element column number cols are multiplied to complete a calculation, and the result is written to the register. Then, according to the non-zero element index add_sign accessed within the block, it jumps to the next position above data. The above calculation is repeated until the value of add_sign is equal to -1, indicating that the current element is the last element of the row in the sub-matrix block. The result in the register is written to the position before the hash transformation in the intermediate result vector mid_output. The position index is obtained by accessing output_hash. Each row in the sub-matrix block uniquely corresponds to a position in output_hash, which stores the position of the row before the hash transformation.

[0129] After the thread bundle completes the parallel computing part, the first thread in the thread bundle queries the number of sub-matrix blocks Res that are currently remaining and not completed. The query process is mutually exclusive and is implemented using atomic instructions. The number of sub-matrix blocks that are remaining and not completed is an integer variable, and the initial value of Res is equal to the number of sub-matrix blocks included in the competitive execution part described in step S2, that is, this value does not include the unfinished part in the parallel computing. If the number of sub-matrix blocks that are remaining and not completed is not zero, it means that there are still uncalculated sub-matrix blocks in the current competitive execution part. The current thread atomically subtracts n from Res to indicate that n sub-matrix blocks are obtained from the competitive execution part, where the range of n is an integer greater than 0, and its maximum value will not be greater than the number of sub-matrix blocks included in the competitive execution part described in step S2. Sparse matrix vector multiplication calculation is performed on the obtained sub-matrix blocks, and the specific steps are the same as the sparse matrix vector multiplication calculation of the above-mentioned parallel execution part.

[0130] S4. Combine the calculation results of each sub-matrix to obtain the final result of matrix-vector multiplication;

[0131] In step S4, the final result is obtained by merging the intermediate result vectors output in step S3. Because the position of each row storing the intermediate result vector in step S3 is the original position before the hash transformation, the final result can be obtained by simply accumulating the elements in the same row. Each thread processes the element accumulation task of one row, and the number of accumulations required for each row is equal to the number of sub-matrix blocks block_col_num in the column direction of the original sparse matrix. Except for the last row of sub-matrix blocks, the remaining rows are accumulated with a step size of HBP_M. The same number of accumulations ensures that each thread has the same amount of computing load, avoiding thread divergence, and each thread accumulates with the same step size so that the address space accessed by the thread bundle is always continuous, so that the memory access merge can be used to reduce the number of memory accesses and improve computing efficiency.

[0132] Combination Figure 4 Describes the process of converting from CSR format to HBP format.

[0133] Figure 4 The format conversion process of a single sub-matrix block in step S101 is described. The conversion process of each sub-matrix block is executed by only one thread, and a single thread may execute format conversion tasks of multiple sub-matrix blocks.

[0134] Step S111 is executed only once in each sub-matrix block. First, the position of the current sub-matrix block in the entire matrix after the two-dimensional division is determined, thereby determining the starting position of the zero row index zero_row of the current sub-matrix block after the conversion and the number of rows contained in the current sub-matrix block. Except for the sub-matrix block located in the last row, the number of rows of the remaining sub-matrix blocks is HBP_M.

[0135] Step S112 is executed for each row in the block. The CSR format provides the position of the first non-zero element of each row in the current sub-matrix block. First, a binary search is used to obtain the first non-zero element of the current row, and then a sequential traversal is performed to find the last non-zero element of the row in the sub-matrix block. Based on this, the number of non-zero elements in the row is obtained. The total number of non-zero elements can be obtained by accumulating each row to determine the position of the last non-zero element of each sub-matrix block, and the position is saved as the starting position of the next sub-matrix block in step S103.

[0136] Step S113 is executed for each row in the block. According to the number of non-zero elements in each row in the block obtained in step S112, a new position can be finally obtained in the hash table through linear hash function conversion, and the original row number is written in this position.

[0137] Step S114 is executed for each row in the block as a group, and the corresponding zero row index zero_row is determined according to the number of zero rows before the current row in the group. If the current row is a zero row, it is recorded as -1. The zero row index is used to determine the first non-zero element position accessed by each row in combination with the position begin_nnz where the first non-zero element of each block is stored in step S103. Then, the non-zero element column number cols and the non-zero element value data in the new storage format are alternately written to each row according to the position after hash transformation. The purpose of alternating writing is to enable each thread in the thread bundle in step S103 to merge the memory access instructions to increase the speed when accessing the sparse matrix.

[0138] Combination Figure 5 The step S2 is described as follows: dividing the sparse matrix into a parallel execution part and a competitive execution part.

[0139] The embodiment of the present invention divides the sparse matrix into parallel execution parts and competitive execution parts. Figure 5 , the division of parallel execution parts follows the following principles:

[0140] In the process of dividing the parallel execution part, it is assumed that the thread warp is used as the basic computing unit and the sub-matrix block is used as the basic storage unit, that is, the data accessed by each thread in a group of thread warps at the same time is located in the same sub-matrix block. Because a group of thread warps requires the same resources during task allocation and storage reading, it is not necessary to pay attention to each thread within it during the division process.

[0141] In the process of dividing the parallel execution part, the same number of thread warps is allocated to each column of the sub-matrix block as much as possible. If it is not satisfied, more thread warps are preferentially allocated to the columns located on the left side of the sparse matrix.

[0142] In the process of dividing the parallel execution part, each column of sub-matrix blocks is equally divided according to the number of column thread warps on the left side of the sparse matrix, so that each thread warp is responsible for the calculation of the same number of sub-matrix blocks.

[0143] After completing the parallel execution part, the remaining undivided sub-matrix blocks are the competitive execution part. Each thread warp executes the same number of sub-matrix blocks in the parallel execution part. Because the non-zero elements are unevenly distributed among the sub-matrix blocks, even if each thread warp executes the same number of sub-matrix blocks, the calculation time of each thread warp may still be different. Therefore, it is necessary to divide the competitive execution part to balance the calculation time between thread warps. Figure 3 It shows all possible situations of dividing the parallel execution part and the competitive execution part. The blue dotted box in the figure is the parallel execution part, and the red dotted box is the competitive execution part. Assuming that the current device has 8 thread warps, Figure 5 (a), each column is assigned an equal number of thread warps, and the remaining contention execution parts are concentrated below the original sparse matrix. Figure 5(b) Each column is assigned a different number of thread warps. In this case, more thread warps are assigned to the columns on the left side of the sparse matrix first. Therefore, the contention execution part is concentrated in the lower right corner of the original sparse matrix. This is the general situation in the allocation process. Figure 5 (c) Each column is assigned a different number of thread warps. At this time, the column on the left side of the sparse matrix is ​​assigned 1 thread warp, and the competitive execution part is concentrated on the right side of the original sparse matrix. This is because the length of the matrix in the row direction is greater than the total number of thread warps.

[0144] Combination Figure 6 This figure shows the range covered by a single execution unit in each parallel step. On the GPU platform, this method uses a thread as the smallest execution unit. In the figure, t represents a thread, ω represents a thread bundle, the gray part represents the sub-matrix block after two-dimensional partitioning, and the white solid line represents the data distribution that a single thread needs to calculate. Figure 6 (a) In step S101, the hash reordering of each row of the sub-matrix block depends on the information of each row in the sub-matrix block. Therefore, each thread is allowed to process one sub-matrix block to achieve parallel processing of step S101 between sub-matrix blocks. Figure 6 (b) In the parallel execution phase of step S103, each thread bundle calculates the same number of sub-matrix blocks, and the sub-matrix blocks calculated by the same thread bundle are located on the same column. The white solid line represents the data distribution that a single thread needs to calculate rather than the storage structure. The actual storage structure is that the rows are stored alternately, so that each thread in the thread bundle can increase the speed by reducing the number of memory accesses when accessing storage. In the competitive execution phase of step S103, the sub-matrix blocks calculated by the same thread bundle may not be located on the same column. The parallel calculation execution process within the same thread bundle is the same as the parallel execution phase of step S103. Step S103 achieves parallelism between different sub-matrix blocks, and parallelism between different rows of the same sub-matrix block. Figure 6 (b), in step S104, the merge operation of the intermediate vector mid_output is performed. After the two-dimensional partitioning, each sub-matrix block corresponds to a vector segment of length HBP_M in the mid_output storage, which is convenient for both the writing of step S103 and the merging operation of step S104. The gray part represents the total sparse matrix. Each thread accumulates a whole row of data as the final result. The two-dimensional partition size HBP_M on the column dimension is an integer multiple of the thread warp size ω, so as to ensure that the distance from the current calculation position to the next calculation position of each thread in each thread warp is always HBP_M.

[0145] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment. In order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in this embodiment can also be applied in the above embodiment.

[0146] like Figure 8 As shown, the present invention also proposes a hash-based sparse matrix vector multiplication optimization device, which includes:

[0147] The initial module divides the sparse matrix to be multiplied by the matrix-vector according to the hardware structure of the neural network hardware accelerator to obtain a plurality of sparse matrix blocks, and performs linear hash mapping on the plurality of sparse matrix blocks to obtain the matrix to be divided;

[0148] A partitioning module, which partitions the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each include a plurality of sub-matrix blocks;

[0149] The execution module, the neural network hardware accelerator performs computing tasks through competition between the sub-matrix blocks, performs computing tasks in parallel within the blocks, obtains multiple sub-matrix calculation results, and restores the original order of writing by searching the hash table;

[0150] The merging module merges the calculation results of the multiple sub-matrices according to the original order to obtain the final result of the matrix-vector multiplication.

[0151] The hash-based sparse matrix vector multiplication optimization device, wherein the initial module includes:

[0152] According to the hardware structure, a two-dimensional partitioning model of the input sparse matrix is ​​constructed, and linear hash mapping is performed on the sparse matrix blocks, and the sparse matrix storage format is converted into the matrix to be partitioned in HBP format. Specifically:

[0153] The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the first non-zero element position of each row in the sparse matrix block currently being processed through binary search, and sequentially traverses until the last element position of the corresponding row in the range of the current sparse matrix block;

[0154] The linear hash mapping within the sparse matrix block takes the number of non-zero elements in each row as input, and uses the characteristic of linear hash transformation to gather similar elements to map the row numbers of each row in the sparse matrix block to a hash table of the same size as before, and enables rows with the same or similar number of non-zero elements to be mapped to adjacent positions; the result of the hash transformation is stored in a fixed-length hash table, whose length is equal to the sum of the number of rows contained in each sparse matrix block after two-dimensional partitioning, and the size is the total number of sparse matrix rows num_of_row multiplied by the number of sparse matrix blocks in the column direction bloc_col_num of the sparse matrix. The initial value of the hash table is recorded as -1, indicating that the current position is empty; the index of the hash table is the order of actual execution, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula:

[0155]

[0156] Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed in sequence according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements in the row of the sparse matrix block. This value is obtained in the data start and end position search module. HBP_M and HBP_N are the row and column sizes of the two-dimensional division, respectively. ρ is the overall sparsity of the input matrix. ω is the size of a group of thread bundles in the single instruction multi-threading mode of the neural network accelerator. The value of HBP_M must be an integer multiple of ω. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the sparse matrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula:

[0157]

[0158] Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num;

[0159] The HBP storage format includes the non-zero element column number cols, the non-zero element value data, the non-zero element index add_sign for accessing the non-zero element in the block, the zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and the hash table output_hash. The length of the HBP storage format on the non-zero element column number cols, the non-zero element value data, and the length of the non-zero element index add_sign are both equal to the number of non-zero elements. The length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks. The location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning is used as the basic unit for data storage and reading.

[0160] The execution module includes:

[0161] The computing task of competing among sub-matrix blocks refers to that each thread is a group, and the computing task of accessing storage and processing several blocks is performed in units of groups, and the threads within the group and between the groups are executed in parallel; when a group of threads finishes executing the parallel execution part, the sub-matrix blocks that have not been executed are selected from the competitive execution part to execute the computing task;

[0162] The competitive execution part uses a ticket lock to avoid multiple accesses to the same sub-matrix block; the ticket lock uses an int type variable, which is modified with the volatile keyword during the parallel computing process to indicate that the variable will not be compiled and optimized. Each access is read from the storage. The variable stores the number of sub-matrix blocks that have not been calculated, and the initial value is equal to the number of sub-matrix blocks included in the competitive execution part; each group of threads atomically accesses the variable to obtain the next computing task after calculating the current task. If the return value is a negative number, it means that the competitive execution part has no unexecuted computing tasks; the content of each sub-matrix block after two-dimensional division that needs multiple random accesses during the calculation process, and the current execution block sequence number obtained by the ticket lock;

[0163] The index of the hash table is the actual execution order of the thread, and the stored value is the order of each row before the hash transformation. Therefore, each thread of the neural network accelerator needs to access the corresponding position of the hash table after calculating a row of the sub-matrix block, and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

[0164] The hash-based sparse matrix vector multiplication optimization device, wherein the partitioning module includes:

[0165] The division of the parallel execution part and the competitive execution part is calculated according to the scale of the sparse matrix and the size of the sparse matrix block. The division process of the parallel execution part follows the principle that each thread is assigned the same number of block calculation tasks, and the blocks to be calculated in the same thread should be located in the same column as much as possible; the part that is not divided into parallel execution is classified as the competitive execution part. Each sub-matrix block in the competitive execution part is competitively executed by the thread bundle that completes the assigned task first during the program running process.

[0166] like Fig. 9 As shown, the present invention further proposes a first electronic device A in another embodiment, including the hash-based sparse matrix-vector multiplication optimization device.

[0167] like Fig.10 As shown, the first electronic device A can also be connected to the data acquisition device C and the information display device D through a wired or wireless information transmission scheme. The data acquisition device C is used to acquire the sparse matrix to be matrix-vector multiplication, such as the image feature vector extracted by the convolutional neural network. The information display device D is used to display the final result of the matrix-vector multiplication analyzed by the present invention, or further display the image category obtained by the fully connected layer based on the analysis of the final result.

[0168] The information display device D can sort and process the data output by the first electronic device A based on the information display mechanism to improve the readability of the data output by the first electronic device A. The information display mechanism can be manually preset, for example, the data output by the first electronic device A is visually displayed, which can be based on the display parameters and / or attributes set by the user. The display parameters can be, for example, the display data range, and the display attributes can be, for example, the display font, color, whether to scroll and play, etc. The user is presented with the key information specified by the user, such as the final result, image category, etc. The user can understand this information more promptly without having to access the secondary page or scroll the page, saving the user's operation. Or the information display mechanism can be an artificial intelligence AI display model, which can learn the user's key information based on the user's previous usage habits, such as viewing time, number of clicks, number of edits, etc., and then automatically present rich and necessary key information to the user.

[0169] The present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a readable storage medium. When the computer program is executed by a processor, the computer can execute the sparse matrix-vector multiplication optimization method provided by the above methods.

[0170] The present invention also proposes a storage medium VIII in another embodiment for storing a computer program for executing the sparse matrix-vector multiplication optimization method. It should be understood that the storage medium in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0171] Fig.11 A schematic block diagram of a second electronic device 1000 that can be used to implement an embodiment of the present invention is shown. The second electronic device 1000 electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The second electronic device 1000 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present invention described and / or required herein. The second electronic device 1000 may be the same or different from the first electronic device A.

[0172] The second electronic device 1000 includes a computing unit I, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory II (ROM) or a computer program loaded from a storage medium VIII into a random access memory (RAM) III. In RAM III, various programs and data required for the operation of the device 1000 can also be stored. The computing unit I, ROM II, and RAM III are connected to each other via a bus IV. An input / output (I / O) interface V is also connected to the bus IV.

[0173] A plurality of components in the second electronic device 1000 are connected to the I / O interface V, including: an input unit VI, such as a keyboard, a mouse, etc.; an output unit VII, such as various types of displays, speakers, etc.; a storage medium VIII, such as a disk, an optical disk, etc.; and a communication unit IX, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit IX allows the second electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0174] The computing unit I may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of computing unit I include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit I performs the various methods and processes described above, such as method steps S1-S4. For example, in some embodiments, the method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage medium VIII. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via ROM II and / or a communication unit IX. When the computer program is loaded into RAM III and executed by the computing unit I, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit I may be configured to execute the method in any other appropriate manner (e.g., by means of firmware).

[0175] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the implementation modes, and they can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A hash-based sparse matrix-vector multiplication optimization method, characterized in that: include: In the initial step, according to the hardware structure of the neural network hardware accelerator, the sparse matrix to be subjected to matrix-vector multiplication is divided to obtain a plurality of sparse matrix blocks, and linear hash mapping is performed on the plurality of sparse matrix blocks to obtain the matrix to be divided; a partitioning step, partitioning the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure, to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each include a plurality of sub-matrix blocks; Execution step: the neural network hardware accelerator performs computing tasks on the sub-matrix blocks competitively, performs computing tasks in parallel within the blocks, obtains multiple sub-matrix computing results, and restores the original writing order by searching the hash table; The merging step combines the calculation results of the multiple sub-matrices in order according to the original order to obtain the final result of the matrix-vector multiplication.

2. The hash-based sparse matrix vector multiplication optimization method according to claim 1, characterized in that: This initial step includes: According to the hardware structure, a two-dimensional partitioning model of the input sparse matrix is ​​constructed, and linear hash mapping is performed on the sparse matrix blocks, and the sparse matrix storage format is converted into the matrix to be partitioned in HBP format. Specifically: The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing steps mainly consist of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the first non-zero element position of each row in the sparse matrix block currently being processed through binary search, and sequentially traverses until the last element position of the corresponding row in the range of the current sparse matrix block; The linear hash mapping within the sparse matrix block takes the number of non-zero elements in each row as input, and uses the characteristic of linear hash transformation to aggregate similar elements to map the row numbers of each row in the sparse matrix block to a hash table of the same size as before, and enables rows with the same or similar number of non-zero elements to be mapped to adjacent positions; the result of the hash transformation is stored in a fixed-length hash table, whose length is equal to the sum of the number of rows contained in each sparse matrix block after two-dimensional partitioning, and the size is the total number of sparse matrix rows num_of_row multiplied by the number of sparse matrix blocks in the column direction of the sparse matrix block_col_num. The initial value of the hash table is recorded as -1, indicating that the current position is empty; the index of the hash table is the order of actual execution, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula: Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed sequentially according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements in the row of the sparse matrix block. This value is obtained in the data start and end position search step. HBP_M and HBP_N are the row and column sizes of the two-dimensional partition respectively. ρ is the overall sparsity of the input matrix. ω is the size of a group of thread bundles in the single instruction multi-threading mode of the neural network accelerator. The value of HBP_M must be an integer multiple of ω. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the sparse matrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula: Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num; The HBP storage format includes the non-zero element column number cols, the non-zero element value data, the non-zero element index add_sign for accessing the non-zero element within the block, the zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and the hash table output_hash. The length of the HBP storage format on the non-zero element column number cols, the non-zero element value data, and the length of the non-zero element index add_sign are both equal to the number of non-zero elements. The length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks. The location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning serves as the basic unit for data storage and reading.

3. The hash-based sparse matrix vector multiplication optimization method according to claim 1, characterized in that: The division step includes: The division of the parallel execution part and the competitive execution part is calculated according to the scale of the sparse matrix and the size of the sparse matrix block. The division process of the parallel execution part follows the principle that each thread is assigned the same number of block calculation tasks, and the blocks to be calculated in the same thread should be located in the same column as much as possible; the part that is not divided into parallel execution is classified as the competitive execution part. Each sub-matrix block in the competitive execution part is competitively executed by the thread bundle that completes the assigned task first during the program running process.

4. The hash-based sparse matrix vector multiplication optimization method according to claim 2, characterized in that: The execution steps include: The computing task of competing among sub-matrix blocks refers to that each thread is a group, and the computing task of accessing storage and processing several blocks is performed in units of groups, and the threads within the group and between the groups are executed in parallel; when a group of threads finishes executing the parallel execution part, the sub-matrix blocks that have not been executed are selected from the competitive execution part to execute the computing task; The competitive execution part uses a ticket lock to avoid multiple accesses to the same sub-matrix block; the ticket lock uses an int type variable, which is modified with the volatile keyword during the parallel computing process to indicate that the variable will not be compiled and optimized. Each access is read from the storage. The variable stores the number of sub-matrix blocks that have not been calculated, and the initial value is equal to the number of sub-matrix blocks included in the competitive execution part; each group of threads atomically accesses the variable to obtain the next computing task after calculating the current task. If the return value is a negative number, it means that the competitive execution part has no unexecuted computing tasks; the content of each sub-matrix block after two-dimensional division that needs multiple random accesses during the calculation process, and the current execution block sequence number obtained by the ticket lock; The index of the hash table is the actual execution order of the thread, and the stored value is the order of each row before the hash transformation. Therefore, each thread of the neural network accelerator needs to access the corresponding position of the hash table after calculating a row of the sub-matrix block, and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

5. A hash-based sparse matrix vector multiplication optimization device, characterized in that: include: The initial module divides the sparse matrix to be multiplied by the matrix-vector according to the hardware structure of the neural network hardware accelerator to obtain a plurality of sparse matrix blocks, and performs linear hash mapping on the plurality of sparse matrix blocks to obtain the matrix to be divided; A partitioning module, which partitions the matrix to be partitioned according to the size of the matrix to be partitioned and the hardware structure to obtain a parallel execution part and a competitive execution part of the matrix to be partitioned, wherein the parallel execution part and the competitive execution part each include a plurality of sub-matrix blocks; The execution module, the neural network hardware accelerator performs computing tasks through competition between the sub-matrix blocks, performs computing tasks in parallel within the blocks, obtains multiple sub-matrix calculation results, and restores the original order of writing by searching the hash table; The merging module merges the calculation results of the multiple sub-matrices according to the original order to obtain the final result of the matrix-vector multiplication.

6. The hash-based sparse matrix vector multiplication optimization device according to claim 5, characterized in that: This initial module includes: According to the hardware structure, a two-dimensional partitioning model of the input sparse matrix is ​​constructed, and linear hash mapping is performed on the sparse matrix blocks, and the sparse matrix storage format is converted into the matrix to be partitioned in HBP format. Specifically: The format conversion is implemented in parallel between the divided sparse matrix blocks. The processing module mainly consists of three parts: data start and end position search, linear hash mapping within the sub-matrix block, and HBP storage format generation. The data start and end position search obtains the first non-zero element position of each row in the sparse matrix block currently being processed through binary search, and sequentially traverses until the last element position of the corresponding row in the range of the current sparse matrix block; The linear hash mapping within the sparse matrix block takes the number of non-zero elements in each row as input, and uses the characteristic of linear hash transformation to aggregate similar elements to map the row numbers of each row in the sparse matrix block to a hash table of the same size as before, and enables rows with the same or similar number of non-zero elements to be mapped to adjacent positions; the result of the hash transformation is stored in a fixed-length hash table, whose length is equal to the sum of the number of rows contained in each sparse matrix block after two-dimensional partitioning, and the size is the total number of sparse matrix rows num_of_row multiplied by the number of sparse matrix blocks in the column direction of the sparse matrix block_col_num. The initial value of the hash table is recorded as -1, indicating that the current position is empty; the index of the hash table is the order of actual execution, and the value stored in the hash table is the corresponding original row position. The specific hash transformation used is shown in the following formula: Wherein, output_hash is the position of the corresponding row of the output. In the matrix-vector multiplication calculation, the calculation is performed in sequence according to this position. Its range is greater than or equal to 0 and less than the number of rows in the block end_row. block_row_nnz is the number of non-zero elements in the row of the sparse matrix block. This value is obtained in the data start and end position search module. HBP_M and HBP_N are the row and column sizes of the two-dimensional division, respectively. ρ is the overall sparsity of the input matrix. ω is the size of a group of thread bundles in the single instruction multi-threading mode of the neural network accelerator. The value of HBP_M must be an integer multiple of ω. i is the position offset. When the position of the hash map conflicts, i is changed to find the adjacent free position. end_row is the number of rows contained in the sparse matrix block. The number of rows contained in the last row block in the sparse matrix may be less than HBP_M, which needs to be judged. The specific value of end_row is shown in the following formula: Where num_of_row is the number of rows of the input matrix as a whole, block_row_num is the number of sparse matrix blocks in the row direction after the input matrix is ​​divided into two dimensions, block_m is the row number corresponding to the sparse matrix block currently being processed, and the value range of block_m is greater than or equal to 0 and less than block_row_num; The HBP storage format includes the non-zero element column number cols, the non-zero element value data, the non-zero element index add_sign for accessing the non-zero element in the block, the zero row index zero_row, the location begin_nnz where the first non-zero element of each block is stored, and the hash table output_hash. The length of the HBP storage format on the non-zero element column number cols, the non-zero element value data, and the length of the non-zero element index add_sign are both equal to the number of non-zero elements. The length of the zero row index zero_row and the hash table output_hash is equal to the number of rows multiplied by the number of column blocks. The location begin_nnz where the first non-zero element of each block is stored is equal to the total number of sparse matrix blocks. Each sparse matrix block after two-dimensional partitioning is used as the basic unit for data storage and reading. The execution module includes: The computing task of competing among sub-matrix blocks refers to that each thread is a group, and the computing task of accessing storage and processing several blocks is performed in units of groups, and the threads within the group and between the groups are executed in parallel; when a group of threads finishes executing the parallel execution part, the sub-matrix blocks that have not been executed are selected from the competitive execution part to execute the computing task; The competitive execution part uses a ticket lock to avoid multiple accesses to the same sub-matrix block; the ticket lock uses an int type variable, which is modified with the volatile keyword during the parallel computing process to indicate that the variable will not be compiled and optimized. Each access is read from the storage. The variable stores the number of sub-matrix blocks that have not been calculated, and the initial value is equal to the number of sub-matrix blocks included in the competitive execution part; each group of threads atomically accesses the variable to obtain the next computing task after calculating the current task. If the return value is a negative number, it means that the competitive execution part has no unexecuted computing tasks; the content of each sub-matrix block after two-dimensional division that needs multiple random accesses during the calculation process, and the current execution block sequence number obtained by the ticket lock; The index of the hash table is the actual execution order of the thread, and the stored value is the order of each row before the hash transformation. Therefore, each thread of the neural network accelerator needs to access the corresponding position of the hash table after calculating a row of the sub-matrix block, and write the register result to the position before the hash transformation in the intermediate result vector mid_output.

7. The hash-based sparse matrix vector multiplication optimization device according to claim 5, characterized in that: The partitioning module includes: The division of the parallel execution part and the competitive execution part is calculated according to the scale of the sparse matrix and the size of the sparse matrix block. The division process of the parallel execution part follows the principle that each thread is assigned the same number of block calculation tasks, and the blocks to be calculated in the same thread should be located in the same column as much as possible; the part that is not divided into parallel execution is classified as the competitive execution part. Each sub-matrix block in the competitive execution part is competitively executed by the thread bundle that completes the assigned task first during the program running process.

8. An electronic device, characterized in that: It includes a hash-based sparse matrix-vector multiplication optimization device as described in claims 5-7, and the electronic device is connected to an information display device, which is used to display the final result of the matrix-vector multiplication according to display parameters and attributes set by the user or through an artificial intelligence model.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the hash-based sparse matrix-vector multiplication optimization methods described in claims 1-4.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the hash-based sparse matrix-vector multiplication optimization method described in any one of claims 1 to 4 are implemented.

Images