A sparse computing unit, method, device and medium under GPGPU architecture

Abstract

The present application relates to the field of sparse matrix calculation, and discloses a sparse calculation unit, method, device and medium under the GPGPU architecture, wherein the sparse calculation unit includes: a sparse matrix operation unit arranged in a streaming multiprocessor, the sparse matrix operation unit is connected to a transmitting module and a writing back module; the sparse matrix operation unit includes a selector, an algorithm module, an accumulator and a plurality of second buffer modules; the sparse matrix operation unit is used to calculate the sparse matrix; the algorithm module stores a calculation algorithm for calculating the matrix elements, and the calculation algorithm includes an improved SPGEMM algorithm and an addition algorithm; a memory access unit scheduling module connected to a plurality of streaming multiprocessors is used to coordinate and schedule the memory access units on the plurality of streaming multiprocessors. Adding a sparse matrix calculation unit increases the processing capability of GPGPU for sparse matrices and improves the operation speed of GPGPU for sparse matrix calculation tasks.

Description

Technical Field

[0001] The present application relates to the field of sparse matrix computing, and specifically to a sparse computing unit, method, device and medium under a GPGPU architecture. Background Art

[0002] In recent years, general-purpose computing on Graphics Processing Units (GPGPU) is a graphics processing unit (GPU) that uses graphics processing tasks to calculate general-purpose computing tasks originally handled by the central processing unit, rather than just for graphics rendering, which can significantly improve computing efficiency. At the same time, GPGPU has broad application prospects in scientific computing, deep learning, image processing, data analysis and many other scientific fields.

[0003] Sparse computing, as a method of matrix operation, also has advantages in the above-mentioned fields. However, when GPGPU performs sparse matrix operations, GPU sparse matrix operations occupy a large amount of resources. This can easily lead to a lack of computing resources for many GPU parallel computing units (CUDA cores or stream processors), thereby hindering the calculation of other processes, resulting in low computing power of GPGPU. In addition, GPGPU in existing technologies has poor support for different data types. Summary of the invention

[0004] In order to solve the above problems, the present application proposes a sparse computing unit, method, device and medium under the GPGPU architecture, wherein the sparse computing unit includes multiple streaming multiprocessors, each of which includes a thread warp scheduler, an instruction cache, a decoding module, a first buffer module, an operand collector, a scoreboard, a launch module, a write-back module, a memory access unit, a data cache and a shared storage unit; characterized in that the sparse computing unit under the GPGPU architecture also includes: a sparse matrix operation unit arranged in the streaming multiprocessor, the sparse matrix operation unit is connected to the launch module and the write-back module; the sparse matrix operation unit includes a selector, an algorithm module, an accumulator and multiple second buffer modules; the sparse matrix operation unit is used to calculate a sparse matrix, the algorithm module stores a calculation algorithm, and calculates matrix elements, the calculation algorithm includes an improved SPGEMM algorithm and an addition algorithm; a memory access unit scheduling module connected to the multiple streaming multiprocessors is used to coordinate and schedule the memory access units on the multiple streaming multiprocessors.

[0005] In one example, the selector is used to extract the matrix elements to be calculated; the multiple second buffer modules are connected to the selector, and are used to determine whether the matrix elements extracted by the selector are correct, and to temporarily store the extracted matrix elements; the algorithm module is arranged between the multiple second buffer modules and the accumulator; the accumulator is used to accumulate the calculation results of the matrix elements.

[0006] The present application also provides a sparse computing method under a GPGPU architecture, which is applied to a sparse computing unit under the above-mentioned GPGPU architecture. The sparse computing method includes: dividing a matrix to be calculated saved in a preset format into multiple intermediate sub-matrices through a memory access unit scheduling module, and the number of the intermediate sub-matrices is related to the number of streaming multiprocessors; sending a target computing task and the intermediate sub-matrices to a thread bundle scheduling module in the multiple streaming multiprocessors; dividing the target computing task into multiple computing sub-tasks through the thread bundle scheduling module, and dividing the intermediate sub-matrices again, and allocating the computing sub-tasks and the corresponding strip queues to different threads in strip queues consistent with the number of threads; completing the multiplication operation corresponding to the computing sub-task through a sparse matrix operation module to obtain an operator result; sending the operator result to the memory access unit scheduling module through the memory access unit, and splicing the operator results of each streaming multiprocessor through the memory access unit scheduling module to obtain a calculation result corresponding to the target computing task.

[0007] In one example, the multiplication operation corresponding to the calculation subtask is completed through the sparse matrix operation module to obtain the operator result, which specifically includes: based on the calculation subtask, determining the elements to be calculated and the calculation order; determining the first element to be calculated in the first bar queue through the selector, and determining the second element to be calculated in the second bar queue; buffering through the second buffer module during the calculation process to ensure that the selector correctly extracts data and temporarily stores the extracted data; after the calculation is completed, the calculation results are accumulated through the accumulator to obtain the operator result corresponding to the current thread.

[0008] In one example, sending the operator result to the memory access unit scheduling module through the memory access unit specifically includes: accumulating the calculation result through an accumulator to obtain the operator result corresponding to the current thread, and then storing the operator result in the memory access unit in the current streaming multiprocessor; receiving a data acquisition request from the memory access unit scheduling module, and determining the corresponding target streaming multiprocessor according to the data acquisition request; taking out the operator result in the memory access unit of the target streaming multiprocessor, and sending it to the memory access unit scheduling module.

[0009] In one example, the method of splicing the operator results of each streaming multiprocessor through the memory access unit scheduling module to obtain the calculation result corresponding to the target computing task specifically includes: integrating the operator results corresponding to all streaming multiprocessors in the secondary cache through the memory access unit scheduling module to obtain the calculation result; after obtaining the calculation result, the method also includes: storing the calculation result in a total storage unit under the GPGPU architecture; uploading the calculation result to a host computer connected to the GPGPU architecture through the total storage unit.

[0010] In one example, before the memory access unit scheduling module divides the matrix to be calculated saved in a preset format into a plurality of intermediate sub-matrices, the method further includes: saving the matrix to be calculated in an Ellpack-ltpack format.

[0011] In one example, the first buffer module is used to ensure that the strip queue is allocated in order and is correctly collected by the operand collector;

[0012] Before the operand collector completes data collection of the current batch of strip queues, the first buffer module is used to prevent the next batch of strip queue data from entering the operand collector.

[0013] The present application also provides a sparse computing device under a GPGPU architecture, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: dividing a matrix to be calculated saved in a preset format into multiple intermediate sub-matrices through a memory access unit scheduling module, and the number of the intermediate sub-matrices is related to the number of streaming multiprocessors; sending a target computing task and the intermediate sub-matrices to a thread bundle scheduling module in the multiple streaming multiprocessors; dividing the target computing task into multiple computing sub-tasks through the thread bundle scheduling module, and dividing the intermediate sub-matrices again, and allocating the computing sub-tasks and the corresponding strip queues to different threads in strip queues consistent with the number of threads; completing the multiplication operation corresponding to the computing sub-task through a sparse matrix operation module to obtain an operator result; sending the operator result to the memory access unit scheduling module through a memory access unit, and splicing the operator results of each streaming multiprocessor through the memory access unit scheduling module to obtain a computing result corresponding to the target computing task.

[0014] The present application also provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: divide a matrix to be calculated saved in a preset format into a plurality of intermediate sub-matrices through a memory access unit scheduling module, wherein the number of the intermediate sub-matrices is related to the number of streaming multiprocessors; send a target computing task and the intermediate sub-matrices to a thread bundle scheduling module within the plurality of streaming multiprocessors; divide the target computing task into a plurality of computing sub-tasks through the thread bundle scheduling module, and divide the intermediate sub-matrices again, and assign the computing sub-tasks and the corresponding strip queues to different threads in strip queues consistent with the number of threads; complete the multiplication operation corresponding to the computing sub-task through a sparse matrix operation module to obtain an operator result; send the operator result to the memory access unit scheduling module through the memory access unit, and splice the operator results of each streaming multiprocessor through the memory access unit scheduling module to obtain a computing result corresponding to the target computing task.

[0015] The method proposed in this application can bring the following beneficial effects: by designing a memory access unit scheduling module, memory access units on different SMs can be coordinated for collective scheduling, thereby improving the use function and efficiency of memory access units. By adding sparse matrix calculation units, the processing capability of GPGPU for sparse matrices is increased, and the operation speed of GPGPU for sparse matrix calculation tasks is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 Schematic diagram of a pipeline framework in a single streaming multiprocessor in an embodiment of the present application;

[0018] Figure 2 This is a schematic diagram of a GPGPU architecture in an embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of a sparse matrix computing unit architecture in an embodiment of the present application;

[0020] Figure 4 A schematic diagram of a sparse computing method under a GPGPU architecture in an embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of a matrix expansion method in an embodiment of the present application;

[0022] Figure 6 This is a schematic diagram of the structure of a sparse computing device under a GPGPU architecture in an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0024] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0025] This application provides a sparse computing unit under the GPGPU architecture. Now, the GPGPU architecture is explained. First, the entire GPGPU is composed of many streaming multiprocessors (SMs). The work of each streaming multiprocessor is based on threads. The threads work on the pipeline of each SM in units of thread bundles. Figure 1 As shown in the figure, the improvement of the pipeline part of a single streaming multiprocessor is that for the functional modules of the entire pipeline, the thread warp scheduler, instruction cache, decoding module, first buffer module, operand collector, scoreboard, etc. in the first half are close to or similar to the normal streaming multiprocessor architecture. The main update is that the sparse matrix operation unit is added to the second half of the streaming multiprocessor architecture on the basis of other common functional logic units. At the same time, Figure 2 As shown in FIG. 1 , a memory access unit scheduling module is set in the GPGPU architecture. In a single streaming multiprocessor, the sparse matrix operation unit is connected to the emission module and the write-back module. Figure 3 As shown, the sparse matrix operation unit includes a selector, an algorithm module, an accumulator, and a plurality of second buffer modules; the sparse matrix operation unit is used to calculate the sparse matrix; the algorithm module stores a calculation algorithm for calculating the matrix elements, and the calculation algorithm here includes an improved SPGEMM algorithm and an addition algorithm. The memory access unit scheduling module is used to coordinate and schedule the memory access units on the plurality of streaming multiprocessors. The memory access unit breaks through the limitation of working only between a single streaming multiprocessor, and can coordinate with multiple streaming multiprocessors with the help of the memory access unit scheduling module.

[0026] The first buffer module is used to prevent the next batch of strip queue data from entering the operand collector before the current batch of strip queues is collected by the operand collector. In this way, there is no need to design an arbitration module here to occupy more additional resources. Only the completion flag of the operand collector is needed to guide the next batch of data to be processed to enter.

[0027] In one embodiment, Figure 3 As shown, in the sparse matrix operation unit, the selector is used to extract the matrix elements to be calculated, and multiple second buffer modules are connected to the selector to determine whether the matrix elements extracted by the selector are correct and temporarily store the extracted matrix elements. The algorithm module is arranged between the multiple second buffer modules and the accumulator, and the accumulator is used to accumulate the calculation results of the matrix elements.

[0028] Figure 4 A schematic diagram of a sparse computing method under a GPGPU architecture provided in one or more embodiments of this specification. The method can be applied to the above-mentioned sparse computing unit, and some input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0029] The analysis method involved in the embodiments of the present application can be implemented by a terminal device or a server, and the present application does not impose any special restrictions on this. For the convenience of understanding and description, the following embodiments are described in detail by taking a server as an example.

[0030] It should be noted that the server can be a single device or a system composed of multiple devices, that is, a distributed server, and this application does not make any specific limitations on this.

[0031] like Figure 4 As shown, the embodiment of the present application provides a sparse computing method under a GPGPU architecture, including:

[0032] S401: dividing a matrix to be calculated saved in a preset format into a plurality of intermediate sub-matrices through a memory access unit scheduling module, where the number of the intermediate sub-matrices is related to the number of streaming multiprocessors.

[0033] First, the memory access unit scheduling module divides the matrix to be calculated saved in the Ellpack-ltpack format into multiple intermediate sub-matrices, and the number of intermediate sub-matrices is related to the number of streaming multiprocessors. Specifically, in order to save computing time, the number of intermediate sub-matrices can be lower than the number of streaming multiprocessors, or the number of intermediate sub-matrices can be a multiple of the number of streaming multiprocessors. Since the storage in the Ellpack-ltpack format does not require the use of pointers to index data, it only needs to extract non-zero elements. The operation process of extracting non-zero elements has better performance advantages compared to occupying a large number of PC pointers for calculations.

[0034] S402: Send the target computing task and the intermediate sub-matrix to the thread warp scheduling modules in the plurality of streaming multiprocessors.

[0035] Send the target computing task and the intermediate sub-matrix to the thread number scheduling module in multiple streaming multiprocessors. This application takes the current GPGPU with 4 streaming multiprocessors as an example, which is a relatively common configuration combination. However, in theory, if the performance does not meet the requirements, more SMs can be instantiated to complete the entire GPGPU design if the logical architecture is reasonable. Generally, a streaming multiprocessor can only operate one thread bundle at the same time. A thread bundle consists of multiple threads. Inside the streaming multiprocessor, data enters the streaming multiprocessor through the thread bundle. The thread bundle scheduling unit splits the thread bundle into many threads and performs computing operations in the streaming multiprocessor.

[0036] S403: Divide the target computing task into multiple computing subtasks through the thread warp scheduling module, and divide the intermediate submatrix again, and allocate the computing subtasks and corresponding strip queues to different threads in strip queues consistent with the number of threads.

[0037] After the thread warp scheduling module obtains the target computing task and the intermediate sub-matrix, it will split the intermediate sub-matrix again to obtain a strip queue, and the number of strip queues is consistent with the number of threads of the thread warp scheduling module. At the same time, the thread warp scheduling module will divide the original target computing task into multiple computing sub-tasks according to the strip queues obtained by splitting, and assign the corresponding computing sub-tasks and strip queues to different threads. When splitting the intermediate sub-matrix, a matrix frame of fixed specifications (such as 3*3) can be used for indexing, and the unused part can be filled with zeros to obtain a strip queue. The above expansion process is as follows: Figure 5 shown.

[0038] S404: completing the multiplication operation corresponding to the computing subtask through the sparse matrix operation module to obtain an operator result.

[0039] After the matrix with a larger dimension is divided twice to obtain multiple strip queues, the calculation tasks corresponding to each strip queue can be completed through the coefficient matrix operation module, thereby obtaining multiple operator results. It should be noted that the calculation tasks here include multiplication operations between different matrix elements and accumulation operations on multiplication results.

[0040] When performing calculations, it is first necessary to determine the elements to be calculated and the calculation order based on the calculation subtasks, and then determine the first element to be calculated in the first bar queue through the selector, and determine the second element to be calculated in the second bar queue. During the calculation process, the second buffer module is used for buffering to ensure that the selector extracts data correctly and temporarily stores the extracted data; after the calculation is completed, the calculation results are accumulated through the accumulator to obtain the operator results corresponding to the current thread. In layman's terms, when performing calculations, the corresponding elements are first selected through the selector, and buffered through the buffer module. After the calculation is completed, the current accumulated number C0 is formed through the accumulator.

[0041] S405: Send the operator result to the memory access unit scheduling module through the memory access unit, and splice the operator results of each streaming multiprocessor through the memory access unit scheduling module to obtain the calculation result corresponding to the target calculation task.

[0042] After the sparse matrix operation module calculates the operator result, it will send the operator result C0 to the memory access unit scheduling module through the memory access unit, and then perform the final result synthesis with C1, C2, etc. output by other streaming multiprocessors through the memory access scheduling module. Because for large matrices, multiple results may need to be submitted, and multiple feature maps composed of C0-CN are formed. The final operation can be completed after multiple accumulations. This operation is completed in the secondary cache and stored in the total storage unit after synthesis. At the same time, the total storage unit is connected to the host computer, and the results and contents in the total storage unit can be directly scheduled by the host computer to extract the current GPGPU calculation results.

[0043] In one embodiment, after obtaining the operator result corresponding to the current thread, the sparse matrix operation module will store the operator result in the memory access unit within the current streaming multiprocessor. When the server receives a data acquisition request from the memory access unit scheduling module, the server will determine the corresponding target streaming multiprocessor based on the data acquisition request, and then take out the operator result in the memory access unit of the target streaming multiprocessor and send it to the memory access unit scheduling module.

[0044] The improved SPGEMM algorithm proposed in the present application is explained here: it is different from the operation type of traditional sparse matrix operations. Compared with SPGEMM, which can perform matrix operations of different dimensions at will, the improved SPGEMM algorithm aims to give full play to the advantages of the algorithm, that is, when performing small-dimensional matrix operations, compared with the three existing algorithms of COO, CSR, and CSC, which all need to use pointers to locate data, the improved SPGEMM algorithm stores non-zero elements in an array of fixed length, so it is a matrix operation of smaller dimension allocated to each streaming multiprocessor. That is to say, the improved SPGEMM algorithm proposed in the present application specifies the scale of operations on the basis of the original algorithm, and splits large-scale matrix operations with the help of the high parallelism of GPGPU, and improves the operation efficiency of the original algorithm through the scheduling of memory access units.

[0045] At the same time, the actual matrix division is relatively simple. Assuming that the top-level matrix is ​​16*16 and there are 4 streaming multiprocessors in the GPGPU architecture, the memory access scheduling unit can first convert the matrix into 4 1*64 intermediate sub-matrices, that is, each streaming multiprocessor will be assigned a 1*64 intermediate sub-matrix, and then each intermediate sub-matrix will be split into multiple 1*9 strip queues through the thread warp scheduling module, and then the calculation is completed in the block.

[0046] Since high-concurrency storage logic will occupy a large number of execution units, thus affecting the execution efficiency of the entire system, what needs to be considered is a more reasonable allocation of resources so that more resources can be used for computing. Therefore, the strip queue is more conducive to the entire GPGPU to read data.

[0047] like Figure 6 As shown, an embodiment of the present application further provides a sparse computing device under a GPGPU architecture, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can:

[0048] The matrix to be calculated saved in a preset format is divided into multiple intermediate sub-matrices through a memory access unit scheduling module, and the number of the intermediate sub-matrices is related to the number of streaming multiprocessors; the target computing task and the intermediate sub-matrix are sent to the thread bundle scheduling module in the multiple streaming multiprocessors; the target computing task is divided into multiple computing sub-tasks through the thread bundle scheduling module, and the intermediate sub-matrix is ​​divided again, and the computing sub-tasks and the corresponding strip queues are allocated to different threads in strip queues consistent with the number of threads; the multiplication operation corresponding to the computing sub-task is completed through a sparse matrix operation module to obtain an operator result; the operator result is sent to the memory access unit scheduling module through the memory access unit, and the operator results of each streaming multiprocessor are spliced ​​through the memory access unit scheduling module to obtain the calculation result corresponding to the target computing task.

[0049] The embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0050] The matrix to be calculated saved in a preset format is divided into multiple intermediate sub-matrices through a memory access unit scheduling module, and the number of the intermediate sub-matrices is related to the number of streaming multiprocessors; the target computing task and the intermediate sub-matrix are sent to the thread bundle scheduling module in the multiple streaming multiprocessors; the target computing task is divided into multiple computing sub-tasks through the thread bundle scheduling module, and the intermediate sub-matrix is ​​divided again, and the computing sub-tasks and the corresponding strip queues are allocated to different threads in strip queues consistent with the number of threads; the multiplication operation corresponding to the computing sub-task is completed through a sparse matrix operation module to obtain an operator result; the operator result is sent to the memory access unit scheduling module through the memory access unit, and the operator results of each streaming multiprocessor are spliced ​​through the memory access unit scheduling module to obtain the calculation result corresponding to the target computing task.

[0051] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0052] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0053] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0054] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0055] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0056] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0057] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0058] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0059] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0060] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0061] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A sparse computing method under GPGPU architecture, characterized in that: A sparse computing unit applied to a GPGPU architecture, the sparse computing unit includes a plurality of streaming multiprocessors, each of which includes a thread warp scheduler, an instruction cache, a decoding module, a first buffer module, an operand collector, a scoreboard, a launch module, a write-back module, a memory access unit, a data cache and a shared storage unit; the sparse computing unit under the GPGPU architecture also includes: a sparse matrix operation unit arranged in the streaming multiprocessor, the sparse matrix operation unit is connected to the launch module and the write-back module; the sparse matrix operation unit includes a selector, an algorithm module, an accumulator and a plurality of second buffer modules; the sparse matrix operation unit is used to calculate a sparse matrix; the algorithm module stores a calculation algorithm for calculating matrix elements, the calculation The calculation algorithm includes an improved SPGEMM algorithm and an addition algorithm; the improved SPGEMM algorithm is to store non-zero elements in an array of fixed length, so that each stream multiprocessor is allocated to a matrix operation with a smaller dimension; a memory access unit scheduling module connected to the multiple stream multiprocessors is used to coordinate and schedule the memory access units on the multiple stream multiprocessors; the selector is used to extract the matrix elements to be calculated; the multiple second buffer modules are connected to the selector, and are used to determine whether the matrix elements extracted by the selector are correct, and temporarily store the extracted matrix elements; the algorithm module is set between the multiple second buffer modules and the accumulator; the accumulator is used to accumulate the calculation results of the matrix elements; the sparse calculation method includes: The memory access unit scheduling module divides the matrix to be calculated saved in a preset format into a plurality of intermediate sub-matrices, wherein the number of the intermediate sub-matrices is related to the number of the streaming multiprocessors; Sending the target computing task and the intermediate sub-matrix to the thread warp scheduling modules in the plurality of streaming multiprocessors; Divide the target computing task into multiple computing subtasks through the thread warp scheduling module, and divide the intermediate submatrix again, and allocate the computing subtasks and corresponding strip queues to different threads in strip queues consistent with the number of threads; The multiplication operation corresponding to the computing subtask is completed by a sparse matrix operation module to obtain an operator result; The operator result is sent to the memory access unit scheduling module through the memory access unit, and the operator results of each streaming multiprocessor are spliced ​​through the memory access unit scheduling module to obtain the calculation result corresponding to the target calculation task.

2. The sparse computing method under the GPGPU architecture according to claim 1, characterized in that: The multiplication operation corresponding to the computing subtask is completed by the sparse matrix operation module to obtain the operator result, specifically including: Based on the calculation subtasks, determine the elements to be calculated and the calculation order; Determine a first element to be calculated in the first bar queue by using a selector, and determine a second element to be calculated in the second bar queue; During the calculation process, the second buffer module performs buffering to ensure that the selector extracts data correctly and temporarily stores the extracted data; After the calculation is completed, the calculation results are accumulated through the accumulator to obtain the operator result corresponding to the current thread.

3. The sparse computing method under the GPGPU architecture according to claim 2, characterized in that: The sending the operator result to the memory access unit scheduling module through the memory access unit specifically includes: After accumulating the calculation results through the accumulator to obtain the operator result corresponding to the current thread, the operator result is stored in the memory access unit in the current streaming multiprocessor; Receiving a data acquisition request from a memory access unit scheduling module, and determining a corresponding target streaming multiprocessor according to the data acquisition request; The operator results in the memory access unit of the target streaming multiprocessor are taken out and sent to the memory access unit scheduling module.

4. The sparse computing method under the GPGPU architecture according to claim 2, characterized in that: The memory access unit scheduling module is used to splice the operator results of each streaming multiprocessor to obtain the calculation result corresponding to the target computing task, specifically including: The operation results corresponding to all the streaming multiprocessors are integrated in the secondary cache through the memory access unit scheduling module to obtain the calculation result; After obtaining the calculation result, the method further includes: Storing the calculation result in a total storage unit under the GPGPU architecture; The calculation result is uploaded to a host computer connected to the GPGPU architecture through the total storage unit.

5. The sparse computing method under the GPGPU architecture according to claim 1, characterized in that: Before the memory access unit scheduling module divides the matrix to be calculated stored in a preset format into a plurality of intermediate sub-matrices, the method further includes: The matrix to be calculated is saved in Ellpack-ltpack format.

6. The sparse computing method under the GPGPU architecture according to claim 1, characterized in that: The first buffer module is used to ensure that the strip queues are allocated in order and are correctly collected by the operand collector; Before the operand collector completes data collection of the current batch of strip queues, the first buffer module is used to prevent the next batch of strip queue data from entering the operand collector.

7. A sparse computing device under GPGPU architecture, characterized in that: The sparse computing device comprises: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the method according to any one of claims 1 to 6.

8. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to execute the steps of the method according to any one of claims 1 to 6.

Images