Tensor core-based graph neural network computation acceleration method, system, medium and electronic device

Abstract

The application provides a Tensor Core-based graph neural network calculation acceleration method, system, medium and electronic equipment, and the method comprises the following steps: dividing a sparse matrix into a plurality of sub-matrix blocks, the size of each sub-matrix block being matched with the calculation granularity of a matrix operation unit of a Tensor Core; and rearranging all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core. The Tensor Core-based graph neural network calculation acceleration method, system, medium and electronic equipment can effectively improve the calculation performance and efficiency of a graph neural network on a Tensor Core.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of graph neural networks, and in particular to a graph neural network calculation acceleration method and system based on Tensor Core, a medium and an electronic device. BACKGROUND

[0002] A graph neural network (GNN) is a type of neural network model that is specifically designed to process graph-structured data. GNNs can capture complex dependencies between nodes in a graph through the structure of the nodes and edges, and are widely used in social network analysis, recommendation systems, chemical molecule modeling, and other fields. In the calculation of GNNs, the features of nodes are usually updated through aggregation operations on neighboring nodes.

[0003] Due to the sparsity of graph data, the connections between nodes are often much lower than the dense relationships in fully connected networks, so matrices in GNNs are mostly sparse. The calculation of large-scale graph neural networks relies on efficient sparse matrix multiplication to avoid unnecessary waste of computing resources. However, the efficiency of sparse matrix operations is relatively low, especially on parallel computing platforms such as GPUs, which can lead to a lot of redundant calculations. Therefore, how to efficiently handle sparse matrix operations in GNNs has become a key problem.

[0004] In GNNs, mixed-precision quantization is a key method to improve computational efficiency. This quantization method takes advantage of the non-uniform characteristics of graph structures, and adaptively allocates quantization precision according to the different features of nodes and the needs of network layers, thereby reducing the demand for memory and computing resources while ensuring the performance of GNNs. For example, the A2Q algorithm is a typical mixed-precision quantization method. The memory required by the quantized model is used as a regular term in the loss function, so that the quantization precision can be adaptively adjusted during model training to learn the appropriate quantization precision. For example, nodes with high in-degree may have a greater impact on the graph structure, so they are given higher precision. Mixed-precision quantization ensures high precision for key nodes and layers while accelerating the overall calculation by reducing the precision of other parts. Through flexible precision allocation, the resource utilization of the model is improved.

[0005] GPUs are widely used hardware accelerators in deep learning, and their highly parallel computing capabilities enable them to effectively accelerate large-scale matrix operations. To further improve the performance of matrix operations in deep learning, NVIDIA introduced Tensor Cores in its Volta architecture and later GPUs. A Tensor Core is a hardware unit specifically designed for matrix multiplication, capable of performing matrix multiplication and accumulation (MMA) operations of different sizes, such as 16x16x16 or 16x8x16, and different precisions, such as FP16, INT8, and INT1, greatly improving the throughput of matrix operations.

[0006] Tensor Cores are optimized for matrix operations common in deep learning, offering particularly high computational throughput when dealing with dense matrices. However, the hardware characteristics of Tensor Cores are primarily geared towards dense matrix computations; directly utilizing Tensor Cores for sparse matrix multiplication may not fully realize their potential. Fixed-size computational blocks can lead to wasted GPU memory and computational resources, making the efficient utilization of sparse matrices on Tensor Cores an important research direction.

[0007] Tensor Core units only support specific structured sparsity and do not support unstructured sparsity in GNNs. To fully leverage the performance advantages of Tensor Cores when processing sparse matrices, sparse matrix compression and rearrangement techniques are crucial. Common traditional matrix compression techniques include BCSR (Block Compressed Row Storage) and SR-BCRS (Segmented Row Block Compressed Row Storage), which reduce memory requirements and accelerate computation by compressing zero elements in the sparse matrix. However, even with these compression methods, the non-zero blocks of the sparse matrix still contain a significant number of zero elements, leading to unnecessary memory accesses and computations.

[0008] Furthermore, Tensor Cores only support fixed-precision operations such as 1-bit, 4-bit, and 8-bit, while mixed-precision computations require flexible adjustment of the precision of different nodes. Therefore, accelerating mixed-precision computations on Tensor Cores is a challenge. Summary of the Invention

[0009] In view of the above-mentioned problems, the purpose of this invention is to provide a method, system, medium, and electronic device for accelerating graph neural network computation based on Tensor Core, which can effectively improve the computational performance and efficiency of graph neural networks on Tensor Core.

[0010] In a first aspect, the present application provides a Tensor Core-based graph neural network computing acceleration method, which comprises the following steps: dividing a sparse matrix into a plurality of sub-matrix blocks, the size of each sub-matrix block matching the calculation granularity of the matrix operation unit of the Tensor Core; rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0011] In an implementation form of the first aspect, the rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core comprises the following steps:

[0012] setting a threshold;

[0013] when the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, setting the sub-matrix block as a target sub-matrix block;

[0014] constructing a horizontal sub-matrix group by arranging the target sub-matrix blocks in a horizontal direction;

[0015] obtaining the number of common neighbors of the row where the all-zero rows in the target sub-matrix block are located in the vertical direction with all horizontal sub-matrix groups;

[0016] moving the row where the all-zero rows are located to the row where the all-zero rows are located in the horizontal sub-matrix group with the largest number of common neighbors to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0017] In an implementation form of the first aspect, further comprising performing compression acceleration processing on a feature matrix; wherein the compression acceleration processing on the feature matrix comprises the following steps:

[0018] performing bit splitting on the feature matrix for the maximum precision to obtain a split matrix, and generating a quantization mask for each split matrix, each bit in the quantization mask indicating whether the corresponding element in the split matrix is quantized to a preset precision;

[0019] performing a bitwise OR operation on the quantization mask according to a 32-bit granularity to generate a compressed mask;

[0020] only performing data calculation and access on the elements of the compressed mask that are not 0.

[0021] In an implementation form of the first aspect, further comprising performing compression acceleration processing on a left matrix and a right matrix; the compression acceleration processing on the left matrix and the right matrix comprises the following steps:

[0022] The sub-matrix block is divided into a plurality of sub-matrix units in a granularity of 32 bits;

[0023] A compression mask is set for each sub-matrix unit; wherein when the elements of the sub-matrix unit are all 0, the compression mask is 0, otherwise the compression mask is 1;

[0024] The compression encodings of the left matrix and the right matrix are subjected to a bitwise AND operation to obtain a result mask;

[0025] When the multiplication operation of the left matrix and the right matrix is performed, the sub-matrix units with the result mask of 0 are skipped.

[0026] In an implementation form of the first aspect, the compression masks of the left matrix and the right matrix are saved in a shared register.

[0027] In an implementation form of the first aspect, further comprising when the result mask of the left matrix and the right matrix contains four 1s, the sub-matrix units corresponding to the result mask are spliced into a sub-matrix combination unit;

[0028] The prefix sums of the result masks of the left matrix and the right matrix are calculated respectively, and the prefix sums are used as the indexes of the sub-matrix combination unit;

[0029] The corresponding data of the left matrix and the right matrix are obtained based on the indexes to complete the operation of the sub-matrix combination unit of the left matrix and the right matrix.

[0030] In an implementation form of the first aspect, the corresponding data of the left matrix and the right matrix are stored in a shared register.

[0031] In a second aspect, the present application provides a Tensor Core-based graph neural network computing acceleration system, which comprises a division module and a rearrangement module.

[0032] The division module is configured to divide a sparse matrix into a plurality of sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core;

[0033] The rearrangement module is configured to rearrange the all-zero rows in the sub-matrix block to construct an all-zero sub-matrix block in the sparse matrix and skip the calculation of the all-zero sub-matrix block in the Tensor Core.

[0034] In a third aspect, the present application provides a storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the above-mentioned Tensor Core-based graph neural network computing acceleration method.

[0035] In a fourth aspect, the present application provides an electronic device, comprising a processor and a memory;

[0036] The memory is used to store a computer program;

[0037] The processor is used to execute the computer program stored in the memory, so that the electronic device executes the above-mentioned Tensor Core-based graph neural network calculation acceleration method.

[0038] As described above, the Tensor Core-based graph neural network calculation acceleration method, system, medium and electronic device of the present application have the following beneficial effects:

[0039] (1) By rearranging the non-zero elements of the sparse matrix to adapt to the hardware characteristics of Tensor Core for efficient parallel calculation, the compression rate of the sparse matrix is effectively improved, the effective calculation density is increased, and the resource utilization of Tensor Core is improved;

[0040] (2) Through the dynamic matrix compression format for Tensor Core, the change of matrix sparsity can be adapted in real time, and the effective combination of compression and calculation is realized;

[0041] (3) Mixed precision operation can be performed on Tensor Core;

[0042] (4) The calculation performance and efficiency of the graph neural network are effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A flowchart showing the Tensor Core-based graph neural network calculation acceleration method of the present application in an embodiment;

[0044] Figure 2 A schematic diagram showing the rearrangement of the sparse matrix in the present application in an embodiment;

[0045] Figure 3 A bit splitting schematic diagram showing the mixed precision quantization in the present application in an embodiment;

[0046] Figure 4 A schematic diagram showing the quantization mask in the present application in an embodiment;

[0047] Figure 5 A schematic diagram showing the dynamic compression in the present application in an embodiment;

[0048] Figure 6 A structural schematic diagram showing the Tensor Core-based graph neural network calculation acceleration system of the present application in an embodiment;

[0049] Figure 7 Fig. 1 shows a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] Other advantages and effects of the present application will be apparent to those skilled in the art from the following description of specific embodiments of the present application. The present application can be implemented or applied in other different embodiments, and various modifications or changes can be made based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] It should be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application, and only the components related to the present application are shown in the drawings, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change in shape, number and proportion, and the layout of the components may be more complex.

[0052] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings of the embodiments of the present application.

[0053] As shown in Figure 1 In an embodiment, the Tensor Core-based graph neural network calculation acceleration method of the present application includes steps S1-S2.

[0054] Step S1, divide the sparse matrix into a plurality of sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core.

[0055] Specifically, in the calculation of the Tensor Core, the sparse matrix is divided into a plurality of sub-matrix blocks of fixed size. The size of each sub-matrix block matches the calculation granularity of the matrix operation unit (such as 16x8 or 16x16) of the Tensor Core.

[0056] Step S2, rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0057] Specifically, in order to fully utilize the parallel computing capability of the GPU Tensor Core, the present application rearranges the all-zero rows in the sub-block matrix to construct an all-zero sub-block. For the all-zero sub-block, the calculation and memory access can be omitted, thereby improving the calculation efficiency of the graph neural network.

[0058] In an embodiment, rearranging all-zero rows in the sub-block matrix to construct all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in Tensor Core includes the following steps:

[0059] 21) Set a threshold.

[0060] The threshold is related to the number of row rearrangements of subsequent sub-matrix blocks, so the size of the threshold cannot be too large, otherwise it will increase the complexity of row rearrangement. Preferably, the threshold is 1.

[0061] 22) When the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, set the sub-matrix block as a target sub-matrix block.

[0062] As shown in the following table, take the size of Tensor Block as 4x3 for example, and set the threshold of each sub-matrix block as 1. Therefore, find the sub-matrix block whose all-zero rows are greater than 0 and less than or equal to 1. Figure 2 As shown in the following table, take the size of Tensor Block as 4x3 for example, and set the threshold of each sub-matrix block as 1. Therefore, find the sub-matrix block whose all-zero rows are greater than 0 and less than or equal to 1. Figure 2 The first layer includes rows 1-4, and the second layer includes rows 5-8. The two layers include 6 sub-matrix blocks in total. The second sub-matrix block in the first layer and the third sub-matrix block in the second layer are target sub-matrix blocks, and the all-zero rows in the two blocks are the 3rd row and the 7th row, respectively.

[0063] 23) Construct a row of sub-matrix blocks in the horizontal direction as a horizontal sub-matrix group.

[0064] The following table shows an example of a horizontal sub-matrix group. Figure 2 The following table shows an example of a horizontal sub-matrix group.

[0065] 24) Get the number of common neighbors of the row where the all-zero row in the target sub-matrix block is located in the vertical direction with all horizontal sub-matrix groups.

[0066] For the second sub-matrix block in the first layer, the 3rd row is an all-zero row, so the 3rd row of the entire Figure 2 has 1 common neighbor with the first horizontal sub-matrix group; and 2 common neighbors with the second horizontal sub-matrix group. The common neighbor refers to the number of non-zero elements in the same vertical direction as the target element.

[0067] 25) Move the row where the all-zero row is located to the row where the all-zero row is located in the horizontal sub-matrix group with the most common neighbors, to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0068] Wherein, since the common neighbor of the second transverse sub-matrix group is 2, which is greater than the number of common neighbors of the first transverse sub-matrix group, the third row is moved to the all-zero row (i.e. the seventh row) of the second transverse sub-matrix group; correspondingly, the seventh row is moved to the third row, thereby constructing two all-zero matrices as shown in the right graph of Figure 2 Therefore, by allocating the all-zero row to the sub-matrix block with more common neighbors in the above manner, the density of the overall sub-matrix block is improved, and the sparsity is reduced. After rearrangement, a new sparse matrix layout is generated, and the all-zero block is skipped. In this compression scheme, the non-zero elements of the sparse matrix are rearranged into a block structure suitable for efficient parallel computation by Tensor Core, which not only reduces storage overhead, but also improves the computation efficiency of Tensor Core. Therefore, by using the above rearrangement-based sparse matrix block compression scheme, the hardware acceleration capability of Tensor Core can be fully utilized while maintaining the storage efficiency of the sparse matrix, thereby improving the performance of sparse matrix multiplication in the graph neural network. In the calculation process of the graph neural network, especially in the processing scenario of large-scale sparse graph data, the present application can significantly reduce the calculation time and improve the overall system performance.

[0069] In the calculation of the graph neural network, three matrices are mainly involved, namely the adjacency matrix A, the feature matrix X and the weight matrix W. In mixed precision quantization, the adjacency matrix A and the weight matrix W are generally quantized to a fixed precision, while the feature matrix X is quantized to a dynamic precision. Different nodes of different network layers are quantized to different precisions. Since the precision that can be supported by Tensor Core is limited, the existing method mainly uses the bit splitting method to support the calculation of extended precision, that is, the features of different bits are separated into separate matrices. Since only part of the nodes are quantized to high precision, this bit splitting method will bring greater sparsity in mixed precision calculation, resulting in a large amount of invalid calculation and memory access. As shown in Figure 3 The feature matrix X in the above formula (1) has only one node quantized to 3 bits, so the bit splitting method will result in high sparsity of the third bit. Considering that mixed precision calculation will introduce greater sparsity to the feature matrix of the graph neural network, and considering the change of the sparsity of the sparse matrix in the calculation process of the graph neural network, the present application also includes a compression and acceleration process for the feature matrix to meet the calculation requirements of different sparse matrices.

[0070] For the feature matrix X, bit splitting is performed for maximum precision to obtain a split matrix, and a quant mask is generated for each split matrix. Each bit in the quant mask represents whether the corresponding element in the split matrix is quantized to a preset precision. The quant masks are subjected to a bitwise OR (OR) operation in 32-bit granularity to generate a compress mask, which is used to represent whether the corresponding 32 bits of the feature matrix X are 0. As shown in Figure 4 For the feature matrix X quantized to 3 bits at the highest precision, three corresponding compress masks are generated. Data calculation and access are only performed for elements in which the compress mask is not 0.

[0071] The Tensor Core-based graph neural network calculation acceleration method of the present application further includes compressed acceleration calculation of the left matrix and the right matrix. For example, the left matrix is the adjacency matrix A, and the right matrix is the feature matrix X.

[0072] In an embodiment, when the left matrix and the right matrix are subjected to compressed acceleration calculation, the sub-matrix block is divided into a plurality of sub-matrix units sub-blocks in 32-bit granularity. A compress mask is set for each sub-matrix unit. When the elements of the sub-matrix unit are all 0, the compress mask is 0, otherwise the compress mask is 1. For the sub-matrix unit with a compress mask of 0, its memory access can be omitted. The compressed encoding of the left matrix and the right matrix is subjected to a bitwise AND (AND) operation to obtain a result mask. When the multiplication operation of the left matrix and the right matrix is performed, the sub-matrix unit with a result mask of 0 is skipped. As shown in Figure 5 Each sub-matrix block is divided into 4 sub-matrix units, and the compress masks of the 2nd bit of the left matrix and the 3rd bit of the right matrix are 0, so the storage of the corresponding two sub-matrix units can be omitted. The 2nd and 3rd bits of the corresponding result mask are both 0, so the multiplication of the 2nd and 3rd sub-matrix units of the left matrix and the right matrix can be skipped.

[0073] In an embodiment, the compress masks of the left matrix and the right matrix are saved in a shared register (Uniform Register, UR). At the same time, each UR register can save a compress mask of 32-bit length, and each bit corresponds to a sub-matrix unit sub-block.

[0074] In order to effectively organize the calculation of the sub-block, four effective sub-matrix units sub-blocks need to be spliced into a complete sub-matrix combination unit. For the result mask obtained by the bitwise AND operation, when the result masks of the left matrix and the right matrix contain four 1s, the sub-matrix units corresponding to the result mask are spliced into a sub-matrix combination unit. It should be noted that the four 1s can be four consecutive 1s, or four non-consecutive 1s (that is, there can be 0 between two 1s). For the left matrix and the right matrix, the prefix sums corresponding to the respective result masks are calculated, and the prefix sums are used as the indexes of the respective obtained sub-matrix combination units. Based on the indexes, the corresponding data of the left matrix and the right matrix are obtained, and the operation of the sub-matrix combination units of the left matrix and the right matrix is completed. The corresponding data of the left matrix and the right matrix are stored in a shared register, so as to reduce the access to the global memory.

[0075] In the present application, each sub-matrix combination unit corresponds to the operation of one warp. Each Cooperative Thread Array (CTA) is responsible for calculating one sub-matrix combination unit of the result matrix. Each warp reads the corresponding data of the left matrix and the right matrix in the shared register according to the index of the calculated sub-matrix combination unit, and initializes the fragment of the Tensor Core. After calling the wmma interface provided by CUDA to execute the Tensor Core matrix multiplication, the corresponding result is obtained, and the result is stored back to the corresponding position, wherein the corresponding row and column in the result matrix are equal to the row of the left matrix and the column of the right matrix, respectively.

[0076] The protection scope of the Tensor Core-based graph neural network calculation acceleration method described in the embodiments of the present application is not limited to the execution order of the steps listed in the embodiments. Any scheme realized by adding, replacing or replacing steps of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0077] The embodiments of the present application also provide a Tensor Core-based graph neural network calculation acceleration system, which can implement the Tensor Core-based graph neural network calculation acceleration method described in the present application. However, the implementation device of the Tensor Core-based graph neural network calculation acceleration system described in the present application includes but is not limited to the structure of the Tensor Core-based graph neural network calculation acceleration system listed in the embodiments. Any structure deformation and replacement of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0078] As Figure 6As shown, in an embodiment, the Tensor Core-based graph neural network computing acceleration system of the present application comprises a division module 61 and a rearrangement module 62.

[0079] The division module 61 is configured to divide the sparse matrix into a plurality of sub-matrix blocks, each of which has a size matching the calculation granularity of the matrix operation unit of the Tensor Core.

[0080] The rearrangement module 62 is connected to the division module 61 and is configured to rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0081] Wherein, the structure and principle of the division module 61 and the rearrangement module 62 correspond to the above-mentioned Tensor Core-based graph neural network computing acceleration method one by one, and therefore will not be repeated here.

[0082] In several embodiments provided by the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules / units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutual elements can be indirect coupling or communication connection through some interface, device or module or unit, which can be electrical, mechanical or other forms.

[0083] The modules / units described as separate components can or can not be physically separated, and the components shown as modules / units can or can not be physical modules, i.e. they can be located in one place or distributed on a plurality of network units. Part or all of the modules / units can be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in each embodiment of the present application can be integrated in one processing module, or each module / unit can be physically separated, or two or more modules / units can be integrated in one module / unit.

[0084] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, and that the described examples are merely illustrative of the principles of the application. In order to clearly illustrate the interchangeability of hardware and software, various components and steps have been described above generally in terms of their functionality, without reference to the detailed programming techniques in which such functionality can be implemented. Such programming techniques are well known to those of ordinary skill in the art and are intended to be encompassed by the described examples. The described examples can be implemented in one or more computer programs or software modules that are executed by one or more computers or processors.

[0085] The embodiments of the present application also provide a computer readable storage medium. Those of ordinary skill in the art can understand that all or part of the steps of the Tensor Core-based graph neural network computing acceleration method described above can be completed by a processor instructed by a program, and the program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid state disk (SSD)), etc.

[0086] The embodiments of the present application also provide an electronic device. The electronic device includes a processor and a memory.

[0087] The memory is used to store a computer program.

[0088] The memory includes a ROM, a RAM, a disk, a U disk, a memory card, or an optical disc, and various media that can store program codes.

[0089] The processor is connected with the memory, and is used to execute the computer program stored in the memory, so that the electronic device executes the Tensor Core-based graph neural network computing acceleration method described above.

[0090] Preferably, the processor can be a general processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; also can be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0091] As shown in Figure 7 The electronic device of the present application is in the form of a general computing device. Components of the electronic device can include, but are not limited to, one or more processors or processing units 71, a memory 72, a bus 73 that connects the various system components including the memory 72 and the processing unit 71.

[0092] The bus 73 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures including a Industry Standard Architecture (ISA), Micro Channel Architecture (MAC), Enhanced ISA (EISA), Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0093] The electronic device typically includes a variety of computer system readable media. Such media can be any available media that is accessible by the electronic device and includes both volatile and non-volatile media, removable and non-removable media.

[0094] The memory 72 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 721 and / or cache memory 722. The electronic device can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 723 can be used for reading from and writing to non-removable, non-volatile magnetic media (e.g., a "hard drive"). Figure 7 not shown, is typically provided as residual storage space on the storage system 723. Although not specifically shown, such computer system typically will also include other removable / non-removable, volatile / non-volatile computer system storage media including, but not limited to, a magnetic floppy disk drive (with related flexible disk controller "FDC" (not shown)) for easily reading from and writing to a removable, non-removable, volatile, non-volatile storage medium (e.g., an external hard drive not shown). Figure 7A disk drive, a floppy disk drive, and / or other mass storage device can be provided for in the example of FIG. 8, which can be used to read from or write to a removable, nonvolatile magnetic or optical media (e.g., a floppy disk). The disk drive can be connected to the system bus 73 by a data media interface. The memory 72 can include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiments of the application.

[0095] Program / utility 724 having a set (at least one) of program modules 7241 can be stored in memory 72 by way of example, and not limitation, as well as an operating system, one or more application programs, other program modules, and program data, each or some combination thereof, can include implementation of a networking environment. Program modules 7241 generally carry out the functions and / or methodologies of embodiments of the application as described herein.

[0096] The electronic device can also communicate with one or more external devices such as a keyboard or a pointing device, a display, etc. through I / O interface 74 and with one or more devices that enable a user to interact with the electronic device and / or one or more devices that enable the electronic device to communicate with one or more other computing devices. Such communication can occur via I / O interface 74. Additionally, the electronic device can communicate with one or more networks such as a local area network (LAN), a wide area network (WAN), and / or the Internet through network adapter 75. As Figure 7 illustrated, network adapter 75 can communicate with the other components of the electronic device through bus 73. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with the electronic device. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0097] The foregoing embodiments serve merely as illustration of the principles of the present application and the efficacy thereof, and are not intended to limit the present application in any way. Any skilled person can make modifications or changes to the foregoing embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical thought of the present application shall be covered by the claims of the present application.

Claims

1. A Tensor Core-based graph neural network computing acceleration method, characterized in that: The method comprises the following steps: The sparse matrix is divided into a plurality of sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core; The all-zero rows in the sub-matrix blocks are rearranged to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core; The all-zero rows in the sub-block matrix are rearranged to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core, comprising the following steps: A threshold is set; When the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, the sub-matrix block is set as a target sub-matrix block; A row of sub-matrix blocks in the transverse direction is constructed into a transverse sub-matrix group; The number of common neighbors of the row where the all-zero rows in the target sub-matrix block are located in the vertical direction and all transverse sub-matrix groups is obtained; The row where the all-zero rows are located is moved to the row where the all-zero rows in the transverse sub-matrix group with the maximum number of common neighbors are located, to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core; Further comprising compressed acceleration processing of a feature matrix; wherein the compressed acceleration calculation of the feature matrix comprises the following steps: For the feature matrix, bit splitting is performed for the maximum precision to obtain a split matrix, and a quantization mask is generated for each split matrix, each bit in the quantization mask indicating whether the corresponding element in the split matrix is quantized to a preset precision; The quantization mask is subjected to a bitwise OR operation in 32-bit granularity to generate a compressed mask; Only elements of the compressed mask that are not 0 are subjected to data calculation and access; Further comprising compressed acceleration calculation of left and right matrices; the compressed acceleration calculation of the left and right matrices comprises the following steps: The sub-matrix blocks are divided into a plurality of sub-matrix units in 32-bit granularity; A compressed mask is set for each sub-matrix unit; wherein when the elements of the sub-matrix unit are all 0, the compressed mask is 0, otherwise the compressed mask is 1; The compressed encodings of the left and right matrices are subjected to a bitwise AND operation to obtain a result mask; When performing multiplication operations on the left and right matrices, the sub-matrix units with the result mask of 0 are skipped; The compressed masks of the left and right matrices are saved in a shared register.

2. The Tensor Core-based graph neural network computing acceleration method according to claim 1, wherein: Further comprising when the result mask of the left and right matrices contains four 1s, the sub-matrix units corresponding to the result mask are spliced into a sub-matrix combination unit; The prefix sums of the result masks of the left and right matrices are calculated respectively, and the prefix sums are used as the indexes of the sub-matrix combination unit; Based on the indexes, the corresponding data of the left and right matrices is obtained to complete the operation of the sub-matrix combination unit of the left and right matrices.

3. The Tensor Core-based graph neural network computing acceleration method according to claim 2, wherein: The corresponding data of the left and right matrices are stored in a shared register.

4. A Tensor Core-based graph neural network computing acceleration system, characterized in that: The system comprises a division module and a rearrangement module; The dividing module is configured to divide the sparse matrix into a plurality of sub-matrix blocks, each of which has a size matching a calculation granularity of a matrix operation unit of a TensorCore; The rearranging module is configured to rearrange all-zero rows in the sub-matrix blocks to construct all-zero sub-block matrices in the sparse matrix and skip calculation of the all-zero sub-block matrices in the Tensor Core; Rearranging all-zero rows in the sub-block matrices to construct all-zero sub-block matrices in the sparse matrix and skip calculation of the all-zero sub-block matrices in the Tensor Core comprises the following steps: setting a threshold; when the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, setting the sub-matrix block as a target sub-matrix block; constructing a row of sub-matrix blocks in a horizontal direction into a horizontal sub-matrix group; obtaining a number of common neighbors of a row where all-zero rows in the target sub-matrix block are located in a vertical direction with all horizontal sub-matrix groups; moving the row where the all-zero rows are located to a row where all-zero rows in a horizontal sub-matrix group with the largest number of common neighbors are located to construct all-zero sub-block matrices in the sparse matrix and skip calculation of the all-zero sub-block matrices in the Tensor Core; Further comprising compressing and accelerating calculation of a feature matrix; wherein compressing and accelerating calculation of the feature matrix comprises the following steps: bit splitting the feature matrix for maximum precision to obtain a split matrix, and generating a quantization mask for each split matrix, each bit in the quantization mask indicating whether a corresponding element in the split matrix is quantized to a preset precision; performing a bitwise OR operation on the quantization masks in 32-bit granularity to generate a compressed mask; only performing data calculation and access for elements in the compressed mask that are not 0; Further comprising compressing and accelerating calculation of a left matrix and a right matrix; compressing and accelerating calculation of the left matrix and the right matrix comprises the following steps: dividing the sub-matrix block into a plurality of sub-matrix units in 32-bit granularity; setting a compressed mask for each sub-matrix unit; wherein the compressed mask is 0 when elements in the sub-matrix unit are all 0, otherwise the compressed mask is 1; performing a bitwise AND operation on compressed encodings of the left matrix and the right matrix to obtain a result mask; skipping sub-matrix units with the result mask being 0 when performing multiplication operation on the left matrix and the right matrix; saving the compressed masks of the left matrix and the right matrix in a shared register.

5. A storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the Tensor Core-based graph neural network calculation acceleration method of any one of claims 1 to 3.

6. An electronic device, comprising: It comprises: a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program stored in the memory to enable the electronic device to perform the Tensor Core-based graph neural network calculation acceleration method of any one of claims 1 to 3.

Images