Graph neural network calculation acceleration method and system based on Tensor Core, medium and electronic equipment

Abstract

The invention provides a Tensor Core-based graph neural network calculation acceleration method, a Tensor Core-based graph neural network calculation acceleration system, a medium and electronic equipment, and the method comprises the following steps: dividing a sparse matrix into a plurality of sub-matrix blocks, the size of each sub-matrix block being matched with the calculation granularity of a Tensor Core matrix operation unit; all-zero lines in the sub-block matrix are rearranged, so that an all-zero sub-block matrix is constructed in the sparse matrix, and calculation of the all-zero sub-block matrix is skipped in Tensor Core. According to the Tensor Core-based graph neural network calculation acceleration method, the Tensor Core-based graph neural network calculation acceleration system, the medium and the electronic equipment, the calculation performance and efficiency of the graph neural network can be effectively improved on the Tensor Core.

Description

Technical Field

[0001] The present invention relates to the technical field of graph neural networks, and in particular to a graph neural network computing acceleration method, system, medium, and electronic device based on Tensor Core. Background Art

[0002] Graph Neural Network (GNN) is a type of neural network model that specializes in processing graph structured data. GNN can capture the complex dependencies between nodes in a graph through the structure of nodes and edges in the graph, and is widely used in social network analysis, recommendation systems, chemical molecular modeling and other fields. In the calculation of GNN, the features of a node are usually updated through the aggregation operation of neighboring nodes.

[0003] Due to the sparsity of graph data, the connections between nodes are often much lower than the dense relationships in fully connected networks, so most matrices in GNNs are sparse. The calculation of large-scale graph neural networks relies on efficient sparse matrix multiplication to avoid unnecessary waste of computing resources. However, sparse matrix operations are inefficient, especially on parallel computing platforms such as Graphic Processing Units (GPUs), which leads to more redundant calculations. Therefore, how to efficiently handle sparse matrix operations in GNNs has become a key issue.

[0004] In GNN, mixed precision quantization is a key method to improve computational efficiency. This quantization method takes advantage of the uneven characteristics of the graph structure and adaptively allocates quantization precision according to the different characteristics of the nodes and the requirements of the network layers, thereby reducing the requirements for memory and computing resources while ensuring the performance of GNN. For example, the A2Q algorithm is a typical mixed precision quantization method. The memory required by the quantized model is used as a regularization term in the loss function, so that the quantization precision can be adaptively adjusted during the model training process to learn the appropriate quantization precision. For example, nodes with high in-degree may have a greater impact on the graph structure, so they will be given higher precision. While ensuring the high precision of key nodes and layers, mixed precision quantization accelerates the overall calculation by reducing the precision of other parts. Through flexible precision allocation, the resource utilization of the model is improved.

[0005] GPU is a widely used hardware accelerator in deep learning. Its highly parallel computing capability enables it to effectively accelerate large-scale matrix operations. In order to further improve the performance of matrix operations in deep learning, NVIDIA introduced Tensor Core in its Volta architecture and later GPUs. Tensor Core is a hardware unit specially designed for matrix multiplication. It can perform matrix multiplication and accumulation (MMA) operations of different scales such as 16x16x16 or 16x8x16, and different precisions such as FP16, INT8 and INT1, greatly improving the throughput of matrix operations.

[0006] Tensor Core is optimized for common matrix operations in deep learning, and can provide very high computing throughput, especially when processing dense matrices. However, the hardware characteristics of Tensor Core are mainly aimed at dense matrix calculations. In sparse matrix multiplication, directly using Tensor Core may not be able to realize its full potential. Fixed-size computing blocks may lead to waste of video memory and computing resources, which makes how to effectively use sparse matrices on Tensor Core an important research direction.

[0007] The Tensor Core unit only supports specific structured sparsity and does not support unstructured sparsity in GNN. In order to give full play to the performance advantages of Tensor Core when processing sparse matrices, the compression and rearrangement technology of sparse matrices is the key. Among the traditional matrix compression technologies, common formats include BCSR (Block Compressed Row Storage) and SR-BCRS (Segmented Row Block Compressed Row Storage), which reduce memory storage requirements and speed up operations by compressing zero elements in sparse matrices. However, the non-zero blocks of sparse matrices compressed using only this format still contain a large number of zero elements, resulting in unnecessary memory access and calculations.

[0008] In addition, Tensor Core only supports fixed-precision operations such as 1-bit, 4-bit, and 8-bit, while mixed-precision calculations require flexible adjustment of the precision of different nodes. Therefore, how to accelerate mixed-precision calculations on Tensor Core is a challenge. Summary of the invention

[0009] In view of the above-mentioned problems, the purpose of the present invention is to provide a graph neural network computing acceleration method, system, medium, and electronic device based on Tensor Core, which can effectively improve the computing performance and efficiency of the graph neural network on Tensor Core.

[0010] In a first aspect, the present invention provides a graph neural network computing acceleration method based on Tensor Core, the method comprising the following steps: dividing a sparse matrix into multiple sub-matrix blocks, the size of each sub-matrix block matching the computing granularity of the matrix operation unit of the Tensor Core; rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skipping the calculation of the all-zero sub-block matrix in the Tensor Core.

[0011] In an implementation of the first aspect, rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skipping the calculation of the all-zero sub-block matrix in the Tensor Core includes the following steps:

[0012] Set a threshold;

[0013] When the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, setting the sub-matrix block as a target sub-matrix block;

[0014] Constructing a horizontal row of sub-matrix blocks into a horizontal sub-matrix group;

[0015] Obtain the number of common neighbors of the row where the all-zero row in the target sub-matrix block is located with all the horizontal sub-matrix groups in the vertical direction;

[0016] Move the row where the all-zero row is located to the row where the all-zero row of the horizontal sub-matrix group with the largest number of common neighbors is located, so as to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0017] In an implementation of the first aspect, the method further includes performing compression acceleration processing on the feature matrix; wherein the compression acceleration calculation on the feature matrix includes the following steps:

[0018] For the feature matrix, bit splitting is performed for maximum precision to obtain a split matrix, and a quantization mask is generated for each split matrix, where each bit in the quantization mask indicates whether a corresponding element in the split matrix is ​​quantized to a preset precision;

[0019] Performing a bitwise OR operation on the quantization mask at a 32-bit granularity to generate a compression mask;

[0020] Data calculation and access are performed only for the elements whose compression mask is not 0.

[0021] In an implementation of the first aspect, the method further includes performing compression and accelerated calculation on the left matrix and the right matrix; the compression and accelerated calculation on the left matrix and the right matrix includes the following steps:

[0022] Divide the sub-matrix block into a plurality of sub-matrix units according to a granularity of 32 bits;

[0023] A compression mask is set for each sub-matrix unit; wherein when all elements of the sub-matrix unit are 0, the compression mask is 0, otherwise the compression mask is 1;

[0024] Performing a bitwise AND operation on the compression codes of the left matrix and the right matrix to obtain a result mask;

[0025] When performing a multiplication operation of the left matrix and the right matrix, the sub-matrix units whose result mask is 0 are skipped.

[0026] In an implementation manner of the first aspect, compression masks of the left matrix and the right matrix are stored in a shared register.

[0027] In an implementation of the first aspect, the method further includes, when the result mask of the left matrix and the right matrix contains four 1s, splicing the sub-matrix units corresponding to the result mask into a sub-matrix combination unit;

[0028] Calculate the prefix sums corresponding to the result masks of the left matrix and the right matrix respectively, and use the prefix sums as the indexes of the submatrix combination units;

[0029] Corresponding data of the left matrix and the right matrix are obtained based on the index, and the operation of the sub-matrix combination unit of the left matrix and the right matrix is ​​completed.

[0030] In an implementation manner of the first aspect, corresponding data of the left matrix and the right matrix are stored in a shared register.

[0031] In a second aspect, the present invention provides a graph neural network computing acceleration system based on Tensor Core, the system comprising a partitioning module and a rearrangement module;

[0032] The partitioning module is used to divide the sparse matrix into multiple sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core;

[0033] The rearrangement module is used to rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0034] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned Tensor Core-based graph neural network computing acceleration method.

[0035] In a fourth aspect, the present invention provides an electronic device, comprising: a processor and a memory;

[0036] The memory is used to store computer programs;

[0037] The processor is used to execute the computer program stored in the memory so that the electronic device performs the above-mentioned Tensor Core-based graph neural network calculation acceleration method.

[0038] As described above, the Tensor Core-based graph neural network computing acceleration method, system, medium, and electronic device of the present invention have the following beneficial effects:

[0039] (1) By rearranging the non-zero elements of the sparse matrix, it adapts to the hardware characteristics of the Tensor Core for efficient parallel computing, thereby effectively improving the compression rate of the sparse matrix, increasing the effective computing density, and improving the resource utilization of the Tensor Core;

[0040] (2) Through the dynamic matrix compression format for Tensor Cores, it can adapt to changes in matrix sparsity in real time and achieve an effective combination of compression and computation;

[0041] (3) Ability to perform mixed precision operations on Tensor Cores;

[0042] (4) Effectively improved the computing performance and efficiency of graph neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Shown is a flow chart of a Tensor Core-based graph neural network computing acceleration method in one embodiment of the present invention;

[0044] Figure 2 It is a schematic diagram showing the rearrangement of a sparse matrix in one embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram showing bit splitting in mixed precision quantization in one embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram showing a quantization mask in one embodiment of the present invention;

[0047] Figure 5 Shown is a schematic diagram of dynamic compression in an embodiment of the present invention;

[0048] Figure 6 Shown is a schematic diagram of the structure of a graph neural network computing acceleration system based on Tensor Core in one embodiment of the present invention;

[0049] Figure 7 It is a schematic structural diagram of an electronic device of the present invention in one embodiment. DETAILED DESCRIPTION

[0050] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0051] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0052] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0053] like Figure 1 As shown, in one embodiment, the graph neural network computing acceleration method based on Tensor Core of the present invention includes steps S1 and S2.

[0054] Step S1: Divide the sparse matrix into multiple sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core.

[0055] Specifically, in the calculation of the Tensor Core, the sparse matrix is ​​divided into multiple sub-matrix blocks of fixed size. The size of each sub-matrix block matches the calculation granularity of the matrix operation unit (such as 16x8 or 16x16) of the Tensor Core.

[0056] Step S2: rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0057] Specifically, in order to fully utilize the parallel computing capability of the GPU Tensor Core, the present invention rearranges the all-zero rows in the sub-block matrix to construct an all-zero sub-block. For the all-zero sub-block, calculation and memory access can be omitted, thereby improving the computational efficiency of the graph neural network.

[0058] In one embodiment, rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skipping the calculation of the all-zero sub-block matrix in the Tensor Core includes the following steps:

[0059] 21) Set a threshold.

[0060] The threshold is related to the number of subsequent row rearrangements of the sub-matrix blocks, so the threshold cannot be too large, otherwise the complexity of the row rearrangement will be increased. Preferably, the threshold is 1.

[0061] 22) When the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, the sub-matrix block is set as a target sub-matrix block.

[0062] like Figure 2 As shown, taking the size of Tensor Block as 4x3 as an example, the threshold of each sub-matrix block is set to 1. Therefore, the sub-matrix blocks with all zero rows greater than 0 and less than or equal to 1 are searched. Figure 2 The 1-4 rows are one layer, and the 5-8 rows are another layer. The upper and lower layers include 6 sub-matrix blocks in total. The second sub-matrix block in the first layer and the third sub-matrix block in the second layer are the target sub-matrix blocks, and the all-zero rows are the 3rd row and the 7th row respectively.

[0063] 23) A horizontal row of sub-matrix blocks is constructed as a horizontal sub-matrix group.

[0064] in, Figure 2 The first horizontal sub-matrix group includes three sub-matrix blocks consisting of rows 1 to 4, and the second horizontal sub-matrix group includes three sub-matrix blocks consisting of rows 5 to 8.

[0065] 24) Obtain the number of common neighbors of the row where the all-zero row in the target sub-matrix block is located with all the horizontal sub-matrix groups in the vertical direction.

[0066] Among them, for the second sub-matrix block of the first layer, the third row is all zero rows, so the entire Figure 2 The third row of has one common neighbor with the first horizontal sub-matrix group and two common neighbors with the second horizontal sub-matrix group. The common neighbors refer to the number of non-zero elements in the same vertical direction as the target element.

[0067] 25) Move the row where the all-zero row is located to the row where the all-zero row of the horizontal sub-matrix group with the largest number of common neighbors is located, so as to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0068] Among them, since the number of common neighbors of the second horizontal sub-matrix group is 2, which is greater than the number of common neighbors of the first horizontal sub-matrix group, the third row is moved to the all-zero row (i.e., the seventh row) of the second horizontal sub-matrix group; accordingly, the seventh row is moved to the third row, thereby constructing the following Figure 2 The two all-zero matrices shown in the right figure. Therefore, the all-zero rows are distributed to sub-matrix blocks with a larger number of common neighbors in the above manner, thereby improving the density of the overall sub-matrix blocks and reducing sparsity. After rearrangement, a new sparse matrix layout is generated, skipping the all-zero blocks. In this compression scheme, the non-zero elements of the sparse matrix are rearranged into a block structure suitable for efficient parallel computing by Tensor Core, which not only reduces storage overhead but also improves the computing efficiency of Tensor Core. Therefore, through the above-mentioned sparse matrix block compression scheme based on rearrangement, the hardware acceleration capability of Tensor Core can be fully utilized while maintaining the storage efficiency of the sparse matrix, thereby improving the performance of sparse matrix multiplication in graph neural networks. In the calculation process of graph neural networks, especially in the processing scenario of large-scale sparse graph data, the present invention can significantly reduce the calculation time and improve the overall system performance.

[0069] In the calculation of graph neural networks, three matrices are mainly involved, namely the adjacency matrix A, the feature matrix X and the weight matrix W. In mixed precision quantization, the adjacency matrix A and the weight matrix W are generally quantized to fixed precision, while the feature matrix X is quantized to dynamic precision. Different nodes in different network layers will be quantized to different precisions. Due to the limited precision that Tensor Core can support, existing methods mainly use bit splitting to support extended precision calculations, that is, separating features of different bits into separate matrices. Since only some nodes are quantized to high precision, this bit splitting method will bring greater sparsity in mixed precision calculations, resulting in a large number of invalid calculations and memory accesses. For example Figure 3 In the feature matrix X, only one node is quantized to 3 bits, so the bit splitting method will result in a higher sparsity of the third bit. Considering that mixed precision calculations will introduce greater sparsity to the feature matrix of the graph neural network, and considering the changes in the sparsity of the sparse matrix during the graph neural network calculation process, the Tensor Core-based graph neural network calculation acceleration method of the present invention also includes compressing and accelerating the feature matrix to cope with the calculation requirements of different sparse matrices.

[0070] For the feature matrix X, bit splitting is performed for maximum precision to obtain split matrices, and a quantization mask is generated for each split matrix. Each bit in the quantization mask indicates whether the corresponding element in the split matrix is ​​quantized to a preset precision. The quantization mask is bitwise ORed at a 32-bit granularity to generate a compression mask to indicate whether the corresponding 32 bits of the feature matrix X are 0. Figure 4 As shown, for the feature matrix X quantized to 3 bits with the highest precision, three corresponding compress masks are generated. Data calculation and access are performed only for the elements whose compression masks are not 0.

[0071] The Tensor Core-based graph neural network computing acceleration method of the present invention also includes compressing and accelerating the left matrix and the right matrix. For example, the left matrix is ​​an adjacency matrix A, and the right matrix is ​​a feature matrix X.

[0072] In one embodiment, when the left matrix and the right matrix are compressed and accelerated, the sub-matrix block is divided into multiple sub-matrix units sub-block according to the granularity of 32 bits. A compression mask (compress mask) is set for each sub-matrix unit. When the elements of the sub-matrix unit are all 0, the compression mask is 0, otherwise the compression mask is 1. For the sub-matrix unit whose compression mask is 0, its memory access can be omitted. A bitwise AND operation is performed on the compression codes of the left matrix and the right matrix to obtain a result mask (result mask). When performing the multiplication operation of the left matrix and the right matrix, the sub-matrix unit whose result mask is 0 is skipped. Figure 5 As shown in FIG. 1 , each submatrix block is divided into 4 submatrix units, where the compression mask of the second bit of the left matrix and the third bit of the right matrix is ​​0, so the storage of the corresponding two submatrix units can be omitted. The second and third bits in the corresponding result mask are both 0, so the multiplication of the second and third submatrix units of the left matrix and the right matrix can be skipped.

[0073] In one embodiment, the compression masks of the left matrix and the right matrix are stored in a shared register (Uniform Register, UR). At the same time, each UR register can store a total of 32 bits of compression masks, and each bit corresponds to a sub-matrix unit sub-block.

[0074] In order to effectively organize the calculation of the sub-matrix unit sub-block, it is necessary to splice 4 valid sub-matrix units sub-block into a complete sub-matrix combination unit. For the result mask obtained by bitwise AND, when the result mask of the left matrix and the right matrix contains four 1s, the sub-matrix units corresponding to the result mask are spliced ​​into a sub-matrix combination unit. It should be noted that the four 1s can be four consecutive 1s or four discontinuous 1s (that is, 0 can also be included between two 1s). For the left matrix and the right matrix, the prefix sum corresponding to each result mask is calculated respectively, and the prefix sum is used as the index of the sub-matrix combination unit obtained respectively. Based on the index, the corresponding data of the left matrix and the right matrix are obtained, and the operation of the sub-matrix combination unit of the left matrix and the right matrix is ​​completed. Among them, the corresponding data of the left matrix and the right matrix are stored in a shared register to reduce the access to the global memory.

[0075] In the present invention, each sub-matrix combination unit corresponds to the operation of a warp. Each cooperative thread array (CTA) is responsible for calculating a sub-matrix combination unit of the result matrix. Each warp reads the corresponding data of the left matrix and the right matrix in the shared register according to the index of the calculated sub-matrix combination unit, and initializes the fragment of the Tensor Core. After calling the Tensor Core matrix multiplication performed by the wmma interface provided by CUDA, the corresponding result is obtained and the result is stored back to the corresponding position, where the corresponding rows and columns in the result matrix are equal to the rows of the left matrix and the columns of the right matrix, respectively.

[0076] The protection scope of the Tensor Core-based graph neural network computing acceleration method described in the embodiment of the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0077] An embodiment of the present invention also provides a graph neural network computing acceleration system based on Tensor Core, which can implement the graph neural network computing acceleration method based on Tensor Core described in the present invention. However, the implementation device of the graph neural network computing acceleration system based on Tensor Core described in the present invention includes but is not limited to the structure of the graph neural network computing acceleration system based on Tensor Core listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0078] like Figure 6As shown, in one embodiment, the graph neural network computing acceleration system based on Tensor Core of the present invention includes a partitioning module 61 and a rearrangement module 62.

[0079] The partitioning module 61 is used to partition the sparse matrix into a plurality of sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core.

[0080] The rearrangement module 62 is connected to the partitioning module 61 and is used to rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

[0081] Among them, the structure and principle of the division module 61 and the rearrangement module 62 correspond one to one with the above-mentioned graph neural network calculation acceleration method based on Tensor Core, so they will not be repeated here.

[0082] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0083] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0084] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0085] The embodiment of the present invention also provides a computer-readable storage medium. A person of ordinary skill in the art can understand that all or part of the steps in the Tensor Core-based graph neural network computing acceleration method of the above embodiment can be completed by instructing the processor through a program, and the program can be stored in a computer-readable storage medium, and the storage medium is a non-transitory medium, such as a random access memory, a read-only memory, a flash memory, a hard disk, a solid-state hard disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof. The above storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a digital video disc (DVD)), or a semiconductor medium (for example, a solid-state disk (SSD)), etc.

[0086] An embodiment of the present invention further provides an electronic device, which includes a processor and a memory.

[0087] The memory is used to store computer programs.

[0088] The memory includes: ROM, RAM, disk, USB flash drive, memory card or CD and other media that can store program codes.

[0089] The processor is connected to the memory and is used to execute the computer program stored in the memory so that the electronic device executes the above-mentioned Tensor Core-based graph neural network calculation acceleration method.

[0090] Preferably, the processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0091] like Figure 7 As shown, the electronic device of the present invention is in the form of a general computing device. The components of the electronic device may include but are not limited to: one or more processors or processing units 71, a memory 72, and a bus 73 connecting different system components (including the memory 72 and the processing unit 71).

[0092] Bus 73 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0093] Electronic devices typically include a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device, including volatile and non-volatile media, removable and non-removable media.

[0094] The memory 72 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 721 and / or cache memory 722. The electronic device may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 723 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 73 via one or more data medium interfaces. The memory 72 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0095] A program / utility 724 having a set (at least one) of program modules 7241 may be stored, for example, in the memory 72, such program modules 7241 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 7241 generally perform the functions and / or methods of the embodiments described herein.

[0096] The electronic device may also communicate with one or more external devices (e.g., keyboards, pointing devices, displays, etc.), one or more devices that enable a user to interact with the electronic device, and / or any device that enables the electronic device to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via input / output (I / O) interface 74. Furthermore, the electronic device may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via network adapter 75. Figure 7 As shown, the network adapter 75 communicates with other modules of the electronic device via the bus 73. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0097] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A graph neural network computing acceleration method based on Tensor Core, characterized by: The method comprises the following steps: Divide the sparse matrix into multiple sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the Tensor Core; The all-zero rows in the sub-block matrix are rearranged to construct an all-zero sub-block matrix in the sparse matrix and skip calculation of the all-zero sub-block matrix in the Tensor Core.

2. The method for accelerating graph neural network calculation based on Tensor Core according to claim 1, characterized in that: Rearranging the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skipping the calculation of the all-zero sub-block matrix in the Tensor Core includes the following steps: Set a threshold; When the number of all-zero rows in the sub-matrix block is greater than 0 and less than or equal to the threshold, setting the sub-matrix block as a target sub-matrix block; Constructing a horizontal row of sub-matrix blocks into a horizontal sub-matrix group; Obtain the number of common neighbors of the row where the all-zero row in the target sub-matrix block is located with all the horizontal sub-matrix groups in the vertical direction; Move the row where the all-zero row is located to the row where the all-zero row of the horizontal sub-matrix group with the largest number of common neighbors is located, so as to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

3. The method for accelerating graph neural network calculation based on Tensor Core according to claim 1, characterized in that: It also includes compression and acceleration processing of feature matrices; The compression and acceleration calculation of the feature matrix includes the following steps: For the feature matrix, bit splitting is performed for maximum precision to obtain a split matrix, and a quantization mask is generated for each split matrix, where each bit in the quantization mask indicates whether a corresponding element in the split matrix is ​​quantized to a preset precision; Performing a bitwise OR operation on the quantization mask at a 32-bit granularity to generate a compression mask; Data calculation and access are performed only for the elements whose compression mask is not 0.

4. The method for accelerating graph neural network calculation based on Tensor Core according to claim 1, characterized in that: It also includes compressing and accelerating the calculation of the left matrix and the right matrix; compressing and accelerating the calculation of the left matrix and the right matrix includes the following steps: Divide the sub-matrix block into a plurality of sub-matrix units according to a granularity of 32 bits; A compression mask is set for each sub-matrix unit; wherein when all elements of the sub-matrix unit are 0, the compression mask is 0, otherwise the compression mask is 1; Performing a bitwise AND operation on the compression codes of the left matrix and the right matrix to obtain a result mask; When performing a multiplication operation of the left matrix and the right matrix, the sub-matrix units whose result mask is 0 are skipped.

5. The method for accelerating graph neural network calculation based on Tensor Core according to claim 4, characterized in that: The compression masks of the left matrix and the right matrix are stored in a shared register.

6. The method for accelerating graph neural network calculation based on Tensor Core according to claim 4, characterized in that: It also includes, when the result mask of the left matrix and the right matrix contains four 1s, splicing the sub-matrix units corresponding to the result mask into a sub-matrix combination unit; Calculate the prefix sums corresponding to the result masks of the left matrix and the right matrix respectively, and use the prefix sums as the indexes of the submatrix combination units; Corresponding data of the left matrix and the right matrix are obtained based on the index, and the operation of the sub-matrix combination unit of the left matrix and the right matrix is ​​completed.

7. The method for accelerating graph neural network calculation based on Tensor Core according to claim 6, characterized in that: The corresponding data of the left matrix and the right matrix are stored in a shared register.

8. A graph neural network computing acceleration system based on Tensor Core, characterized by: The system includes a partitioning module and a rearrangement module; The partitioning module is used to divide the sparse matrix into multiple sub-matrix blocks, and the size of each sub-matrix block matches the calculation granularity of the matrix operation unit of the TensorCore; The rearrangement module is used to rearrange the all-zero rows in the sub-block matrix to construct an all-zero sub-block matrix in the sparse matrix and skip the calculation of the all-zero sub-block matrix in the Tensor Core.

9. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the graph neural network computing acceleration method based on Tensor Core described in any one of claims 1 to 7.

10. An electronic device, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the electronic device performs the graph neural network computing acceleration method based on Tensor Core described in any one of claims 1 to 7.

Images