A Sparse Sparse Matrix Multiplication Array with High DSP Resource Utilization for Graph Neural Networks Based on FPGA

Abstract

The present invention discloses a sparse matrix multiplication array with high DSP resource utilization rate for graph neural networks based on FPGA, including a preprocessing module for generating required functional configuration parameters, eigenvector matrix, and adjacency matrix according to the used GNN model and its dataset, and controlling the slice size to slice the eigenvector matrix and adjacency matrix; an eigenmatrix cache module for caching the valid values of the eigenvector matrix after slicing; an adjacency matrix cache module for caching the valid values of the adjacency matrix after slicing; a congestion mitigation array for transmitting the valid values of the adjacency matrix to the pairing array according to the congestion mitigation strategy; the pairing array obtains the valid values of the adjacency matrix transmitted by the congestion mitigation array and the valid values of the eigenvector matrix transmitted by the eigenmatrix cache module, and inputs the valid values of the adjacency matrix and the valid values of the eigenvector matrix into the multiplication array in a certain order; the multiplication array performs multiplication processing according to the received valid values of the adjacency matrix and the valid values of the eigenvector matrix, and outputs a partial aggregation result.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and more specifically, to a sparse matrix multiplication array device with high DSP resource utilization rate for graph neural networks based on FPGA. Background Art

[0002] In recent years, application scenarios such as e-commerce, knowledge graphs, or molecular biology have modeled objects with complex relationships as non-Euclidean data structures such as graphs or manifolds. This type of data structure is difficult to extract and analyze potential information through traditional neural networks. Graph neural networks have unparalleled advantages in processing this type of unstructured data and have been successfully deployed on platforms such as Alibaba's AliGraph, Euler platform, and Pontry platform.

[0003] In a graph, each vertex represents an object, and the vertex contains a feature vector, which is the attribute of the object. The edges between vertices represent the relationships between objects. Taking e-commerce as an example, if products are used as vertices, it is easy to have more than a million node counts, and the number of edges between products varies greatly in a graph but usually follows a power-law distribution. In different application scenarios, the characteristics of the features owned by each vertex are also very different, ranging from extremely sparse with a large vector dimension in the knowledge graph to medium dense with a medium vector dimension in the community forum graph. This irregularity poses a huge challenge to the deployment process of GNN.

[0004] Each layer of GNN inference mainly includes two stages: aggregation and combination. In the aggregation stage, each node will aggregate the feature vectors of its neighbors for update. Since the adjacency matrix between nodes is extremely sparse and the feature matrix is relatively sparse, the computational intensity involved is not high. However, the two designed matrices both basically satisfy the power-law distribution and have a large dimension, so a large number of irregular memory accesses are involved. In the combination stage, the aggregated edges and nodes are transformed through a combination function, and the weight matrix involved is dense, so dense matrix multiplication calculations are involved. Currently, existing GNN accelerators are mainly deployed on application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs). Compared with acceleration schemes on other devices, these GNN accelerators have higher computational efficiency and are more energy-efficient. However, due to the long development cycle of ASICs, their customized characteristics make them lack flexibility. For a fixed-scale accelerator, the performance utilization rate on different data sets in a fixed application scenario may vary greatly, making it difficult to overcome the data dependence on the input graph features and difficult to meet different performance goals in different application scenarios.

[0005] In the existing sparse matrix multiplication array designs for the GNN aggregation stage, data density is mainly increased by merging sliced data or removing zeros from sparse matrix slices, while reusing the input data as much as possible while maintaining data regularity. This results in complex data preprocessing and many ineffective operations during the calculation process, leading to low effective utilization of multipliers. Less than 25% of the multipliers used can perform effective operations per cycle on average. Summary of the Invention

[0006] To solve the problems of the deficiencies and defects in the above prior art, the present invention provides a sparse matrix multiplication array device with high DSP resource utilization for graph neural networks based on FPGA. It reduces the number of required multiplication units while ensuring the input data reuse rate in the GNN aggregation stage, improving the overall utilization rate of the multiplication units.

[0007] To achieve the above object of the present invention, the following technical solutions are adopted:

[0008] A sparse matrix multiplication array device with high DSP resource utilization for graph neural networks based on FPGA, which is applied to the GNN aggregation stage and uses the DSP resources on the FPGA for calculation. It includes a preprocessing module, an adjacency matrix cache module, a congestion mitigation array, a pairing array, a multiplication array, and a feature matrix cache module.

[0009] The preprocessing module is used to generate the required functional configuration parameters, feature vector matrix, and adjacency matrix according to the used GNN model and its dataset, and control the slice size to slice the feature vector matrix and adjacency matrix.

[0010] The feature matrix cache module is used to cache the valid values of the feature vector matrix after slicing.

[0011] The adjacency matrix cache module is used to cache the valid values of the adjacency matrix after slicing.

[0012] The congestion mitigation array transmits the valid values of the adjacency matrix to the pairing array according to the congestion mitigation strategy.

[0013] The pairing array obtains the valid values of the adjacency matrix transmitted by the congestion mitigation array and the valid values of the feature vector matrix transmitted by the feature matrix cache module, and inputs the valid values of the adjacency matrix and the feature vector matrix into the multiplication array in a certain order.

[0014] The multiplication array performs multiplication processing based on the received valid values of the adjacency matrix and the feature vector matrix, and outputs partial aggregation results.

[0015] Preferably, after the effective value of the adjacency matrix is processed by slicing, it is first COO-encoded and then input into the adjacency matrix cache module; the COO encoding represents the effective value of the adjacency matrix through its row, column, and value parts; the row, column, and value of the adjacency matrix are transmitted to the congestion mitigation array together;

[0016] In the feature matrix cache module, the columns of the feature vector matrix required by the pairing array are stored separately, and the rows and values of the feature vector matrix are stored and called separately in the multiplication array.

[0017] Preferably, the pairing array includes several pairing unit groups, and each pairing unit group includes several serially connected pairing units; the serially connected multiple pairing units are also connected to the multiplication units in the multiplication array through a pairing controller;

[0018] In the pairing unit, the pairing results are counted to obtain the congestion situation and whether there is a valid value; and the highest and lowest significant bits in each pairing result are searched by the binary search method, and it is judged whether there are two different valid values by comparing the obtained results; after setting the highest and lowest significant bits of the pairing result to zero, a wired-AND operation is performed to obtain whether there are more than two valid values.

[0019] In a specific embodiment, it further includes a pairing grouping controller and an array grouping controller;

[0020] The pairing grouping controller groups the pairing units in the pairing array according to different GNN models and the preprocessing of their data sets and the implementation of the multiplication units in the multiplication array, so that different parallel pairing units can calculate different slices of the adjacency matrix, and the serially connected pairing units can also distribute the output results to the multipliers of the corresponding multiplication units;

[0021] The array grouping controller groups the congestion mitigation array according to the grouping situation of the pairing grouping controller, so that data in different groups do not transfer and interfere with each other during the data transmission process.

[0022] Furthermore, the congestion mitigation array adopts a congestion mitigation array with a depth of log2(A efftp ) - 1 and a width of A efftp ; where the cache register in the pairing unit has a depth of 0, and the cache register is connected to the congestion unit in the congestion mitigation array with a depth of log2(A efftp ) - 2; and the congestion information is obtained in the direction from the congestion unit to the cache register, and the valid value information is obtained in the direction from the cache register to the congestion unit;

[0023] The positions are (Depth_m, Position_n)(Depth_m, Position_n ± 2 m ),

[0024] (Depth_m - 1, Position_n), (Depth_m - 1, Position_n ± 2 m ) of the four blocking units are grouped together. Among the two blocking units with shallower depths in the same group, they are freely transferred to the two blocking units with deeper depths according to the blocking mitigation strategy;

[0025] Meanwhile, for a blocking unit (Depth_m, Position_n) at the same depth, at the end of each clock cycle, it transfers the content of its buffer register to another blocking unit (Depth_m, Position_n + 2 m+1 ), so that the valid values of any blocking unit can be transferred to any paired module, mitigating the blockage to the greatest extent.

[0026] Furthermore, when the adjacency matrix cache inputs the valid values of the adjacency matrix into the blocking mitigation array, without disturbing the front - back order of the input data, it inputs the valid values of the adjacency matrix into the idle blocking mitigation unit closest to the paired unit at the same position;

[0027] To improve the efficiency of inputting the valid values of switching features and film changing, when the valid values of the adjacency matrix are input into the blocking mitigation array, the feature value number is carried to index the input of the valid values of the feature vector matrix of the highest different film or different batch, so that blockage occurs when there are valid values in the blocking cache array closest to the adjacency matrix cache module, or when a piece of the adjacency matrix marked as A_sign = x is traversed and there is data marked as x + 1 in the blocking mitigation array.

[0028] Furthermore, the multiplication unit, after receiving the output of the paired unit, first caches the valid values of the adjacency matrix. At this time, it indexes the corresponding feature register according to the feature value number and uses the highest and lowest valid value positions of the pairing result as addresses for indexing to obtain the required valid values of the feature vector matrix; in the next cycle, after multiplying with the valid values of the adjacency matrix, it combines the columns of the valid values of the adjacency matrix and the rows of the valid values of the feature vector matrix into a partially aggregated result valid value in COO encoding for delivery to the subsequent combination module.

[0029] Preferably, the valid values of the adjacency matrix are stored in the adjacency matrix cache module in the order of first the column coordinates from smallest to largest, and then the row coordinates from smallest to largest;

[0030] The eigenvector matrix outputs new valid values after traversing each small adjacent matrix. According to the characteristics of the dataset, if the number of valid values within a slice is less than the set threshold, the slice size is adjusted to minimize the difference between the number of valid values in each slice and the set threshold. If the number of valid values within a slice is greater than the set threshold, the number of input valid values is adjusted according to the size of the set threshold to minimize the change in the amount of data input per cycle for the same slice.

[0031] Further, when the number of valid values of the adjacent matrix input per cycle is A efftp , and when allocating DSP perA multiplication units to each valid value of the adjacent matrix, if the utilization rate of the multiplication units is full, the time to complete the single-layer aggregation operation is:

[0032]

[0033] where Agg_opera_num is the total number of pairings in the single-layer aggregation stage, Clk aggregration represents the number of clock cycles required for a valid adjacent value from input to output, and clk_period represents the time required for each clock cycle.

[0034] Furthermore, since the total number of pairings in the single-layer aggregation stage is difficult to obtain directly, the minimum clock cycle Clk compare required for the single-layer pairing operation is calculated to estimate the minimum time of the single-layer aggregation operation, as follows:

[0035] In the pairing array, when the number of valid values of the eigenvector matrix input each time is F efftp , for each group of A efftp valid values of the adjacent matrix and F efftp valid values of the eigenvector for pairing, it takes at least one cycle:

[0036] Let: ceil(x) be the ceiling function

[0037] When the vertex dimension size, row slice size, and column slice size in the adjacent matrix are A_size, A_rowsize, and A_colsize respectively, we get:

[0038] The number of row slices in the adjacent matrix:

[0039] The number of column slices in the adjacent matrix:

[0040] When the eigenvector dimension size, row slice size, and column slice size in the eigenvector matrix are Fea_size, Fea_rowsize, and Fea_colsize respectively, we get:

[0041] Fea_rowsize = A_colsize

[0042] Number of column slices in the eigenvector matrix:

[0043] If the width of the adjacency matrix cache can store A efftp valid values, and the number of valid values of the M-th slice of the adjacency matrix is A_effnum M , then its depth is:

[0044]

[0045] If the width of the eigenvector matrix cache can store F efftp valid values, and the number of valid values of the N-th slice of the eigenvector matrix is Fea_effnum N , then its depth is:

[0046]

[0047] Therefore, the number of clock cycles Clk required for the single-layer pairing operation compare is:

[0048]

[0049] The time for the single-layer aggregation operation is at least:

[0050] T aggregration_min =(Clk compare + Clk aggregration ) × clk_period.

[0051] The beneficial effects of the present invention are as follows:

[0052] 1) The present invention makes full use of the characteristics of the extremely sparse adjacency matrix and the multiplication associativity of the graph neural network. By changing the data slice size, traversal order, and the amount of data output per cycle, using a pairing array with a congestion mitigation strategy and a supporting congestion mitigation array, the number of valid values input to the multiplication array is maximized, and the utilization rate of the multiplication array is maximized. Compared with other solutions, it requires fewer multiplier numbers for the same amount of computation.

[0053] 2) The memory access of the present invention is regular. Each time the valid values of the adjacency matrix and the eigenvector matrix read from the adjacency matrix cache module and the eigenvector matrix cache module make full use of the bandwidth and are traversed in order. This also makes the present invention have universality on different data sets, and can output the results required by the combination module according to the slice size without complex data preprocessing such as vertex rearrangement. Description of the Drawings

[0054] Figure 1 This is the principle block diagram of the sparse matrix multiplication array device of the present invention.

[0055] Figure 2 This is the module diagram of the congestion mitigation unit of the present invention.

[0056] Figure 3 This is the module diagram of the pairing unit of the present invention.

[0057] Figure 4 This is the schematic diagram of the congestion mitigation array of the present invention.

[0058] Figure 5 This is the module diagram of the multiplication unit of the present invention.

[0059] Figure 6 This is the performance comparison of the present invention with other accelerator aggregation modules. Detailed implementation mode

[0060] The following describes the present invention in detail in conjunction with the accompanying drawings and specific implementation modes.

[0061] Example 1

[0062] As Figure 1 shown, a sparse matrix multiplication array device with high DSP resource utilization for graph neural networks based on FPGA is applied to the GNN aggregation stage. It uses the DSP resources on the FPGA for calculation, including a preprocessing module, an adjacency matrix cache module, a congestion mitigation array, a pairing array, a multiplication array, a feature matrix cache module,

[0063] The preprocessing module is used to generate the required functional configuration parameters, feature vector matrix, and adjacency matrix according to the used GNN model and its dataset, and control the slice size to slice the feature vector matrix and adjacency matrix;

[0064] The feature matrix cache module is used to cache the valid values of the feature vector matrix after slicing;

[0065] The adjacency matrix cache module is used to cache the valid values of the adjacency matrix after slicing;

[0066] The congestion mitigation array transfers the valid values of the adjacency matrix to the pairing array according to the congestion mitigation strategy;

[0067] The pairing array obtains the valid values of the adjacency matrix transferred by the congestion mitigation array and the valid values of the feature vector matrix transferred by the feature matrix cache module, and inputs the valid values of the adjacency matrix and the feature vector matrix into the multiplication array in a certain order;

[0068] The multiplication array performs multiplication processing based on the received valid values of the adjacency matrix and the eigenvector matrix, and outputs partial aggregation results.

[0069] In a specific embodiment, after the valid values of the adjacency matrix are processed by slicing, they are first COO-encoded and then input into the adjacency matrix cache module; the COO encoding represents the valid values of the adjacency matrix through its three parts: row, column, and value; the rows, columns, and values of the adjacency matrix are transmitted to the congestion mitigation array together.

[0070] In the eigenmatrix cache module, the columns of the eigenvector matrix required by the pairing array are stored separately, and the rows and values of the eigenvector matrix are stored and called separately in the multiplication array.

[0071] In a specific embodiment, the pairing array includes several pairing unit groups, and each pairing unit group includes several serially connected pairing units; the serially connected multiple pairing units are also connected to the multiplication units in the multiplication array through a pairing controller.

[0072] In the pairing unit, the pairing results are counted to obtain the congestion situation and whether there are valid values; the highest and lowest significant bits in each pairing result are searched by the binary search method, and whether there are two different valid values is judged by comparing the obtained results; after the highest and lowest significant bits of the pairing result are set to zero, a wired-AND operation is performed to obtain whether there are more than two valid values.

[0073] In a specific embodiment, each pairing unit can receive two valid values simultaneously and output two pairing results. If there is a group of pairing result valid values missing in the pairing results, the two output pairing results are the highest and lowest two pairing results of the same input valid value.

[0074] In this embodiment, counting the pairing results only needs to know whether there are four situations in total: 0, 1, 2, or more values. Whether there are valid values is obtained by performing a wired-AND operation on the pairing results. The highest and lowest significant bits of the pairing results are searched by the binary search method, and whether there are two different valid values is judged by comparing the obtained results. After the highest and lowest significant bits of the pairing result are set to zero, a wired-AND operation is performed to obtain whether there are more than two valid values.

[0075] In a specific embodiment, it further includes a pairing grouping controller and an array grouping controller.

[0076] The pairing grouping controller groups the pairing units in the pairing array according to different GNN models and the preprocessing of their data sets and the implementation of the multiplication units in the multiplication array, so that different parallel pairing units can calculate different slices of the adjacency matrix, and the serially connected pairing units can also distribute the output results to the multipliers of the corresponding multiplication units.

[0077] The described array grouping controller groups the congestion mitigation array according to the grouping situation of the pairing grouping controller, so that data in different groups do not transfer and interfere with each other during the data transmission process.

[0078] During preprocessing, the slice size will be controlled to slice the feature vector matrix and the adjacency matrix.

[0079] To cooperate with the subsequent combination stage processing, this embodiment will slice the data into smaller pieces and transmit them to different paired arrays in parallel. The parallel paired arrays are divided into cases such as (32, 32), or even (16, 16, 16, 16), etc., so that different groups do not interfere with each other and perform the calculation of small piece aggregation work simultaneously.

[0080] Or in the same group of series-connected pairing units, when multiple multipliers are used in the multiplication unit, the pairing units can also be grouped. At this time, a pairing grouping controller is required.

[0081] At this time, the congestion mitigation array also needs to be changed according to the subsequent pairing grouping, so that data between different groups will not transfer to each other during the transfer of congestion mitigation. At this time, an array grouping controller is required.

[0082] In a specific embodiment, the described congestion mitigation array has a depth of log2(A efftp ) - 1 and a width of A efftp ; where the cache register in the pairing unit has a depth of 0, and the cache register is connected to the congestion unit in the congestion mitigation array with a depth of log2(A efftp ) - 2; and the congestion information is obtained in the direction from the congestion unit to the cache register, and the valid value information is obtained in the direction from the cache register to the congestion unit.

[0083] Four congestion units at positions (Depth_m, Position_n) (Depth_m, Position_n ± 2 m ),

[0084] (Depth_m - 1, Position_n), (Depth_m - 1, Position_n ± 2 m ) are grouped into a set. In the same set, the two congestion units with shallower depths freely transfer to the two congestion units with deeper depths according to the congestion mitigation strategy.

[0085] At the same time, for one congestion unit (Depth_m, Position_n) at the same depth, at the end of each clock cycle, the content of its cache register is transferred to another congestion unit (Depth_m, Position_n + 2m+1 ) so that the valid values of any blocked unit can be transmitted to any paired module, alleviating the blockage to the greatest extent.

[0086] In a specific embodiment, when the adjacency matrix cache module inputs the valid values of the adjacency matrix into the blockage alleviation array, without disrupting the order of the input data before and after, the valid values of the adjacency matrix are input into the idle blockage alleviation unit closest to the paired unit at the same position;

[0087] To improve the efficiency of inputting the valid values of switching features and film changing, when the valid values of the adjacency matrix are input into the blockage alleviation array, the eigenvalue number is carried to index the input of the valid values of the eigenvector matrix of the highest different film or different batch, so that blockage occurs only when there are valid values in the blockage cache array closest to the adjacency matrix cache module, or when traversing an adjacency matrix marked as A_sign = x and there is data marked as x + 1 in the blockage alleviation array.

[0088] In this embodiment, blockage needs to occur when both conditions of "traversing an adjacency matrix marked as A_sign = x" and "there is data marked as x + 1 in the blockage alleviation array" are satisfied simultaneously. For Figure 5 example, there is a "column sum and data cache of multiple groups of eigenvector matrices". When A_sign = x, it indexes the column sum and data cache of the x - th group of eigenvector matrices. When the data of x + 1 still exists, this data needs to index the data of the x + 1 - th group in the cache at this time, which causes the valid values of a new batch of eigenvector matrices not to be input to replace the data of the x + 1 part, resulting in blockage. Another example is when only 4 groups of feature valid values can be stored. When x = 3 and x + 1 = 4, 4 exceeds the index range of the 4 - th group, and at this time, the 0 - th group will be indexed. Next, x + 2 will index the 1 - th group, and so on.

[0089] In a specific embodiment, the valid values in the adjacency matrix slice may be very few, so a method with a small output quantity and a high update frequency is adopted for output. At the same time, the valid values of the adjacency matrix are stored in the adjacency matrix cache module in the order of column coordinates from small to large first, and then row coordinates from small to large;

[0090] The eigenvector matrix outputs new valid values after each small slice of the adjacency matrix is traversed; according to the characteristics of the data set, if the number of valid values in the slice is less than the set threshold, the slice size is adjusted to minimize the difference between the valid values of each slice and the set threshold; if the number of valid values in the slice is greater than the set threshold, the number of input valid values is adjusted according to the size of the set threshold to minimize the change amount of the data volume input in each cycle of the same slice.

[0091] In a specific embodiment, after receiving the output of the pairing unit, the multiplication unit first caches the effective values of the adjacency matrix. At this time, the corresponding eigenregister is indexed according to the eigenvalue number, and the highest and lowest effective value positions of the pairing result are used as addresses for indexing to obtain the effective values of the required eigenvector matrix. In the next cycle, after multiplying with the effective values of the adjacency matrix, the column of the adjacency matrix effective value and the row of the eigenvector matrix effective value are combined into a partial aggregation result effective value in COO encoding for delivery to the subsequent combination module.

[0092] Embodiment 2

[0093] Based on the sparse matrix multiplication array device with high DSP resource utilization of the graph neural network based on FPGA described in Embodiment 1, the sparse matrix multiplication array device is a computing array dedicated to the aggregation stage of GNN. It utilizes the characteristics that the multiplication associative law is satisfied during the calculation process of part of the GNN model, the input data dimension is large, extremely sparse and the distribution is a power-law distribution. The effective values of the adjacency matrix and the eigenvector matrix are input into the multiplication unit in a certain order, and after passing through a congestion mitigation array, they are input into a point-to-point multiplication array, and finally the aggregation partial result required by the subsequent combination module is obtained.

[0094] Since the point-to-point multiplication array can be understood as: when the multiplier is the data d0 with coordinates (col0, row0) and the multiplicand is the data d1 with coordinates (col1, row1), when col0 is equal to row1, the value of d0 * d1 is accumulated into the coordinates (row0, col1) of the result matrix. Therefore, by controlling the slice size and cooperating with the subsequent combination module to change the ranges of row0 and col1, the size of the output matrix can be determined, and the ranges of col0 and row1 can be minimized as much as possible to improve the reuse rate of the input data.

[0095] For the multiplication in the aggregation stage, the multiplier is the adjacency matrix, whose data characteristics are that the row and column coordinate dimensions are consistent, the dimension is large, it is extremely sparse and satisfies the power-law distribution. The multiplicand is the eigenvector matrix, whose data characteristics change according to the data set. The row coordinate dimension is large, and the column coordinate is basically less than the address depth of a single 36K BRAM on the FPGA, which is 1024. The data sparsity of the eigenvector matrix can range from extremely sparse to completely dense, and when sparse, the data follows a power-law distribution, which may lead to a very low pairing success rate between the column coordinates of the adjacency matrix and the row coordinates of the eigenvector matrix. Since there may be very few valid values in the adjacency matrix slices, an output method with a small number of outputs and a high update frequency is adopted, and the valid values are traversed in the order of column coordinates from small to large first, and then row coordinates from small to large. The eigenvector matrix only outputs new valid values after each small piece of the adjacency matrix is traversed, and in order to increase the number of successful pairings between the column coordinates of the adjacency matrix and the row coordinates of the eigenvector matrix, the number of valid feature values updated each time can be as large as possible.

[0096] In this embodiment, when the number of valid values of the adjacency matrix input per cycle is A efftp , and when allocating DSP perA multiplication units to each valid value of the adjacency matrix, if the utilization rate of the multiplication units is full, the time required to complete the single-layer aggregation operation is:

[0097]

[0098] where Agg_opera_num is the total number of pairings in the single-layer aggregation stage, Clk aggregration represents the number of clock cycles required for a valid adjacency value to be input and output, and clk_period represents the time required for each clock cycle.

[0099] In a specific embodiment, since the total number of pairings in the single-layer aggregation stage is difficult to obtain directly, the minimum clock cycle Clk compare required for the single-layer pairing operation is calculated to estimate the minimum time of the single-layer aggregation operation, as follows:

[0100] In the pairing array, when the number of valid values of the eigenvector matrix input each time is F efftp , each group of A efftp valid values of the adjacency matrix and F efftp valid feature values need to spend at least one cycle when paired:

[0101] Let: ceil(x) be the ceiling function

[0102] When the vertex dimension size, row slice size, and column slice size in the adjacency matrix are A_size, A_rowsize, and A_colsize respectively, we get:

[0103] Number of row slices in the adjacency matrix:

[0104] Number of column slices in the adjacency matrix:

[0105] When the feature dimension size, row slice size, and column slice size in the eigenvector matrix are Fea_size, Fea_rowsize, and Fea_colsize respectively, we get:

[0106] Fea_rowsize = A_colsize

[0107] Number of column slices in the eigenvector matrix:

[0108] If the width of the adjacency matrix cache can store efftp A valid values, and the number of valid values in the Mth slice of the adjacency matrix is A_effnum M , then its depth is:

[0109]

[0110] If the width of the eigenvector matrix cache can store efftp F valid values, and the number of valid values in the Nth slice of the eigenvector matrix is Fea_effnum N , then its depth is:

[0111]

[0112] Therefore, the number of clock cycles Clk required for the single-layer pairing operation compare is:

[0113]

[0114] The single-layer aggregation operation time is at least:

[0115] T aggregration_min = (Clk compare + Clk aggregration ) × clk_period.

[0116] Embodiment 3

[0117] Based on the sparse matrix multiplication array device with high DSP resource utilization of the graph neural network based on FPGA described in Embodiment 1 and Embodiment 2, its structure is as Figure 1 shown. This embodiment will be specifically described for one embodiment as follows: The sparse matrix multiplication array device includes a preprocessing module, an adjacency matrix cache module, a congestion mitigation array, a pairing array, a multiplication array, and a feature matrix cache module.

[0118] First, preprocess the GNN model and the dataset, and generate the required functional configuration parameters according to the used GNN model and its dataset. Taking the Cora dataset of the GCN model as an example: the dimension of the feature vector matrix vector is 1433, the number of vertices is 2708, the number of valid values in the feature vector matrix is 49216, the number of edges is 13264, and the weight dimension is 16 or 128.

[0119] The FPGA used is the Xilinx Alveo U250 board of Xilinx. Each DSP resource on it can implement a 27bit×18bit multiplier and two adders, which is sufficient to implement the 16bit×16bit multiplication required in the graph neural network. Therefore, each multiplier described later refers to using the DSP resources on the FPGA for multiplication calculation. In a specific embodiment, multiple multipliers may be used in the multiplication unit described, but only one multiplier is used in the multiplication unit of this embodiment (as Figure 5 shown), and each multiplier uses one DSP resource.

[0120] Since the address depth of a 36K BRAM in the FPGA is 1024, in order for the subsequent combination module to index this dimension, the slice size Fea_colsize of this dimension is set to 1024. For the adjacency matrix, the vertex dimension can be represented by a 12bit number, and the data bit width is 16bit. In order to save storage space and make the best use of the bandwidth as much as possible, the COO coding method is adopted, and A_rowsize = 2 12 is taken, and A_colsize = 2 4 . At this time, each valid value of the adjacency matrix requires 12 + 4 + 16 = 32bit, and each valid value of the feature vector matrix requires 10 + 4 + 16 = 30bit.

[0121] In the combination stage, the weight dimension is at least 16. When 1024 multipliers are used in this combination stage, 1024÷16 = 64 partial aggregation results need to be output to the combination module per cycle. Therefore, at least 64 multipliers A efftp = 64 are required. Correspondingly, 64 BRAMs of 18K size are used as caches for the adjacency matrix. Since the bit width of this BRAM is 36bit and the depth is 512, 32768 valid values can be stored, and all the valid values of the cora adjacency matrix can be stored on-chip.

[0122] After testing, in a 250 MHz time constraint, a single pairing unit can achieve a maximum of 64 feature effective values and 1 pair of adjacent matrix effective value pairings. If a pairing unit with 32 feature effective values is used, through series connection, a maximum of 32 * 16 = 512 feature effective values and 1 pair of adjacent matrix effective value pairings can be achieved. Therefore, Fea efftp = 512. Here, 512 BRAMs with a size of 18K are used to store the feature vector matrix. This can ensure that at least one set of 512 feature vector effective values is delivered to the pairing unit every cycle while storing the entire feature vector matrix.

[0123] In summary, the following functional configuration parameters can be obtained from the two inputs of the GCN model and the Cora dataset: The three parameters of the slice are A_rowsize = 4096, A_colsize = 16, and Fea_colsize = 1024; the number of multipliers required for aggregation is 64, the width of the congestion mitigation array is 64, and the depth is log2 64 - 1 = 5. A total of 512 + 64 = 576 BRAMs with a size of 18K are required for the cache in the aggregation stage.

[0124] The COO encoding that appeared earlier is a common sparse matrix encoding method. It represents all the effective values of the sparse matrix through three parts: its row (col), column (row), and value (data). In this way, there is no need to store the invalid 0 data in the matrix, saving the required number of registers. In the sparse matrix multiplication array described in the present invention, the row of the feature vector required by the pairing unit is stored separately in the feature vector matrix cache, while the col and data are stored and called separately in the multiplication unit. However, all the contents of the adjacent matrix effective values must be transmitted together due to the congestion mitigation array.

[0125] The enable signal of the adjacent matrix cache is only valid when there are no effective values in the congestion mitigation array with the maximum depth. At this time, the effective values input from the cache are judged according to the congestion signal. If the unit with depth a is not empty, the effective value will be transmitted to the unit with depth a + 1 regardless of whether the unit with depth a - 1 has a value.

[0126] The working principle of the congestion mitigation unit in the congestion mitigation array is as Figure 2 shown, mainly divided into two parts: judging and transmitting the congestion situation from the pairing unit to the cache direction, and scheduling the transmission of the adjacent matrix effective values and congestion mitigation from the cache to the pairing unit direction. Suppose there is a congestion unit at the b-th from the top to the bottom with depth a. Divide the congestion units at the same depth into groups of every 2 a+1 adjacent ones. In the same group, it is also divided into the upper half of 2 a and the lower half of 2 aLet's assume that b belongs to the upper half.

[0127] The congestion judgment and transmission are related to all layers. This judgment requires the following four signals: whether its own register is empty, the b+2th register at the same depth, a Whether the register of the bth blocking unit in the a-1th layer is empty, whether the bth blocking unit in the a-1th layer is blocked, and whether the b+2th blocking unit in the a-1th layer is blocked. a Whether the blocking unit is blocked. There are three situations:

[0128] 1. The two blocked units in front are blocked at the same time, making it impossible to forward the data. At this time, the depth is b of a, b+2 a The two blocking units retain their own data, and determine their own blocking status based on whether they are empty and transmit them to the blocking unit in the a+1 layer.

[0129] 2. There is only one blocked unit in front. If only one of the two blocked units at depth a has data, it will be transmitted to the unblocked blocked unit. If both blocked units have data, the one at the same position will be transmitted first. The blocked unit on the other side is blocked, and the blocked signal will be transmitted to the a+1 layer.

[0130] 3. There is no blockage ahead. At this time, data can be transmitted forward and both input ports are not blocked.

[0131] After all the congestion units receive the congestion signal, the effective value transmission and congestion relief scheduling strategy are carried out. If the effective value transmission is consistent with the above congestion judgment, the effective value is transmitted to the same position b at the depth of a-1 as much as possible; if it cannot be transmitted, the position b+2 at the same depth is observed. a Can it be transmitted to the front? If the position is b+2 a If the register is empty while forward transmission is possible, the valid value of position b at depth a is also transferred to position b+2 at depth a-1. a When b belongs to the lower half of the above mentioned 2 a In the unit, b+2 a Changed to b-2 a That's it.

[0132] In addition, in order to prevent only the 2 a+1 The group where the congestion relief unit is located is densely congested. At the end of each cycle, the register values of all units at depth a and position b are transferred to the register at depth a and position b+2. a+1 Register. When b+2 a+1 >A efftp When it is transferred to position b+2 a+1 -A efftp in the position.

[0133] In the blockage mitigation array, b, b±2 at depth a a , b, b±2 at depth a - 1 a Four blockage mitigation units form a group to transmit blockage signals and data to each other, so as to Figure 3 Take the schematic diagram of the blockage mitigation array as an example. Assume that all units in the blockage mitigation array have values. When both input ports of paired unit 1 are blocked, the blockage signals will be transmitted in the directions of the thick large arrow and the thick medium arrow respectively. The rectangles filled with gray are one of the paths for migrating the register values at the same depth at the end of each cycle. In this way, it can be observed that without the same-depth migration, for all blockage mitigation units at depth a, under certain circumstances, they can be transmitted to 2 a different paired units in front. After adding the same-depth data transmission, they can finally be transmitted to any paired unit, enabling the paired unit to input the maximum number of valid values per cycle.

[0134] In this embodiment, the module diagram of the paired unit is as shown in Figure 4 As shown, each paired unit has two input ports a and b. When not blocked in the previous cycle, the register will compare with the corresponding feature register according to the input feature value number index to obtain the pairing result. Here, paired units paired with 32 feature valid values are used, so two groups of 32-bit pairing results will be generated. At this time, the highest and lowest significant bits of these two groups are obtained by the binary method. When both groups of pairing results have valid values, when the highest and lowest significant bits are the same, it means there is only one valid value, and at this time, it is directly output at the corresponding two ports. When there are valid values and the highest and lowest significant bits are different, it means there are at least two valid values. At this time, if there are more than two valid values at a and no valid values at b, the two output ports of the paired unit will output the highest and lowest valid values of a, so that the number of valid values output per cycle can be as close to A as possible efftp . When there are at least two valid values, the pairing results corresponding to these two significant bits will be set to zero, and the line AND operation is performed on the paired results after being set to zero to obtain the blockage situation of this pairing port. This blockage situation will affect the data transmission of the blockage mitigation array in the next cycle. In addition to the blockage situation, the output content of the paired unit also includes the feature value numbers of ports a and b, the positions of the highest and lowest valid values corresponding to the pairing result, and the valid values of the input adjacency matrix.

[0135] In this embodiment, the module diagram of the multiplication unit is as shown in Figure 5As shown, after receiving the output of the pairing unit, the valid values of the adjacency matrix are first cached. At this time, the corresponding eigenregister is indexed according to the eigenvalue number, and the highest and lowest valid value positions of the pairing result are used as addresses for indexing to obtain the required eigen valid values. In the next cycle, after multiplying with the adjacent valid values, the row of the adjacent valid values and the col of the eigen valid values are combined into a partial aggregation result valid value in COO encoding and sent to the subsequent combination module.

[0136] The comparison results of the acceleration performance of the present invention on multiple public datasets with other accelerators are as Figure 6 . The sparse matrix multiplication array device described in the present invention is applied to a graph neural network accelerator, realizing a graph neural network accelerator that aggregates first and then combines. Compared with other graph neural network accelerators that combine first and then aggregate, it has more versatility for other graph neural networks. And the entire graph neural network accelerator only requires 1024 + 64 = 1084 DSPs, which can be adapted to more mid-range FPGAs for graph neural network acceleration. As Figure 6 shown, compared with the accelerators of PyG, DGL, and Zhang et al., the performance has been significantly improved. Compared with BoostGCN, it uses 3840 DSPs on the FPGA, and the sparse matrix multiplication array device described in the invention has achieved better acceleration results on datasets with a large number of vertices such as Pubmed. And AWB is an accelerator dedicated to GCN, which uses 4096 DSPs and operates at a frequency of 330 MHz. The graph neural network accelerator can still achieve a multiplier utilization rate similar to it while pursuing the versatility of the graph neural network.

[0137] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, rather than limiting the implementation manners of the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A sparse matrix multiplication array device with high DSP resource utilization for graph neural networks based on FPGA, which is applied to the GNN aggregation stage and uses the DSP resources on the FPGA for calculation, and is characterized in that: It includes a preprocessing module, an adjacency matrix cache module, a congestion mitigation array, a pairing array, a multiplication array, and a feature matrix cache module; The preprocessing module is used to generate the required functional configuration parameters, feature vector matrix, and adjacency matrix according to the used GNN model and its dataset, and control the slice size to slice the feature vector matrix and adjacency matrix; The feature matrix cache module is used to cache the valid values of the feature vector matrix after slicing; The adjacency matrix cache module is used to cache the valid values of the adjacency matrix after slicing; The congestion mitigation array transfers the valid values of the adjacency matrix to the pairing array according to the congestion mitigation strategy; The pairing array obtains the valid values of the adjacency matrix transferred by the congestion mitigation array and the valid values of the feature vector matrix transferred by the feature matrix cache module, and inputs the valid values of the adjacency matrix and the valid values of the feature vector matrix into the multiplication array in a certain order; among them, the pairing array includes several pairing unit groups, and each pairing unit group includes several series-connected pairing units; the series-connected multiple pairing units are also connected to the multiplication units in the multiplication array through a pairing controller; The multiplication array performs multiplication processing according to the received valid values of the adjacency matrix and the feature vector matrix, and outputs a partial aggregation result; Among them, The described congestion mitigation array has a depth of log2(A efftp ) - 1 and a width of A efftp congestion mitigation array; wherein the cache register in the paired unit has a depth of 0, and the cache register is connected to the congestion unit in the congestion mitigation array with a depth of log2(A efftp ) - 2; and the congestion information is obtained and transmitted from the congestion unit to the cache register in one direction, and the valid value information is obtained and transmitted from the cache register to the congestion unit in the other direction; at positions (Depth_m,Position_n)(Depth_m,Position_n±2 m ) (Depth_m - 1, Position_n), (Depth_m - 1, Position_n ± 2 m )'s four blocked units are grouped together. Among the two blocked units with shallower depths in the same group, they are freely transferred to the two blocked units with deeper depths according to the blockage mitigation strategy; At the same time, a congestion unit (Depth_m, Position_n) at the same depth transfers the content of its cache register to another congestion unit at the end of each clock cycle; (Depth_m, Position_n + 2 m+1 ) so that the effective value of any blocked unit can be transmitted to any paired module, alleviating the blockage to the greatest extent.

2. The sparse matrix multiplication array device with high DSP resource utilization rate for graph neural network based on FPGA according to claim 1, characterized in that: After being sliced, the valid values of the adjacency matrix are first COO-encoded and then input into the adjacency matrix cache module; the COO encoding represents the valid values of the adjacency matrix through its row, column, and value parts; the row, column, and value of the adjacency matrix are transmitted to the congestion mitigation array together; In the feature matrix cache module, the columns of the feature vector matrix required by the pairing array are stored separately, and the rows and values of the feature vector matrix are stored and called separately in the multiplication array.

3. The sparse matrix multiplication array device with high DSP resource utilization rate for graph neural network based on FPGA according to claim 1, characterized in that: Count the pairing results in the pairing unit to obtain the congestion situation and whether there are valid values; And use the binary search method to search for the highest and lowest valid bits in each pairing result, and judge whether there are two different valid values by comparing the obtained results; After setting the highest and lowest valid bits of the pairing result to zero, perform a wired AND operation to obtain whether there are more than two valid values.

4. The sparse matrix multiplication array device with high DSP resource utilization of the graph neural network based on FPGA according to claim 3, characterized in that: It also includes a pairing grouping controller and an array grouping controller; The described pairing and grouping controller groups the pairing units in the pairing array according to different GNN models and the preprocessing of their datasets and the implementation of the multiplication units in the multiplication array, so that different parallel pairing units can calculate different slices of the adjacency matrix, and the series-connected pairing units can also distribute the output results to the multipliers of the corresponding multiplication units; The described array grouping controller groups the congestion mitigation array according to the grouping situation of the pairing and grouping controller, so that data in different groups do not transfer and interfere with each other during the data transmission process.

5. The sparse matrix multiplication array device with high DSP resource utilization rate of the graph neural network based on FPGA according to claim 3, characterized in that: When the adjacency matrix valid values are input into the congestion mitigation array by the adjacency matrix cache module, without disrupting the order of the input data before and after, the adjacency matrix valid values are input into the idle congestion mitigation unit closest to the pairing unit at the same position; When the adjacency matrix valid values are input into the congestion mitigation array, they carry eigenvalue numbers, which are used to index the input of the valid values of the eigenvector matrix of the highest different slices or different batches, so that congestion occurs when there are valid values in the congestion cache array closest to the adjacency matrix cache module, or when a slice of the adjacency matrix marked as A_sign = x is traversed and there is data marked as x + 1 in the congestion mitigation array.

6. The sparse matrix multiplication array device with high DSP resource utilization rate for graph neural network based on FPGA according to claim 5, characterized in that: The described multiplication unit, after receiving the output of the pairing unit, first caches the adjacency matrix valid values. At this time, it indexes the corresponding eigen registers according to the eigenvalue number and uses the highest and lowest valid value positions of the pairing result as addresses for indexing to obtain the required eigenvector matrix valid values; in the next cycle, after multiplying with the adjacency matrix valid values, it combines the columns of the adjacency matrix valid values and the rows of the eigenvector matrix valid values into a partial aggregation result valid value in COO encoding for delivery to the subsequent combination module.

7. The sparse matrix multiplication array device with high DSP resource utilization rate of the graph neural network based on FPGA according to claim 1, characterized in that: The adjacency matrix valid values are stored in the adjacency matrix cache module in the order of column coordinates from small to large first, and then row coordinates from small to large; The eigenvector matrix outputs new valid values after each small slice of the adjacency matrix is traversed; according to the characteristics of the dataset, if the number of valid values within a slice is less than the set threshold, the slice size is adjusted to minimize the difference between the number of valid values in each slice and the set threshold; if the number of valid values within a slice is greater than the set threshold, the number of input valid values is adjusted according to the size of the set threshold to minimize the change in the amount of data input per cycle in the same slice.

8. The sparse matrix multiplication array device with high DSP resource utilization for graph neural networks based on FPGA according to any one of claims 2 to 6, characterized in that: When the number of effective values of the adjacency matrix input per cycle is A efftp , and DSP perA multiplication units are allocated to each effective value of the adjacency matrix, if the utilization rate of the multiplication units is full, the time required to complete the single-layer aggregation operation is as follows: Among them, Agg_opera_num is the total number of pairings in the single-layer aggregation stage, Clk aggregration represents the number of clock cycles required for an adjacent valid value to go from input to output, and clk_period represents the time required for each clock cycle.

9. The sparse matrix multiplication array device with high DSP resource utilization of the graph neural network based on FPGA according to claim 8, wherein: Since it is difficult to directly obtain the total number of pairings in the single-layer aggregation stage, the minimum clock cycle Clk required for the single-layer pairing operation is calculated compare to estimate the minimum time for the single-layer aggregation operation, as follows: In the pairing array, the number of valid values of the feature vector matrix input each time is F efftp When efftp the number of valid values of each group of A efftp is paired with F feature valid values, it takes at least one cycle for each pair: Let: ceil(x) be the ceiling function When the vertex dimension size, row slice size, and column slice size in the adjacency matrix are A_size, A_rowsize, and A_colsize respectively, we get: Number of row slices in the adjacency matrix: Number of column slices in the adjacency matrix: When the eigen dimension size, row slice size, and column slice size in the eigenvector matrix are Fea_size, Fea_rowsize, and Fea_colsize respectively, we get: Fea_rowsize = A_colsize Number of column slices in the eigenvector matrix: If the width of the adjacency matrix cache can store A efftp valid values, and the number of valid values of the M-th adjacency matrix is A_effnum M , then its depth is: If the width of the eigenvector matrix cache can store F efftp valid values, and the number of valid values of the eigenvector matrix of the Nth slice is Fea_effnum N , then its depth is: Therefore, the clock cycle Clk required for the single-layer pairing operation compare is as follows: The single-layer aggregation operation time is at least: T aggregration_min = (Clk compare + Clk aggregration ) × clk_period。

Images