A hardware description language-based graph convolutional neural network acceleration method

Abstract

The application relates to a hardware description language-based graph convolutional neural network acceleration method and belongs to the field of deep learning hardware acceleration. The method comprises the following steps: step one: a classical graph neural network model GCN is built and trained based on a Pytorch framework, and the model is used for classification, prediction and the like of graph data. Step two: parameters of the GCN model are extracted for quantization and conversion so as to be stored and calculated by FPGA. Step three: a GCN algorithm framework is designed by using a hardware description language, and the algorithm framework comprises a calculation module, a storage module and a cooperation module. Step four: the GCN algorithm framework is compiled and simulated by using a vivado platform. Step five: the verified GCN algorithm framework is deployed in FPGA to perform real-time inference on input graph data flow. Efficient processing of the graph convolutional neural network is realized, multifunctional function calculation is realized in a simple way, the model accuracy is ensured, and the real-time performance of graph data processing and the efficiency of model operation are ensured.

Description

Technical Field

[0001] This invention relates to the field of graph structure data processing acceleration, and more particularly to a graph convolutional neural network acceleration method based on a hardware description language. Background Technology

[0002] In recent years, with the rapid development of big data and artificial intelligence, machine learning technology has made revolutionary progress. In tasks such as computer vision, speech recognition, and natural language processing, commonly used machine learning algorithms are suitable for Euclidean data with defined dimensions and ordered arrangement. However, data in real-world scenarios is increasingly presented in the form of graphs, a complex non-Euclidean data representation. This complex non-Euclidean data representation includes the data itself and the dependencies between data. For example, social networks, protein molecular structures, chemical molecules, and customer data from e-commerce platforms can all be represented in graph form. The complexity of graph data places higher demands on the design and implementation of traditional machine learning algorithms. However, traditional deep learning models cannot handle graph-structured data well. Therefore, researchers have begun to focus on how to apply neural networks to graph-structured data and have proposed various graph neural network models. Against this backdrop, a large number of new machine learning algorithm models based on graph-structured data have emerged—Graph Neural Networks (GNNs).

[0003] Graph Neural Networks (GNNs) require extensive matrix computations and memory access operations, placing high demands on processor computing power and memory. However, current software implementations suffer from inefficiencies, making hardware acceleration solutions for GNNs a crucial area of ​​research. Currently available hardware acceleration solutions include GPUs, CPUs, ASICs, and FPGAs. The application of new GPUs significantly impacts the computational speed and energy efficiency of GNNs. However, GPUs face limitations when processing massive amounts of graph data, primarily insufficient memory scalability and high power consumption. CPUs offer high homogeneity and flexibility, but their performance (computing power) is the worst, making them unsuitable for scenarios requiring real-time graph data computation. Dedicated ASICs are custom-designed and can address performance and power consumption issues, but they lack flexibility, have long design cycles, and require specialized customization, resulting in very high costs for dedicated ASIC hardware acceleration solutions. In contrast, FPGAs achieve ideal levels in terms of flexibility, parallelism, homogeneity, power consumption, and cost, better adapting to network model upgrades and adjustments. Integrating these advantages, research on FPGA deployment and hardware acceleration solutions for GNNs is of great significance.

[0004] Currently, hardware acceleration solutions for graph neural network algorithms are implemented using integrated tools in C / C++, rather than hardware description languages. This results in incomplete hardware-software acceleration and an inability to meet real-time requirements. Therefore, deploying a high-precision, real-time graph convolutional neural network model on an FPGA platform has become a pressing issue. Summary of the Invention

[0005] This invention provides a graph convolutional neural network acceleration method based on hardware description language, which solves the problem of incomplete hardware and software acceleration in the current system. It ensures the real-time performance of graph data processing and the high efficiency of model operation while maintaining model accuracy.

[0006] The technical solution of this invention is: a method for accelerating graph convolutional neural networks based on a hardware description language, comprising the following steps:

[0007] (1) Design a three-layer graph convolutional neural network model GCN, with an outer classification layer to classify nodes in the input graph data. The objective function of the graph convolutional neural network layer is as follows:

[0008]

[0009] in, Let A be the adjacency matrix of an undirected graph with self-joins, and I be the adjacency matrix without self-joins. N It is the identity matrix. W is the degree matrix. (L) This is the trainable weight parameter matrix of the Lth layer graph convolutional neural network, where L = 0, 1, 2, σ(·) represents the activation function, using ReLU(·), H (L) H is the feature matrix in the Lth layer. (L+1) The feature matrix output by the Lth layer;

[0010] (2) During training, the parameters of each layer of the graph convolutional neural network are updated by backpropagation algorithm, with the cross-entropy loss function as the loss function, and the optimizer Adam is established with a learning rate of 0.01 to obtain the trained GCN model.

[0011]

[0012] in, This represents the label of sample i, with 1 for category c and 0 for others. Let N represent the probability that sample i is predicted to be of class c, N represent the total number of samples, Cross represent the loss value, and C represent the set of node classes, where c ∈ C.

[0013] (3) Quantize and transform the parameter matrix, normalized adjacency matrix and feature matrix extracted from the trained GCN model.

[0014] (4) Design a storage module using a hardware description language to store quantized and converted parameters and cache intermediate variables;

[0015] (5) Design the computing module and coordination module using a hardware description language;

[0016] (6) Create a project on the Vivado platform, integrate the contents designed in steps (3), (4) and (5) into the project, compile and simulate the integrated project to obtain the verified GCN model, deploy the verified GCN model in the FPGA, and process the input graph data stream in real time.

[0017] Specifically, in step (1), a three-layer graph convolutional neural network model GCN is built. The feature output of each layer is used as the feature input of the next layer, and the feature output of the last layer is used as the feature input of the classification layer, so as to finally achieve the classification of graph nodes.

[0018] Specifically, in step (3), the three matrices are quantized and transformed. Each element in these matrices is shifted left by 12 bits and then rounded down, which converts the elements in each matrix into integers. At this time, these matrices will have precision loss, but as long as the number of left shift bits is higher, the precision loss will be smaller and will have almost no impact on the output results. In addition, the number of left shift bits of the bias parameters of each network layer is determined according to the number of left shift bits of the feature matrix output by the current layer. Finally, the integers are converted into two's complement form to distinguish the positive and negative signs of each element in each matrix. It can also facilitate the storage of FPGA and the design of hardware description language.

[0019] Specifically, in step (4), the storage module includes weight storage, adjacency matrix storage, feature matrix storage, and intermediate variable caching. The storage module uses a combination of ROM / RAM and two-dimensional register group to realize the automatic storage and reading of parameters.

[0020] Specifically, the calculation module in step (5) includes matrix multiplication operation and activation function ReLU(·). The matrix multiplication operation is implemented by sequential logic circuit and the activation function ReLU(·) is implemented by combinational logic circuit. The coordination module is responsible for controlling the transmission between the stored data stream and the calculation module, as well as the data transmission between each network layer, to ensure that the relevant parameters of each network layer are transmitted to the two-dimensional register group and supplied to the calculation module in real time.

[0021] The beneficial effects of this invention are as follows: Compared with previous acceleration methods, this invention utilizes a hardware description language to design a parallel processing acceleration method where the storage module and the computing module are controlled by a collaborative module. This mainly solves the problem of incomplete software and hardware acceleration, and can meet the requirements of high computing efficiency and real-time performance. Compared with existing inventions, this invention fully utilizes relevant knowledge of digital electronics technology, uses a hardware description language to design more targeted storage modules, computing modules, and collaborative modules, and uses a combination of sequential logic circuits and combinational logic circuits to design the computing module. This fully utilizes the high parallelism and real-time performance of digital circuits and implements the calculation of complex functions in a concise manner. The memory is designed using a combination of ROM / RAM and a two-dimensional register group, enabling automatic reading and storage of massive parameters. This invention can meet real-time requirements and higher acceleration requirements, while also reducing hardware power consumption and lowering costs. Attached Figure Description

[0022] Figure 1 This is a flowchart of the invention;

[0023] Figure 2 This is an overall architecture diagram of the present invention;

[0024] Figure 3 This is a diagram of the GCN model architecture;

[0025] Figure 4 This is an architecture diagram of the storage module and the computing module;

[0026] Figure 5 This is a visualization of the experimental dataset. Detailed Implementation

[0027] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0028] Example 1: As Figure 1 , Figure 2 As shown, a method for accelerating graph convolutional neural networks based on a hardware description language is described. The specific steps of the method are as follows:

[0029] Step S1: Design a three-layer Graph Convolutional Neural Network (GCN) model, with an outer classification layer. The feature output of each layer serves as the feature input of the next layer, and the feature output of the last layer serves as the feature input of the classification layer, ultimately achieving the classification of graph nodes. As a type of graph neural network, the propagation method between layers in a Graph Convolutional Neural Network is...

[0030]

[0031] in, For an undirected graph with added self-connections, let A be the adjacency matrix that does not contain self-connections, and I be the adjacency matrix.N It is the identity matrix. W is the degree matrix. (L) This is the trainable weight parameter matrix of layer L, where L = 0, 1, 2, σ(·) represents the activation function, using ReLU(·), H (L) It is the feature matrix in the Lth layer.

[0032] Step S2: During training, the parameters of each layer of the network are updated using the backpropagation algorithm, with the cross-entropy loss function used. An optimizer, Adam, is established with a learning rate of 0.01 to obtain the trained GCN model.

[0033]

[0034] in, This represents the label of sample i, with 1 for category c and 0 for others. Let represent the probability that sample i is predicted to be of class c, N represent the total number of samples, Cross represent the loss value, and C represent the set of class nodes, where c ∈ C.

[0035] Step S3: Quantize and transform the parameter matrix, normalized adjacency matrix, and feature matrix extracted from the trained GCN model.

[0036] The specific operation is as follows: Shift each element in each matrix to the left by 12 bits, then round down. This involves multiplying each element in each matrix by 4096 and then rounding down. There will be some precision loss in this process, but the higher the number of left shifts, the smaller the precision loss, and the impact on the output is negligible. Furthermore, the number of left shifts for the bias parameters of each network layer depends on the number of left shifts for the feature matrix output by the current layer. Then, convert the integers to two's complement form. Using two's complement representation allows for convenient representation of positive and negative numbers. The two's complement of a positive number is equal to its original code, while the two's complement of a negative number requires inverting the bits of the original code and adding one to the least significant bit. The highest bit of the two's complement is used to represent the sign; a 1 indicates a negative number, and a 0 indicates a positive number. This not only facilitates quick and easy differentiation of the signs of elements in each parameter matrix but also benefits FPGA storage and hardware description language design.

[0037] Step S4: As Figure 4As shown, a storage module is designed using a hardware description language to store the quantized and transformed matrices and cache intermediate variables. The storage module includes weight storage, adjacency matrix storage, feature matrix storage, and intermediate variable caching. It primarily uses a combination of ROM / RAM and a two-dimensional register set to achieve automatic parameter reading and storage. Specifically, the quantized parameters are stored in ROM / RAM, a two-dimensional register set is defined using SystemVerilog, and a horizontal counter and a vertical counter are designed to manipulate each position of the two-dimensional array. Specifically, this involves storing the quantized parameters in ROM / RAM, defining the two-dimensional register set using SystemVerilog, and then designing a horizontal counter and a vertical counter to manipulate each position of the two-dimensional array. Figure 3 The weight parameter matrix and feature matrix (as shown) are automatically stored in different two-dimensional register groups for computation by the calculation module. This is done to effectively improve the parallelism of the algorithm.

[0038] Step S5: As Figure 4 As shown, a computing module and a coordination module are designed using a hardware description language. The computing module includes matrix multiplication operations and the activation function ReLU(·). Matrix multiplication is implemented using sequential logic circuits. The clock edge signal and the enable signal of the coordination module are used as trigger signals for matrix multiplication, as shown in Algorithm 1. Matrix multiplication is implemented using three nested for loops. Each variable represents a two-dimensional register group, which can be assigned values ​​or used for calculations. Then, the bias of each network layer is added to the output feature vector of each network layer. Finally, the activation function ReLU(·) is implemented using combinational logic circuits, which determines the highest bit of the output feature of the current network layer. If it is 0, the data of that variable is retained; if it is 1, the variable is assigned 0, which helps improve computational efficiency. The coordination module is responsible for controlling the transmission of stored data flow between the computing module and the network layers, ensuring that the relevant parameters of each network layer are transmitted to the two-dimensional register group and supplied to the computing module in real time.

[0039]

[0040] Step S6: Create a project on the Vivado platform, integrate the content designed in Steps S3, S4 and S5 into the project, compile and simulate the integrated project to obtain the verified GCN model, deploy the verified GCN model on the FPGA, and process the input graph data stream in real time.

[0041] The integration process involves concatenating three graph convolutional network layers and a classification layer, using the output features of the current network layer as the input features of the next network layer, ultimately achieving the classification of graph data nodes.

[0042] Based on the specific implementation details, the following experiments demonstrate that the invention is indeed feasible:

[0043] 1. Experimental Data

[0044] KarateClub Dataset: This dataset is a social network dataset that records the relationships between members of a small karate club. It consists of 34 members and 78 relationships, where nodes represent club members and edges represent friendships between them. The visualization of this dataset is as follows: Figure 5 As shown, based on the relationship between edges and nodes in this dataset, the graph nodes are divided into four categories.

[0045] 2. Experiment Content

[0046] In the experiment, GCN was used to classify nodes in the dataset into four categories, extracting the adjacency matrix and feature matrix. The adjacency matrix was then symmetrically normalized and used as input to the GCN model. The GCN model was trained on PyCharm to achieve the desired results. A graph convolutional neural network acceleration framework was designed using a hardware description language, simulated, and deployed on an FPGA. The experimental results demonstrate the feasibility of this approach. By comparing the inference time on CPU and GPU, the experimental results are shown in Table 1. The inference time of the GCN model on the FPGA is faster than that on both CPU and GPU.

[0047] Table 1

[0048]

[0049] The core of this invention is to provide a graph convolutional neural network acceleration method based on hardware description language. Combining digital circuit knowledge, a highly parallel graph convolutional neural network acceleration structure is designed. The design of the entire architecture realizes efficient processing of graph convolutional neural networks, realizes the calculation of multifunctional functions in a concise way, and ensures the real-time performance of model inference and the high efficiency of model operation while ensuring model accuracy.

[0050] The specific embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A hardware description language-based graph convolutional neural network acceleration method, characterized in that, Comprise the following steps: (1) design a three-layer graph convolutional neural network model GCN, outside a classification layer, node classification for input graph data, the objective function of graph convolutional neural network layer is as follows: wherein, is the adjacency matrix of the graph with self-loops, A is the adjacency matrix without self-loops, I N is the identity matrix, is the degree matrix, W (L) is the trainable weight parameter matrix of the L-th layer graph convolutional neural network, L = 0, 1, 2, σ(·) represents an activation function, and ReLU(·) is adopted, H (L) is the feature matrix in the L-th layer, H (L+1) is the feature matrix output by the L-th layer; (2) when training, the parameters of each layer of graph convolutional neural network are updated by back propagation algorithm, wherein the loss function adopts cross entropy loss function, and the optimizer Adam is established, the learning rate value is set, and the trained GCN model is obtained; wherein, represents the label of sample i, 1 when the class takes c, and 0 otherwise, represents the probability that sample i is predicted as class c, N represents the total number of samples, Cross is the loss value, and C is the class set of the node, c∈C. (3) the trainable weight parameter matrix, the normalized adjacency matrix and the feature matrix extracted from the trained GCN model are quantized and transformed; (4) combine hardware description language to design storage module for storing quantized and transformed parameters and caching intermediate variables; (5) use hardware description language to design calculation module and coordination module; (6) establish project on vivado platform, integrate the contents designed in steps (3), (4) and (5) into the project, compile and simulate in the integrated project, get the verified GCN model, deploy the verified GCN model in FPGA, and process the input graph data stream in real time.

2. The hardware description language-based graph convolutional neural network acceleration method according to claim 1, characterized in that: In step (1), a three-layer graph convolutional neural network model GCN is built, the feature output of each layer is used as the feature input of the next layer, and the feature output of the last layer is used as the feature input of the classification layer, so as to realize the classification of graph nodes.

3. The hardware description language-based graph convolutional neural network acceleration method according to claim 1, characterized in that: In step (3), the three matrices are quantized and transformed, each element in the matrix is left shifted by 12 bits, and then rounded down, that is, the elements in the matrix are converted to integers, at this time, the matrix will have precision loss, but as long as the left shift number is higher, the precision loss will be smaller, which has little effect on the output result, in addition, the bias parameter of each network layer is left shifted according to the left shift number of the feature matrix output by the current layer, and finally the integer is converted to the complement form, which is used to distinguish the positive and negative of each element in the matrix, and also facilitates the storage of FPGA and the design of hardware description language.

4. The hardware description language-based graph convolutional neural network acceleration method according to claim 1, characterized in that: In step (4), the storage module includes weight matrix storage, adjacency matrix storage, feature matrix storage and intermediate variable cache, and the storage module adopts ROM / RAM and two-dimensional register group to realize automatic storage and reading of parameters.

5. The hardware description language-based graph convolutional neural network acceleration method according to claim 1, characterized in that: In step (5), the calculation module includes matrix multiplication operation and activation function ReLU(·), wherein the matrix multiplication operation is realized by using sequential logic circuit, and the activation function ReLU(·) is realized by using combination logic circuit, the coordination module is responsible for controlling the transmission between the data stream of storage and the calculation module and the data transmission between each network layer, and ensuring that the related parameters of each network layer are transmitted to the two-dimensional register group for real-time supply to the calculation module.

Images