A Hybrid Precision Quantization Method for Graph Convolutional Neural Networks Based on FPGA

Abstract

The present invention discloses a mixed-precision quantization method for a graph convolutional neural network based on FPGA. The present invention includes the steps of quantizing the input matrix of the graph convolutional neural network and different layers with different data bit widths; and the step of quantizing the input matrix of the graph convolutional neural network by using a symmetric linear quantization method. In addition to the improvement of the basic quantization method, the present invention also embeds the idea of mixed-precision quantization into the graph convolutional neural network. For different matrix data, different data bit widths are used for quantization, and for matrix data located in different layers of the GCN, different data bit widths are also used for quantization. The quantization method of the present invention can improve the quantization accuracy of the model while achieving further compression of the entire GCN model, making it easier to be deployed on FPGA.

Description

Technical Field

[0001] The present invention belongs to the technical field of information processing. Specifically, it relates to a mixed-precision quantization method for a graph convolutional neural network based on FPGA. Background Art

[0002] A graph convolutional network (GCN) is a feature extractor for graph-structured data. Graph-structured data contains three basic elements: nodes, connections between nodes, and feature vectors for each node. GCN extracts the features of the entire graph by extracting feature vectors from all adjacent nodes of each node and performing linear calculations. The calculation process of GCN is relatively simple and includes a total of three parts: sparse matrix multiplication, dense matrix multiplication, and the activation function Relu. The main calculation difficulty of GCN lies in the extremely large amount of matrix data and the multiplication calculation of sparse matrices, making the inference process of GCN difficult to optimize. The serial execution of ordinary matrix multiplication and sparse matrix multiplication is a significant feature of GCN.

[0003] A field programmable gate array (FPGA) has the characteristics of low power consumption, low cost, high reconfigurability, high stability, etc., and now has broad development prospects. However, the resource amount of FPGA is relatively limited, while graph-structured data is usually relatively large. As the number of nodes increases, the complexity of the feature matrix will increase linearly, and by definition, the size of the adjacency matrix is equal to the square of the number of nodes, with a faster increase in complexity. Therefore, when the number of nodes is large, GCN requires a large amount of storage resources, which also makes it difficult to implement GCN on FPGA. In order to compress the GCN model and make it adaptable to the hardware architecture, we need to quantize GCN to reduce the scale of data and achieve the lightweight of GCN. In addition, FPGA does not support direct floating-point calculations, so quantization is also a necessary task for FPGA board implementation. However, compared with other well-developed neural networks, the development of graph convolutional neural networks is relatively late, and there is currently no dedicated quantization algorithm for GCN models. When quantizing GCN, other neural network quantization methods are usually copied. These quantization methods do not combine the characteristics of GCN itself, so there is still room for further optimization.

[0004] In existing articles on implementing graph convolutional neural network models on hardware, the focus is on the overall architecture idea and the content of quantization is lacking. In existing papers, there is basically no mention of the content related to GCN quantization, and the methods used for GCN quantization are all general quantization methods. Although the general methods are feasible, there is still room for further optimization. Summary of the Invention

[0005] Aiming at the above-mentioned existing technologies, the purpose of the present invention is to provide a quantization method for graph convolutional neural networks; this method adopts different quantization methods for matrices with different functions and different layers of GCN, can realize the lightweight of the GCN model, quantize floating-point operations into integer operations, and can compress various different types of data in the GCN model as much as possible while having a small accuracy loss, so as to compress the multi-layer graph convolutional network model and make it easier to be deployed on hardware (FPGA).

[0006] The present invention has carried out different quantization methods based on standard symmetric linear quantization for different matrix data in graph convolutional neural networks. At the same time, the present invention also embeds the idea of mixed-precision quantization into graph convolutional neural networks, adopts different data bit widths for quantization of different matrix data, and also uses different data bit widths for quantization of matrix data in different layers of GCN. The technical solution of the present invention is specifically introduced as follows.

[0007] The present invention provides a mixed-precision quantization method for graph convolutional neural networks based on FPGA, which is used for the deployment of graph convolutional neural networks on FPGA; it includes the steps of quantizing the input matrix and different layers of the graph convolutional neural network with different data bit widths, and the step of quantizing the input matrix of the graph convolutional neural network with a symmetric linear quantization method; the input matrix includes an adjacency matrix, a weight matrix, and a feature matrix; wherein:

[0008] In the step of quantizing the input matrix and different layers of the graph convolutional neural network with different data bit widths, the data bit width used for the feature matrix is more than the data bit widths used for the adjacency matrix and the weight matrix;

[0009] The step of quantizing the input matrix of the graph convolutional neural network with the symmetric linear quantization method includes:

[0010] For the weight matrix, select a part of the relatively symmetric interval from the input data for mapping, and map all data outside this symmetric interval to other values (-2 t-1 )); The specific formula for weight quantization is as follows:

[0011]

[0012] Where x is the value before mapping, x q is the value after mapping, t is the bit width of the quantized data, and fq is the boundary of the symmetric interval for mapping;

[0013] The L2 norm is used to measure the error brought by the selected symmetric interval for quantization to determine the symmetric interval. When the norm takes the minimum value, it means that the selected symmetric interval is optimal. The calculation formula of the L2 norm is as follows:

[0014]

[0015] For the adjacency matrix and the feature matrix, a linear mapping method is used for quantization. The quantization formula is as follows:

[0016]

[0017] where f q is also determined by the L2 norm in the same way;

[0018] In the present invention, the data bit width used for the adjacency matrix is Uint 8; the data bit width used for the feature matrix of the first layer of the graph neural network is Uint 16, and the data bit width used for the feature matrix of the second layer is Uint 8; the data bit widths used for the weight matrices of the first and second layers of the graph neural network are both Int 8.

[0019] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0020] In the present invention, matrices sensitive to quantization error are quantized using more data bit widths to further improve the accuracy, while matrices insensitive to quantization error are quantized using fewer data bit widths, so as to compress the entire GCN model as much as possible; the present invention can realize the lightweight of the GCN model, quantize the floating-point operation into an integer operation, and can compress various different types of data in the GCN model as much as possible while having a small loss of accuracy, so as to compress the multi-layer graph convolutional network model and make it easier to be deployed on hardware (FPGA). BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Resource redundancy caused by symmetric quantization.

[0022] Figure 2 Effect after improvement of symmetric quantization. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0023] The technical solutions of the present invention will be introduced in detail below with reference to the drawings and embodiments.

[0024] The present invention designs a quantization method for graph convolutional neural networks, which can compress the multi-layer graph convolutional network model. For various matrix data in graph convolutional neural networks, a general improved symmetric linear quantization method is adopted, and different quantization data bit widths are designed for different data, and the method of mixed precision quantization is embedded.

[0025] 1) Quantization of weights

[0026] In the design of the quantization method for the weight matrix, we need to consider the particularity of graph convolutional neural networks. Since the activation function used in graph convolutional neural networks is Relu, the role of Relu is to set all negative data to zero while retaining all positive data. Therefore, during quantization, we need to ensure that the zero point remains zero after quantization, so as to ensure the accuracy of the activation function before and after quantization and minimize the accuracy loss caused by quantization. Therefore, the basic quantization idea can be determined. For weights, we adopt a symmetric linear quantization method to achieve model compression.

[0027] A relatively large drawback of symmetric linear quantization is that it is prone to resource waste, as Figure 1 shown.

[0028] When the input data is asymmetric, due to the symmetry of the quantized result, there will be relatively large redundancy after quantization. To solve this problem, assuming our goal is to quantize to int8, that is, the interval [-128, 127], we modify the quantized interval to [-127, 127]. We select a relatively symmetric interval from the input data and map it to the quantized [-127, 127], while all data outside this interval is mapped to -128. This can avoid resource waste. For the selection of the symmetric interval, we use the L2 norm to measure the error caused by quantization. When this norm takes the minimum value, it means the interval we select is the optimal one. The effect after improving the quantization algorithm is as Figure 2 shown, and the specific quantization formula is as follows:

[0029]

[0030] The calculation formula of the L2 norm is as follows:

[0031]

[0032] It can be clearly seen that Figure 1 due to the very asymmetric input data in Figure 2 there is relatively large resource waste in the quantized interval, while in a relatively symmetric interval is selected for mapping, which greatly improves the resource utilization efficiency and reduces waste.

[0033] 2) Quantization of the adjacency matrix and the feature matrix

[0034] For a graph convolutional neural network, the input consists of three parts: an adjacency matrix, a weight matrix, and a feature matrix. Among them, the adjacency matrix represents the structure of the entire undirected or directed graph, and all its elements are non-negative; the feature matrix comes from the output of the previous layer of GCN. Since the output of GCN passes through the Relu function, the data contained in the feature matrix is also non-negative. For non-negative data, we hope that the quantized result is also non-negative, and the zero point should be quantized to zero. For ease of implementation, we also use the method of linear mapping for quantization. The quantization formula is as follows:

[0035]

[0036] where f q is also determined by the L2 norm in the same way.

[0037] 3) Mixed precision

[0038] In a graph convolutional neural network, there are an adjacency matrix, a feature matrix, and a weight matrix. Among them, the adjacency matrix and the weight matrix can be regarded as constants, while the feature matrix needs to be read and written multiple times. In addition, the adjacency matrix and the feature matrix are both non-negative matrices, while the weight matrix contains both negative and positive numbers. Therefore, the functions of these three matrices are different, and the quantization methods are also different. Therefore, when implementing on hardware, different quantization data bit widths should be used to quantize them. In addition, for the feature matrix, the output of one layer of GCN will be used as the input of the feature matrix of the next layer to participate in the inference operation. Therefore, the quantization error of the previous layer of GCN will accumulate into the next layer, resulting in a larger error. Considering this point, different data bit widths should also be used to quantize the feature matrices of different layers. In our experiment, we used a two-layer graph convolutional neural network to perform simulation tests on datasets such as cora and pubmed. The target data bit widths for quantization of each variable in the experiment are shown in Table 1, and the final error obtained is about 2%. It can be seen that this quantization method is feasible.

[0039] Table 1 Data bit widths for quantization of different matrices

[0040] Variables to be quantified Meaning Data bit width A Adjacency matrix Uint 8 H1 Feature matrix of the first layer Uint 16 H2 Feature matrix of the second layer Uint 8 W1 Weight matrix of the first layer Int 8 W2 Weight matrix of the second layer Int 8 。

Claims

1. A mixed-precision quantization method for graph convolutional neural networks based on FPGA, characterized in that, It is used for the deployment of graph convolutional neural networks on FPGAs. The graph neural network is used for literature classification tasks to classify all nodes in the graphs in the cora and pubmed literature classification datasets; it includes steps of quantifying the input matrix and different layers of the graph convolutional neural network using different data bit widths, and steps of quantifying the input matrix of the graph convolutional neural network using a symmetric linear quantization method; the input matrix includes an adjacency matrix, a weight matrix, and a feature matrix; where: In the step of quantifying the input matrix and different layers of the graph convolutional neural network using different data bit widths, the data bit width used for the feature matrix is more than the data bit widths used for the adjacency matrix and the weight matrix; The step of quantifying the input matrix of the graph convolutional neural network using the symmetric linear quantization method includes: For the weight matrix, a relatively symmetric interval is selected from the input data for mapping, and all data outside this symmetric interval is mapped to ; The specific formula for weight quantization is as follows: ， Among them, is the value before mapping, is the value after mapping, is the bit width of the quantized data, is the boundary of the symmetric interval of the mapping; Using the L2 norm to measure the error brought by the selected symmetric interval for quantization to determine the symmetric interval. When the norm takes the minimum value, it means the selected symmetric interval is optimal; the calculation formula of the L2 norm is as follows: ， wherein, is the L2 norm, is the actual value after quantization and rounding of the output, is the number of data; For the adjacency matrix and the feature matrix, the quantization formula is as follows: ， Among them is also determined by the L2 norm in the same way.

2. The hybrid precision quantization method of the graph convolutional neural network based on FPGA according to claim 1, wherein The data bit width used for the adjacency matrix is Uint 8; the data bit width used for the feature matrix of the first layer of the graph neural network is Uint 16, and the data bit width used for the feature matrix of the second layer is Uint 8; the data bit widths used for the weight matrices of the first and second layers of the graph neural network are both Int 8.

Images