Method and system for compressing a graph convolutional neural network model based on an FPGA

Abstract

The application belongs to the technical field of artificial intelligence data processing, and particularly relates to a compression method and system of a graph convolutional neural network model based on FPGA. In the compression system of the graph convolutional neural network model of the field programmable gate array system chip of the FPGA, an input module is mainly used for inputting an adjacency matrix A and a parameter matrix W, an adjacency matrix calculation module is mainly used for calculating the adjacency matrix A and the parameter matrix W, a combination module is mainly used for combining a certain layer feature matrix X in the graph convolutional neural network model with the parameter matrix W, an aggregation module is mainly used for merging an X layer in the graph convolutional neural network model with an X layer, a new X layer is obtained after merging, and an output module is mainly used for outputting the compressed graph convolutional neural network model. n The application can reduce the operation rate while ensuring the accuracy of the processing result. n+1 The application can reduce the operation rate while ensuring the accuracy of the processing result. n+2 The application can reduce the operation rate while ensuring the accuracy of the processing result. n+2 The application can reduce the operation rate while ensuring the accuracy of the processing result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence data processing, and particularly relates to a compression method and system of a graph convolutional neural network model based on FPGA. BACKGROUND

[0002] Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. In the field of artificial intelligence data processing, the most commonly used method is to process data by establishing a graph convolutional neural network model (GCN) to analyze the data and obtain the processing result.

[0003] The graph convolutional neural network model is a feature extractor for graph-structured data, which contains three basic elements: nodes, connections between nodes, and feature vectors of each node. The graph convolutional neural network model extracts feature vectors of all adjacent nodes of each node and performs linear calculation to extract the features of the entire graph. The extraction of the feature vectors of the adjacent nodes of each node can be abstracted as the multiplication process of the adjacency matrix and the feature vector matrix. In addition to matrix multiplication, the layers are connected together through an activation function, which brings a certain nonlinearity to the graph convolutional neural network model. The main calculation steps of the graph convolutional neural network model are matrix multiplication and the activation function.

[0004] The complexity trend of traditional artificial intelligence machine learning methods is from simple to complex, for example, from linear perception machine to nonlinear multilayer perception machine, and from simple linear picture filter to graph convolutional neural network model. The graph convolutional neural network model is also derived from the traditional machine learning method, inherits the complexity change, and the development process of the graph convolutional neural network model is basically a template for the development of the graph convolutional neural network model, which copies the calculation process of the graph convolutional neural network model. The activation function is an example. If the calculation mode of the graph convolutional neural network model is completely copied, it is inevitable to bring a part of redundant calculation and reduce the calculation efficiency. In fact, through experiments, it can be found that for the neural network model with fewer layers, the role of the activation function is very small, and even if it is removed, it will not have too much influence on the calculation result. The activation function in the graph convolutional neural network model still plays a relatively large role. In the graph convolutional neural network model, the process of calculating the feature vector of each node's neighbor point is essentially equivalent to the smoothing processing in image processing. This means that if the model lacks an activation function, the graph convolutional neural network model is prone to over-smoothing. The feature vectors on all nodes will gradually tend to be consistent and difficult to distinguish. The addition of the activation function introduces a part of nonlinearity and to some extent solves the problem of over-smoothing.

[0005] In the prior art, for example, a light graph convolutional network model (Lightgcn model), in the light graph convolutional network model, all activation functions are discarded, and the bias of each layer is also discarded, the network is simplified, and the calculation efficiency is improved. However, the problem of over-smoothing of the light graph convolutional network model is very obvious. When the number of layers of the graph convolutional neural network model increases, the training and inference accuracy of the graph convolutional neural network model will decrease significantly. In the prior art, the method of increasing the gradient of the graph convolutional neural network model to reduce over-smoothing is used. Specifically, a path is added between each layer of the graph convolutional neural network model to improve the information flow and increase the gradient of the network, thereby solving the problem of over-smoothing. However, the calculation speed increases, the calculation efficiency slows down, and the computer requirements are higher.

[0006] In existing graph convolutional neural network models, the activation function has certain redundancy, which can cause the calculation efficiency to decrease and the operation time to increase, but if the activation function is completely removed, the graph convolutional neural network model becomes a linear network, which is easy to cause the problem of over-smoothing, and when the number of layers of the graph convolutional neural network model increases and the graph convolutional neural network model becomes deep, the problem of over-smoothing is very obvious, which can cause the performance of the graph convolutional neural network model to decrease, therefore, it is necessary to propose a new solution to solve the two problems, on the basis of simplifying the GCN network, a part of the nonlinearity of the GCN network is retained, so that the GCN network can normally perform training and inference work when the number of layers is deep. SUMMARY

[0007] The main purpose of the present application is to provide a FPGA-based graph convolutional neural network model compression method and system, which can effectively solve the problem that the existing graph convolutional neural network model becomes a linear network if the processing speed is fast, is too smooth, or is too complex and has a slow processing speed if the nonlinear effect is good, and has high requirements for computer hardware.

[0008] To achieve the above purpose, the technical scheme adopted by the present application is:

[0009] The present application provides a FPGA-based graph convolutional neural network model compression system, which is deployed in a field programmable gate array system chip of FPGA and runs in the field programmable gate array system chip of FPGA, and comprises an input module, an adjacency matrix calculation module, a combination module, an aggregation module and an output module.

[0010] The input module is used for inputting an adjacency matrix A and a parameter matrix W.

[0011] The adjacency matrix calculation module is used for calculating the adjacency matrix A and the parameter matrix W.

[0012] The combination module is used for combining a certain layer feature matrix X in the graph convolutional neural network model before compression with the parameter matrix W. n The graph convolutional neural network model before compression comprises a plurality of layers, and X n is used to mark the nth layer of the graph convolutional neural network model.

[0013] The aggregation module is used for merging X n+1 layer and X n+2 layer in the graph convolutional neural network model before compression to obtain a new X n+2 layer.

[0014] The output module is configured to output the compressed graph convolutional neural network model and store the compressed graph convolutional neural network model in an FPGA (Field Programmable Gate Array) system chip.

[0015] In the aggregation module, the (n+1)th layer of the graph convolutional neural network model and the (n+2)th layer of the graph convolutional neural network model are directly combined to obtain a new X n+2 layer by removing the activation functions between adjacent layers.

[0016] In the aggregation module, the activation functions between the X i layer and the X i+1 layer are sequentially removed, and i is an odd number.

[0017] In the aggregation module, if the output compressed feature matrix X n+2 is the last layer, the removal of the activation function is stopped. n+2

[0018] In the present application, a residual network is arranged after each reserved activation function.

[0019] In the original graph convolutional neural network model before compression, a nonlinear activation function is used.

[0020] In the present application, in the process of combining the (n+1)th layer of the graph convolutional neural network model and the (n+2)th layer of the graph convolutional neural network model, the operation of the feature matrix X n and the parameter matrix W is performed in the combination module, and the operation result X n W is operated with the sparse matrix A in the aggregation module. n

[0021] The present application further comprises a storage module for storing the input sparse matrix A, the parameter matrix W and the feature matrix X n In the process of combining the (n+1)th layer of the graph convolutional neural network model and the (n+2)th layer of the graph convolutional neural network model, the feature matrix X n+2 of the (n+2)th layer of the compressed graph convolutional neural network model is also stored in the storage module.

[0022] The present application further provides a method based on the above compression system, comprising the following steps:

[0023] The compression system for running the graph convolutional neural network model is deployed in the FPGA (Field Programmable Gate Array) system chip to realize the compression of the graph convolutional neural network model.

[0024] ​​In this invention, each layer of the graph convolutional neural network model before compression contains two parts: matrix multiplication and a nonlinear activation function. By removing the activation functions of adjacent layers at intervals, after removing the activation function of the (n+1)th layer, the (n+1)th layer and the (n+2)th layer of the graph convolutional neural network model are directly merged, combining the two layers in the graph convolutional neural network model into one layer, and outputting the compressed feature matrix X. n+2 This yields a compressed graph convolutional neural network model.

[0025] In this invention, when removing the activation functions of adjacent layers, X is removed sequentially. i Layers and X i+1 The activation function between layers, where i is an odd number; if the output compressed feature matrix X n+2 In the example, the characteristic matrix X n+2 For the last layer, activation function removal stops. In the compressed graph convolutional neural network model, a residual network is added after each retained activation function. Adding the residual network effectively improves the gradient and solves the problem of over-smoothing. After testing on several different datasets, it was found that this network can guarantee reliable accuracy. Tests on datasets such as Cora, Citeseer, PubMed, and PPI showed that when the GCN layer is shallow, the accuracy of this invention is very close to that of the complete GCN, and it also maintains relatively reliable accuracy when the GCN layer is deep.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] This invention innovates the following aspects of the graph convolutional neural network (GCN) model structure: First, by reducing some activation functions, the model is compressed, thereby simplifying the training and inference processes. Second, a residual network (ResNet) is added to address the oversmoothing problem of GCNs. For a multi-layered GCN, this invention compresses the model. Each layer of a basic GCN contains a matrix multiplication and a non-linear activation function. This invention removes one non-linear activation function from every other layer, leaving the layer without an activation function as a linear layer, which can then be merged with the next layer, reducing computational steps. After merging pairs of layers, for deeper networks, a residual network can be added to further increase the gradient, thus resolving the oversmoothing problem.

[0028] The application is deployed on an FPGA chip, and since the compression adopts a method of merging every two layers once, the parameters of every two layers can be combined into one, greatly optimizing the storage space occupied by the model. The system is divided into an input module, an adjacency matrix calculation module, a combination module, an aggregation module and an output module. Due to the reduction of the number of layers, the use frequency of each module is also correspondingly reduced, thereby greatly simplifying the complexity of the graph convolutional neural network model. Experiments prove that the graph convolutional neural network model of this structure can maintain the original accuracy, make the model lightweight, simplify the calculation process and achieve the expected goal. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The system block diagram of the compression system of the FPGA-based graph convolutional neural network model of the application.

[0030] Figure 2 The system architecture diagram of the compression system of the FPGA-based graph convolutional neural network model of the application.

[0031] Figure 3 The single-layer graph convolutional neural network model in the application.

[0032] Figure 4 The double-layer graph convolutional neural network model after compression in the application. DETAILED DESCRIPTION

[0033] In order to make the technical means, creative features, purposes and effects realized by the application easy to understand, the application is further described below in combination with specific embodiments.

[0034] In the description of the application, it should be noted that the directions or position relationships indicated by the terms "upper", "lower", "inner", "outer", "front end", "rear end", "two ends", "one end" and "the other end" are based on the directions or position relationships shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as limiting the application. The devices or elements indicated or implied, therefore, cannot be understood as limiting the application. In addition, the terms "first" and "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0035] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "provided with", "connection" and the like should be understood broadly, for example, "connection" can be fixed connection, can also be detachable connection, or integrally connected; can be mechanical connection, can also be electrical connection; can be directly connected, can also be indirectly connected through intermediate medium, can be internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0036] Please refer to Figures 1-4 The present application provides a kind of compression method of graph convolutional neural network model based on FPGA, comprising the following steps:

[0037] S1: establish the compression system of graph convolutional neural network model in the field programmable gate array system chip of FPGA, and the compression system of graph convolutional neural network model is mainly used to compress graph convolutional neural network model;

[0038] S2: establish input module, adjacency matrix calculation module, combination module, aggregation module and output module in the compression system of graph convolutional neural network model;

[0039] Input module is mainly used to input adjacency matrix A and parameter matrix W, adjacency matrix calculation module is mainly used for the calculation of adjacency matrix A and parameter matrix W, combination module is mainly used to combine certain layer feature matrix X n In graph convolutional neural network model with parameter matrix W, aggregation module is mainly used to merge X n+1 Layer in graph convolutional neural network model with X n+2 Layer, and new X n+2 Layer is obtained, and output module is mainly used to output compressed graph convolutional neural network model, and store compressed graph convolutional neural network model in the field programmable gate array system chip of FPGA.

[0040] Wherein, graph convolutional neural network model before compression includes several layers, X n It is used to mark the n layer of graph convolutional neural network model, and the essence of X n It is the feature matrix of the n layer of graph convolutional neural network model.

[0041] Further, the present application provides a kind of compression system of graph convolutional neural network model based on FPGA, mainly used to compress graph convolutional neural network model.

[0042] Each layer in original graph convolutional neural network model includes matrix multiplication and nonlinear activation function nonlinear two parts, X n+1 =ReLu(AX nW), since the nonlinearity is a nonlinear function, it is difficult to simplify and compare on the basis of preserving the nonlinearity, therefore the present application adopts then intermittently removes the activation function of the adjacent layer, after removing the activation function of the n+1 layer, the n+1 layer of the graph convolutional neural network model and the n+2 layer of the graph convolutional neural network model are directly merged: X n+2 = ReLu[A 2 (X n W n W n+1) ], wherein: A represents the adjacency matrix input by the input module, W represents the parameter matrix input by the input module, in the training and inference process of the GCN, A is a very sparse matrix and a fixed constant matrix, therefore, A 2 will also be a completely fixed constant matrix, similarly, in the GCN, the parameter matrix W is also a constant matrix, therefore W n W n+1 can be merged into a matrix, thereby realizing the merging of two layers in the graph convolutional neural network model into one layer, and further simplifying to obtain X n+2 = ReLu(A 2 X n W), realizing the reduction of the calculation steps and the reduction of the parameters, both in the training process and in the inference process, and obtaining the simplified compressed feature matrix X n+2 output by the output module, and obtaining the compressed graph convolutional neural network model, in actual use: although the formula of the graph convolutional neural network model after merging and compressing two layers is very similar to the formula of the single-layer graph convolutional neural network model, the physical meaning is completely different, which can be seen in Figure 3 、 Figure 4 , in the figure, the single-layer GCN performs aggregation operation on point A, the points marked with 1 are the neighbors of point A, and the points marked with 2 are the neighbors of all neighbors of point A, for the single-layer graph convolutional neural network model, the matrix multiplication of each layer is equivalent to performing weighted addition calculation on the feature vectors of the neighbors of each node in the graph, the output of each node only contains the information of itself and all its neighbors, and in the formula after merging, it is equivalent to performing linear calculation on each node, each neighbor of the node, and each neighbor of the neighbor, therefore, in the calculation process, each node covers a wider range, and the output of each layer contains more information, and the function played by the single-layer graph convolutional neural network model is essentially different.

[0043] wherein, when removing the nonlinearity of the activation function of the adjacent layer, X i layer and X i+1The activation function between layers, specifically i is odd, for example, X1 and X2 are combined into a layer, X3 and X4 are combined into a layer, the activation function between X2 and X3 is reserved, and so on.

[0044] Wherein, if the output compressed feature matrix X n+2 is removed, the feature matrix X n+2 is the last layer, and the activation function is stopped.

[0045] Wherein, a residual network is set after each reserved activation function nonlinearity, and the addition of the residual network can effectively improve the gradient and solve the problem of excessive smoothing. Since the size of the parameter matrix W is a square matrix, the size is freely set, and the calculation formula after adding the residual network is as follows: X n+2 = ReLu(A 2 X n W) + X n After testing several different data sets, it is found that such a network can guarantee a reliable accuracy. Through the test of Cora, Citeseer, Pubmed, PPI and other data sets, it is found that when the number of GCN layers is shallow, the accuracy of the application is very close to that of the complete GCN, and when the number of GCN layers is deep, a relatively reliable accuracy can also be guaranteed. In the test of cora data set, we use a two-layer GCN network, and it is found that the accuracy of GCN before and after compression is about 80.5%. For larger PPI data sets, the accuracy of our GCN model before and after compression can be stabilized at about 71%. It can be seen that the precision loss brought by this compression method is very small.

[0046] Wherein, the storage module is arranged in the compression system of the graph convolutional neural network model, and the storage module is mainly used for storing the input sparse matrix A, the parameter matrix W and the feature matrix X n , in the merging process of the n+1 layer of the graph convolutional neural network model and the n+2 layer of the graph convolutional neural network model, the feature matrix X n+2 of the n+2 layer of the merged and compressed graph convolutional neural network model is also stored in the storage module. The results generated after each running step in the system are all stored in the storage module.

[0047] Wherein, in the merging process of the n+1 layer of the graph convolutional neural network model and the n+2 layer of the graph convolutional neural network model, the operation of the feature matrix X n and the parameter matrix W is carried out in the combination module, the operation result X n of the feature matrix X n and the parameter matrix W is carried out in the aggregation module.

[0048] The above shows and describes the basic principles and main features of the present application and the advantages of the present application. Those skilled in the art should understand that the present application is not limited to the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. A compression system of a FPGA-based graph convolutional neural network model, characterized in that, The compression system of the graph convolutional neural network model is deployed in a field programmable gate array system chip of an FPGA to run, the compression system of the graph convolutional neural network model comprises an input module, an adjacency matrix calculation module, a combination module, an aggregation module and an output module; wherein: The input module is used for inputting the adjacency matrix A and the parameter matrix W; The adjacency matrix calculation module is used for calculating the adjacency matrix A and the parameter matrix W; The combination module is used for combining a certain layer feature matrix X in the pre-compression graph convolutional neural network model with a parameter matrix W; the pre-compression graph convolutional neural network model comprises a plurality of layers, and X is in the plurality of layers n The combination module is used for combining a certain layer feature matrix X in the pre-compression graph convolutional neural network model with a parameter matrix W; the pre-compression graph convolutional neural network model comprises a plurality of layers, and X is in the plurality of layers n The combination module is used for combining a certain layer feature matrix X in the pre-compression graph convolutional neural network model with a parameter matrix W; the pre-compression graph convolutional neural network model comprises a plurality of layers, and X is in the plurality of layers The polymeric module merges the X n+1 layer in the graph convolutional neural network model before compression with the X n+2 layer by intermittently removing the activation function of the adjacent layer, and the merging obtains a new X n+2 layer. The output module is used for outputting the compressed graph convolutional neural network model and storing the compressed graph convolutional neural network model in the field programmable gate array system chip of the FPGA; wherein: In the aggregation module, when removing the activation function between adjacent layers, remove X i layer by layer i+1 If i is an odd number, remove the activation function between X n+2 layer and X n+2 If the output compressed feature matrix X n+2 is the last layer, stop removing the activation function; set a residual network after each retained activation function.

2. The compression system of claim 1, wherein, A nonlinear activation function is used in the original graph convolutional neural network model before compression.

3. The compression system of claim 1, wherein, In the merging process of the n+1th layer of the graph convolutional neural network model and the n+2th layer of the graph convolutional neural network model, the operation of the feature matrix X n and the parameter matrix W is performed in the combination module, and the operation result X n and the parameter matrix W is the feature matrix X n The operation of the sparse matrix A and the parameter matrix W is performed in the aggregation module.

4. The compression system of claim 1, wherein, Also included is a storage module for storing the input sparse matrix A and the parameter matrix W and the feature matrix X n In the merging process of the (n+1)th layer of the graph convolutional neural network model and the (n+2)th layer of the graph convolutional neural network model, the feature matrix X of the (n+2)th layer of the merged and compressed graph convolutional neural network model is output n+2 Also stored in the storage module.

5. A method based on the compression system of claim 1, characterized in that, The method comprises the following steps: The compression system of the graph convolutional neural network model is deployed in a field programmable gate array system chip of an FPGA to run, the compression system of the graph convolutional neural network model comprises an input module, an adjacency matrix calculation module, a combination module, an aggregation module and an output module; wherein: Each layer in the compressed graph convolutional neural network model comprises two parts of matrix multiplication and a nonlinear activation function, and the activation functions of adjacent layers are removed by intervals, after the activation function of the n+1th layer is removed, the n+1th layer of the graph convolutional neural network model and the n+2th layer of the graph convolutional neural network model are directly combined into one layer, and a compressed feature matrix X is output n+2 , and then a compressed graph convolutional neural network model is obtained. When removing the activation function between adjacent layers, X is removed in turn i layer and X i+1 the activation function between layers, i is an odd number; in the compressed graph convolutional neural network model, a residual network is set after each retained activation function.

Images