Graph-level data classification method based on pyramid graph pooling Transformer

Abstract

The invention relates to a pyramid graph pooling Transformer-based graph level data classification method, which comprises the following steps of: acquiring input graph level data, and extracting local neighborhood information of a graph by utilizing a graph neural network to obtain coding features; based on the coding features, obtaining a plurality of pooling graphs through a plurality of parallel graph pooling operations, performing up-sampling on the pooling graphs respectively, and superposing node features of corresponding positions of the graphs to obtain pyramid features; the pyramid features are spliced with a preset token, and an embedded vector representing the whole graph is obtained by utilizing transform; and based on the embedded vector, obtaining a graph-level data classification result by utilizing a multi-layer perception mechanism. Compared with the prior art, the method has the advantages that the pyramid pooling transformer is adopted, so that not only can the dependency relationship between far and near nodes be captured, but also the length of a node sequence in the transformer can be reduced, and multi-scale information can be extracted.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a graph-level data classification method based on a pyramid graph pooling Transformer. Background Art

[0002] In the field of computer vision, convolutional neural networks have been widely used to process data such as images, videos, and audio due to their powerful feature extraction capabilities. In convolutional neural networks, for graph-level data, the receptive field is increased by increasing the number of network layers. However, as the number of network layers increases, the increase in the number of network layers will bring about problems of over-smoothing and over-squeezing, resulting in a decrease in the performance of graph neural networks.

[0003] In graph-level tasks, it is necessary to predict a target value that describes the entire graph. This target value may depend on the dependencies of distant nodes. The limited receptive field of graph convolutional neural networks is not enough to capture the relationships between distant nodes. Increasing the number of network layers will bring about problems of over-smoothing and over-squeezing.

[0004] Therefore, it is crucial to design a new graph-level data classification architecture to break through the message passing limitation and solve the problems of over-smoothing and over-squeezing. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a graph-level data classification method based on pyramid graph pooling Transformer to realize the classification of graph-level data under graph-level tasks.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] One aspect of the present invention provides a graph-level data classification method based on pyramid graph pooling Transformer, comprising the following steps:

[0008] Step S1, obtaining input graph-level data, using graph neural network to extract local neighborhood information of the graph, and obtaining encoding features;

[0009] Step S2, based on the encoding features, multiple pooled graphs are obtained through multiple parallel graph pooling operations and upsampled respectively, and pyramid features are obtained by superimposing node features at corresponding positions of the graphs;

[0010] Step S3, concatenating the pyramid feature with the preset token, and using the cls token to obtain an embedding vector representing the entire graph;

[0011] Step S4, based on the embedding vector, a multi-layer perception mechanism is used to obtain a graph-level data classification result.

[0012] As a preferred technical solution, in step S1, the hidden embedding of the current node is updated by aggregating the messages of the nodes in the neighborhood of the current node in the graph, so as to realize the extraction of the local neighborhood information of the graph.

[0013] As a preferred technical solution, the process of updating the hidden embedding of the current node with the message of the node in the neighborhood of the current node in the aggregation graph is implemented by the following formula:

[0014]

[0015] Among them, h v represents the hidden embedding of node v, l is the number of layers of the local GNN network, UPDATE() represents updating the hidden layer embedding, AGGREGATE() represents the aggregation operation, and N(v) represents the set of neighbor nodes of node v.

[0016] As a preferred technical solution, the step S2 includes the following sub-steps:

[0017] Step S201, for the coding feature, performing a plurality of graph pooling operations with different ratios to obtain a plurality of up-sampled pooling graphs;

[0018] Step S202: for each pooling graph, the discarded nodes in the pooling graph after upsampling are obtained through graph convolution and depooling operations, the features corresponding to the discarded nodes are filled and the discarded nodes are put back into the pooling graph to obtain a restored pooling graph;

[0019] Step S203, adding the corresponding nodes of each pooling graph to obtain pyramid features.

[0020] As a preferred technical solution, in step S202, the process of obtaining the restored pooled graph through graph convolution and depooling operations is implemented by the following formula:

[0021]

[0022] in, represents the feature matrix of the graph obtained after the unpooling operation, λ represents the feature dimension of the graph obtained after the unpooling operation, UnPool() represents the unpooling operation, GCN() represents the graph convolution operation, assignment(X,Y,Z) represents replacing the features in Y except Z with X, where 0 N×C It is represented as a feature matrix initialized to 0, loc represents the index of the retained node in the pooling graph, N is the total number of nodes, and C is the feature dimension of the node.

[0023] As a preferred technical solution, in step S3, the process of obtaining the embedding vector of the entire graph using the transformer includes:

[0024] Step S301, using the encoded feature as a query of the self-attention mechanism in the transformer, and the pyramid feature as a key and value of the self-attention mechanism in the transformer, to obtain a vector output by the self-attention mechanism;

[0025] Step S302: For the vector output by the self-attention mechanism, the embedding vector of the entire graph is obtained through a feedforward neural network, residual connection and normalization.

[0026] As a preferred technical solution, the vector output by the self-attention mechanism is obtained using the following formula:

[0027]

[0028] in, represents the pyramid representation of node v, W is the learnable weight matrix, LayerNorm() represents the layer normalization operation, represents the embedding of the projection node v, denote the learnable query, key, and value matrices, Θ denotes the number of attention heads, ∥ denotes concatenation, and d trans Represents the dimension of transformer, Softmax() represents the softmax activation function, represents the concatenated output of multiple attention heads, is the vector output by the self-attention mechanism.

[0029] As a preferred technical solution, the embedding vector of the entire graph is obtained by feedforward neural network, residual connection and normalization using the following formula:

[0030]

[0031] in, is the intermediate representation, FFN() represents the feedforward network, is the embedding vector of the entire graph.

[0032] Another aspect of the present invention provides an electronic device, characterized in that it includes: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned graph-level data classification method based on the pyramid graph pooling Transformer.

[0033] Another aspect of the present invention provides a computer-readable storage medium, comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned graph-level data classification method based on pyramid graph pooling Transformer.

[0034] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0035] (1) Realizing the classification of graph-level data: The present invention first uses a graph convolutional neural network to encode the node features in the graph data to obtain an encoded representation, then uses a pyramid pooling operation to perform downsampling of the representation at different scales to obtain multi-scale information of the nodes, and finally inputs the node information of different scales into the transformer network, obtains the classification result through a multi-layer perception mechanism, and realizes the classification of graph-level data.

[0036] (2) Using pyramid pooling transformer to realize multi-dimensional feature extraction: As the limited receptive field of graph neural network makes it difficult to capture the dependency relationship between distant nodes on the graph, the present invention uses pyramid pooling transformer, which can not only capture the dependency relationship between distant and near nodes, but also reduce the length of node sequence in transformer and extract multi-scale information, thereby improving the representation learning ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the pyramid graph pooling Transformer in the embodiment;

[0038] Figure 2 Schematic diagram of the pyramid pooling operation in the embodiment. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] Example 1

[0041] In view of the problems existing in the above-mentioned prior art, this embodiment provides a graph-level data classification method based on pyramid graph pooling transformer, which first uses a graph convolutional neural network to encode node features to obtain an encoded representation, then uses a pyramid pooling operation to perform downsampling of different scales on the representation to obtain multi-scale information of the node, and finally inputs the node information of different scales into the transformer network. The pyramid pooling transformer can not only capture the dependency relationship between nodes at long and short distances, but also reduce the length of the node sequence in the transformer and extract multi-scale information.

[0042] This example mainly focuses on graph-level tasks. For a given graph The goal is to predict the category y of the entire image, where V = {v1, v2, ..., v n} represents a collection of nodes, represents the feature matrix of the node, is the adjacency matrix, which describes the relationship between nodes. If the i-th node v in the graph i With the jth node v j connected, then the element A in row i and column j of A i,j Otherwise, it is recorded as 0.

[0043] See also Figure 1 It is the overall architecture of pyramid graph pooling Transformer, which mainly consists of graph neural network, pyramid pooling and transformer modules. The graph neural network is mainly used to encode the graph structure and extract local neighborhood structure information; pyramid pooling is mainly used to extract multi-scale information of the graph while reducing the length of the transformer node sequence; transformer is mainly used to extract the global structure information of the graph while capturing the long-distance dependencies between nodes.

[0044] For a given input graph, it will first pass through a graph neural network module to extract the local neighborhood information of the graph and obtain the encoded features. Then, it will pass through several parallel graph pooling operations to generate multiple pooled graphs, perform upsampling operations on the pooled graphs, fill the features of the discarded nodes in the pooled graphs with 0, and then add the node features at the corresponding positions of the upsampled graphs to obtain pyramid features. This embodiment adds a special CLS token on top of the pyramid features. CLStoken is obtained by random initialization. Before the model training begins, the embedding vector of the CLS token will be randomly assigned, and these initial values ​​will be updated by the back propagation algorithm during the training process to optimize the classification performance. The embedding output of the CLS token after the transformer can summarize the information of other nodes, and finally a multi-layer perceptron can be used to complete the classification and downstream tasks. In addition, this embodiment further modifies the multi-head self-attention, uses the encoded features after the graph neural network encoding as the query of the self-attention, and uses the pyramid features as the key and value of the self-attention.

[0045] The following will be divided into three parts: graph neural network, pyramid pooling and Transformer.

[0046] Graph neural network: The message passing in the graph neural network of this embodiment is represented by aggregating the messages of its neighboring nodes to update its own hidden embedding. This process can be described by the following formula:

[0047]

[0048] where h v represents the hidden embedding of node v, and l is the number of layers of the local GNN network. For the original graph, the graph neural network can effectively extract the local neighborhood information of the graph.

[0049] Pyramid pooling: see Figure 2 This is a schematic diagram of the pyramid pooling operation module, which mainly includes two operations: pooling and upsampling. The pooling operation is used to reduce the number of nodes on the graph and increase the receptive field. After the encoded graph is obtained using the graph neural network, the graph pooling operations of different ratios are performed to obtain pooled graphs of different ratios. The original graph is encoded by the graph convolutional network to obtain the representation matrix H l , the operation process of pyramid pooling can be expressed as:

[0050]

[0051] in They represent the feature matrices of the pooled images obtained with different pooling ratios, and m represents the number of pooling operations. After obtaining multiple pooled images, this embodiment performs an upsampling operation on the encoded pooled images to restore the structure of the original image to better utilize multi-scale information.

[0052] In the field of images, the existing upsampling operation cannot be directly used on graph data. To address this problem, this embodiment provides a depooling operation for graph data, which is the inverse operation of the pooling operation. It can restore the pooled graph to its previous structure. Specifically, it first records the position of the discarded nodes in the pooled graph in the original graph, fills the features on the discarded nodes with 0, and then puts the discarded nodes back to the corresponding positions in the original graph, thereby obtaining a restored graph. This process can be expressed as:

[0053]

[0054] in represents the feature matrix of the graph obtained after the unpooling operation, where λ represents the feature dimension of the graph obtained after the unpooling operation. The unpooling operation can be specifically expressed as:

[0055]

[0056] Where 0 N×C It is represented as a feature matrix initialized to 0, loc represents the index of the retained node in the pooling graph, and for The row vector containing the loc index is copied to 0 N×C At the corresponding position, other row vectors that do not contain the loc index remain unchanged at 0.

[0057] After performing the unpooling operation to obtain multiple graphs, the features of the corresponding nodes of each graph are added, and the vectors whose node features are all 0 are discarded.

[0058] Transformer: After obtaining the pyramid features, they are input into the transformer. The transformer in this embodiment is used to capture the long-distance dependencies between nodes. First, a linear projection is applied to the pyramid representation to transform it into the transformer dimension, and layer normalization is used to normalize the pyramid representation, which can be expressed as:

[0059]

[0060] Where W is the learnable weight matrix, Represents the pyramid representation of node v. The embedding of the projected node v Sent to the transformer, the node representation update process can be expressed as:

[0061]

[0062] in denote the learnable query, key, and value matrices, Θ denotes the number of attention heads, ∥ denotes concatenation, and d trans Represented as the dimension of transformer. Output It will be passed to a feedforward neural network, which is used to enhance the feature expression capability, followed by a residual connection and a normalization layer. The above process can be expressed as:

[0063]

[0064] in In graph-level tasks, an embedding vector representing the entire graph is required. In this embodiment, a special CLS token is added to the sequence input to the transformer, and the attention values ​​of the CLS token and other tokens are calculated through the multi-head self-attention operation in the transformer. In this way, the CLS token of the corresponding sequence output by the transformer contains the information of the entire graph. Finally, a multi-layer perceptron can be used to complete the downstream task. This process can be expressed as:

[0065]

[0066] in Represents the representation of the entire graph, which is the pyramid representation output through multiple transformer layers, and y represents the output category.

[0067] In order to verify the effectiveness of the method in this embodiment, we selected A Collection of Benchmark Datasets for Learning with Graphs (TUDatasets), Benchmarking GNNs and Open Graph Benchmark (OGB) to carry out experiments.

[0068] Two graph learning benchmark datasets, three graph neural network benchmark datasets, and two open-source graph benchmark datasets were used in the experiment. Among the graph learning benchmark datasets, NCI1 and NCI109 are chemical molecule datasets; among the graph neural network benchmark datasets, CIFAR10 and MNIST are image classification datasets, and ZINC is a chemical molecule dataset; among the open-source graph benchmark datasets, ogbg-molhiv is a chemical molecule dataset, and ogbg-code2 is a programming code dataset. Table 1 describes the details of each dataset.

[0069] NCI1 and NCI109 are data sets for screening the anticancer activity of compounds, which are ovarian cancer cells and non-small cell lung cancer, respectively. The graph in the data set represents the compound molecules, and the nodes and edges represent atoms and chemical bonds, respectively. For each data set, we randomly split it into training set, validation set, and test set according to 8:1:1.

[0070] The ZINC dataset is a molecular graph dataset for regressing the restricted solubility properties of molecules. It consists of 12,000 graphs from the commercial compound screening database ZINC. The number of nodes in each molecular graph in the ZINC dataset ranges from 9 to 37. The feature of each node on the graph represents the type of heavy atom, with 28 possible atom types. Each edge on the graph represents a chemical bond, with 3 possible chemical bond types. The ZINC dataset includes 10,000 training graphs, 1,000 validation graphs, and 1,000 test graphs.

[0071] CIFAR10 and MNIST use superpixels to transform CIFAR10 and MNIST in image classification into graph structures. Superpixels are small areas in the image with adjacent positions and similar features such as texture, brightness and color. They can be obtained by SLIC technology. For each image sample, a neighbor graph containing 8 superpixels is constructed, where the maximum superpixels in CIFAR10 and MNIST are 150 and 75 respectively. The image dataset is divided in the same way as the original division in CIFAR10 and MNIST in image classification. The CIFAR10 dataset contains 45,000 training images, 5,000 validation images and 5,000 test images, and the MINIST dataset contains 55,000 training images, 5,000 validation images and 10,000 test images, where the 5,000 validation images in CIFAR10 and MINIST are randomly selected from the training images.

[0072] ogbg-molhiv is a small molecular property prediction dataset from MoleculeNet. The molecules in the dataset are preprocessed using RDKit. Each graph represents a molecule, the nodes in the graph represent atoms, and the edges between the nodes represent chemical bonds. Its goal is to predict molecular properties, which is a binary classification problem, for example: determining whether a molecule can inhibit the replication of HIV.

[0073] ogbg-code2 is a collection of Abstract Syntax Trees (ASTs) extracted from 450,000 Python methods, which are extracted from the 13,887 most popular projects in GitHub. The main task of this dataset is to predict the sub-tokens that form the method from the method represented by the abstract syntax tree, which is called code summarization.

[0074] Table 1 Detailed information of the dataset

[0075]

[0076] The pyramid graph pooling transformer provided in this embodiment is compared with multiple methods on the benchmark dataset, including basic graph neural network methods, classic global pooling and layered pooling methods, and the latest graph transformer methods. The graph neural network models selected are mainly GCN, GIN and GAT; the classic global pooling and layered pooling methods are mainly Set2Set, SortPool, SAGPool g 、SAGPool h ; The latest graph transformer methods include GT, GraphTrans, SAN, Graphormer, SAT and EGT.

[0077] This example is conducted on a Tesla V100 (32GB) GPU. To ensure fairness in the experiment, we use the same parameter configuration as the previous method. Specifically, we use the adamW optimizer and a weight decay of 1e -5 The model was trained with a learning rate of 0.001. In order to reduce the number of parameters, the most basic Top-k pooling method was used. In the experiment, three different ratios of pyramid pooling were set for each block, and the pooling ratios were 0.9, 0.6, and 0.3. The graph neural network encoding layer was set to 3 layers in the experiment. The pooled features generated by each pooling ratio will pass through a layer of encoder network to encode the pooled features. Ten seeds were randomly selected and each experiment was run 10 times. The average and standard deviation were used to evaluate the model.

[0078] The performance of pyramid graph pooling transformer on the graph learning benchmark dataset is shown in Table 2, where SAGPool g and SAGPool h Represents self-attention pooling using global pooling and hierarchical pooling architectures, respectively.

[0079] Table 2 Performance of pyramid graph pooling transformer on two graph learning benchmark datasets

[0080]

[0081] From Table 2 we have the following findings:

[0082] (1) and SortPool and SAGPool h In comparison, the pure transformer model performs worse than the two pooling methods on the graph learning benchmark dataset. The main reason is that the dataset size is relatively small, and directly applying the transformer to the graph data cannot bring out the performance of the transformer.

[0083] (2) The pyramid graph pooling transformer and GT methods perform better than the pure transformer model on graph learning benchmark datasets. This is mainly because both the pyramid graph pooling transformer and GT use an architecture that combines graph neural networks and transformers, which can overcome the problem of transformer's poor performance on small-scale data.

[0084] (3) The pyramid graph pooling transformer achieves good performance on two graph learning benchmark datasets, outperforming conventional graph neural network methods, classic global pooling and layered pooling methods, and graph transformer methods.

[0085] The experimental results of the pyramid graph pooling transformer on the graph neural network benchmark dataset are shown in Table 3. The experimental results of the baseline method are from the results published in the published articles, and '-' means not provided.

[0086] Table 3 Performance of pyramid graph pooling transformer on three graph neural network benchmark datasets

[0087]

[0088] By comparing with previous methods, the following conclusions can be drawn:

[0089] (1) Compared with the classic graph neural network model, the graph transformer performs better. This is mainly due to the fact that the graph transformer adopts the idea of ​​self-attention, calculates the attention scores of all nodes, and can better capture long-distance dependencies.

[0090] (2) Compared with the graph transformer method, the pyramid graph pooling transformer has better performance. This is mainly because our method not only makes use of the transformer, but also uses the pyramid graph pooling operation to fuse more multi-scale features.

[0091] (3) The pyramid graph pooling transformer performs better than previous graph neural network methods and graph transformer methods on graph neural network benchmark datasets. This is mainly because the pyramid graph pooling transformer not only takes advantage of the graph neural network to capture local information and the graph transformer to capture global information, but also takes advantage of the pyramid pooling operation to extract multi-scale features.

[0092] To further verify the performance of the model, the pyramid graph pooling transformer is applied to the open source graph benchmark dataset. The experimental results are shown in Table 4.

[0093] Table 4 Performance of pyramid graph pooling transformer on two open source graph benchmark datasets

[0094]

[0095] GraphTrans and SAT are the top methods on the list of the open source graph benchmark dataset ogbg-code2. Compared with them, the method of this embodiment can still maintain its leading advantage, which is mainly due to the pyramid pooling operation, which can not only reduce the number of nodes, but also integrate multi-scale features. As the number of nodes on the graph increases, global information becomes extremely important, and GNN is difficult to capture long-distance node information. Therefore, as the scale of the graph increases, the advantages of the graph transformer become more obvious.

[0096] To verify the effectiveness of this method, this embodiment also conducts an ablation experiment. The pyramid graph pooling transformer can not only extract multi-scale features, but also reduce the length of the node sequence in the transformer and reduce the computational complexity of the model. The ablation experiment is used to evaluate the impact of the pyramid pooling operation on the model and remove the pyramid pooling operation.

[0097] Table 5 Effect of pyramid pooling operation on ZINC and ogbg-code2 datasets

[0098]

[0099] It can be seen from Table 5 that when the pyramid pooling transformer is removed, the performance of the model on ZINC and CIFAR10 decreases, which shows that the pyramid pooling operation can extract multi-scale features, and the fusion of multi-scale features can improve the performance of the model.

[0100] The graph neural network is mainly used to capture local neighborhood information, and the graph transformer is mainly used to capture global dependencies and extract long-distance node information on the graph. In order to test the impact of the transformer module on the performance of the entire model, the relevant modules are removed from the pyramid pooling transformer. The representation of the fused node obtained by the pyramid pooling operation is aggregated through the readout function to aggregate all node representations, and then a multi-layer perceptron is used to complete the downstream tasks. The experimental results are shown in Table 6.

[0101] Table 6 Performance of Graph Transformer on two graph neural network benchmark datasets

[0102]

[0103] It can be observed from the table that when the transformer module is removed, the performance of the model decreases, which shows that the transformer plays an important role in extracting global information.

[0104] In summary, this embodiment aims to solve the problem that the limited receptive field of graph neural network makes it difficult to capture the dependencies between long-distance nodes on the graph, and provides a graph-level data classification method based on pyramid graph pooling transformer. The pyramid pooling transformer can not only reduce the length of the node sequence, but also extract multi-scale information. The pyramid pooling transformer is mainly divided into three modules: graph neural network, pyramid pooling and transformer, among which the graph neural network mainly extracts local neighborhood information; pyramid pooling mainly extracts multi-scale information of the graph and reduces the length of the transformer sequence; transformer mainly extracts global information and captures long-distance dependencies between nodes. Experiments were conducted on graph learning benchmark datasets, graph neural network benchmark datasets and open source graph benchmark datasets. The experimental results show that the pyramid graph pooling transformer can significantly improve the processing capability of graph-level representation learning methods on graph-level tasks.

[0105] Example 2

[0106] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the graph-level data classification method based on the pyramid graph pooling Transformer as described in Example 1.

[0107] Example 3

[0108] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the graph-level data classification method based on the pyramid graph pooling Transformer as described in Example 1.

[0109] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A graph-level data classification method based on pyramid graph pooling Transformer, characterized in that: The steps include: Step S1, obtaining input graph-level data, using graph neural network to extract local neighborhood information of the graph, and obtaining encoding features; Step S2, based on the encoding features, multiple pooled graphs are obtained through multiple parallel graph pooling operations and upsampled respectively, and pyramid features are obtained by superimposing node features at corresponding positions of the graphs; Step S3, concatenating the pyramid features with the preset tokens, and using transformer to obtain an embedding vector representing the entire graph; Step S4, based on the embedding vector, a multi-layer perception mechanism is used to obtain a graph-level data classification result.

2. According to claim 1, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: In the step S1, the hidden embedding of the current node is updated by aggregating the messages of the nodes in the neighborhood of the current node in the graph, so as to realize the extraction of the local neighborhood information of the graph.

3. According to claim 2, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: The process of updating the hidden embedding of the current node with messages from nodes in the neighborhood of the current node in the aggregation graph is implemented as follows: Among them, g v represents the hidden embedding of node v, l is the number of layers of the local GNN network, UPDATE() represents updating the hidden layer embedding, AGGREGATE() represents the aggregation operation, and N(v) represents the set of neighbor nodes of node v.

4. According to claim 1, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: The step S2 includes the following sub-steps: Step S201, for the coding feature, performing a plurality of graph pooling operations with different ratios to obtain a plurality of up-sampled pooling graphs; Step S202: for each pooling graph, the discarded nodes in the pooling graph after upsampling are obtained through graph convolution and depooling operations, the features corresponding to the discarded nodes are filled and the discarded nodes are put back into the pooling graph to obtain a restored pooling graph; Step S203, adding the corresponding nodes of each pooling graph to obtain pyramid features.

5. According to claim 4, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: In step S202, the process of obtaining the restored pooled graph through graph convolution and depooling operations is implemented by the following formula: in, It means the feature matrix of the graph is obtained after the unpooling operation. Indicates that the feature dimension of the graph is obtained after the unpooling operation, UnPool() indicates the unpooling operation, GCN() indicates the graph convolution operation, and assignment(X,Y,Z) indicates that the features in Y except Z are replaced by X, where 0 N×C It is represented as a feature matrix initialized to 0, loc represents the index of the retained node in the pooling graph, N is the total number of nodes, and C is the feature dimension of the node.

6. According to claim 1, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: In step S3, the process of obtaining the embedding vector of the entire graph using the transformer includes: Step S301, using the encoded feature as a query of the self-attention mechanism in the transformer, and the pyramid feature as a key and value of the self-attention mechanism in the transformer, to obtain a vector output by the self-attention mechanism; Step S302: For the vector output by the self-attention mechanism, the embedding vector of the entire graph is obtained through a feedforward neural network, residual connection and normalization.

7. A graph-level data classification method based on pyramid graph pooling Transformer according to claim 6, characterized in that: The embedding vector of the entire graph is obtained using the following formula: in, represents the pyramid representation of node v, W is the learnable weight matrix, LayerNorm() represents the layer normalization operation, represents the embedding of the projection node v, denote the learnable query, key, and value matrices, Θ denotes the number of attention heads, ∥ denotes concatenation, and d trans Represents the dimension of transformer, Softmax() represents the softmax activation function, represents the concatenated output of multiple attention heads, is the vector output by the self-attention mechanism, is the intermediate representation, FFN() represents the feedforward network, is the embedding vector of the entire graph.

8. According to claim 1, a graph-level data classification method based on pyramid graph pooling Transformer is characterized in that: The graph-level data includes a set of nodes, a feature matrix of the nodes, and an adjacency matrix.

9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the graph-level data classification method based on the pyramid graph pooling Transformer as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: It includes one or more programs for execution by one or more processors of an electronic device, and the one or more programs include instructions for executing the graph-level data classification method based on the pyramid graph pooling Transformer as described in any one of claims 1-8.

Images