Node classification method, system, equipment and product based on improved graph Transform model

Through graph data clustering partitioning and self-attention mechanism optimization, combined with virtual nodes and similarity screening, the calculation complexity and noise problems of graph Transformer model in large-scale graph data processing are solved, and the performance of node classification is improved.

CN118916757APending Publication Date: 2024-11-08SHENZHEN RES INST OF WUHAN UNIVERSITY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410947014.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-16
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing graph Transformer model has high computational complexity when processing large graph data, and the fully connected self-attention mechanism introduces a large amount of noise information, which reduces the performance of the model.

Method used

By introducing the METIS algorithm to cluster and partition the graph data, adjust the sampling method of the self-attention mechanism to calculate the node, and introduce a node screening mechanism based on similarity, using virtual nodes and simple graph convolution neural networks to aggregate information, optimize self-attention calculations .

Benefits of technology

It significantly reduces the computational complexity of the model, reduces interference from irrelevant information, and improves the accuracy and performance of node classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916757A_ABST
    Figure CN118916757A_ABST
Patent Text Reader

Abstract

The invention discloses a node classification method, system, equipment and product based on an improved graph Transform model, and the method comprises the steps: carrying out the clustering and partitioning of input graph structure source data containing to-be-classified nodes, dividing an original large graph into a plurality of small sub-graphs, and carrying out the remapping and arrangement of the nodes and edge data, and obtaining a series of sub-graph data {G1, G2,..., Gk} = {(V1, E1), (V2, E2),..., (Vk, Ek)}; then generating a virtual representative node for each sub-graph, and aggregating information of each sub-graph into the virtual node to obtain a sub-graph virtual representative node feature; thirdly, the node feature information is input into an improved graph Transform model to be subjected to coding calculation; and finally, inputting the final node vector representation obtained by coding calculation into a multi-layer perceptron for node classification operation to obtain a classification result. According to the method, the calculation complexity of model self-attention operation is greatly reduced, node classification tasks of small, medium and large graph data sets can be considered at the same time, and the classification performance is comprehensively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of graph machine learning and graph data mining, and relates to a node classification method, device and product of a graph Transformer model, and specifically to a node classification method, device and product based on an improved graph Transformer model. Background Art

[0002] Graph data is ubiquitous in the real world, from the interactions between people in social networks to the connectivity between cities in transportation networks; from the huge link structure of the Internet to the interactions between molecules and proteins in biology, all can be represented in the form of graphs. The ubiquity and importance of graph data have stimulated the urgent research demand for graph data mining. The node classification task is one of the core and basic tasks in many graph data mining tasks. In this task, it is necessary to predict the category of the node based on the characteristics of the node and the information of its neighbor nodes in the graph. User classification in social networks is a typical example. By analyzing the user's social relationship network and personal characteristics, the user's interests, occupations and other information can be predicted.

[0003] At present, graph convolutional neural networks and graph transformer models are the two main research directions for solving the above tasks. Among them, thanks to the introduction of the self-attention mechanism, the graph transformer model can more effectively capture the global information and long-distance relationship dependencies in the graph compared to the graph convolutional neural network, and thus has stronger graph data analysis and processing capabilities.

[0004] However, existing graph Transformer models usually use a fully connected self-attention calculation mechanism on the entire input graph. This mechanism requires the model to calculate each node of the input graph with all other nodes, which causes the computational complexity of the model to be proportional to the square of the number of graph nodes. This high computational complexity places high demands on computing resources, severely limiting the application of graph Transformer models in node classification tasks in large graph data scenarios.

[0005] In addition, the fully connected self-attention mechanism is equivalent to treating the input graph as a fully connected graph when processing nodes, that is, each node needs to integrate information from all other nodes in the graph. Although this mechanism can theoretically enhance the model's ability to capture global information, in practical applications, it may introduce a large amount of noise information from irrelevant or distant neighbor nodes, thereby reducing the model's sensitivity to key information and judgment accuracy, ultimately leading to a decline in model performance. Summary of the invention

[0006] In order to solve the above technical problems, the present invention proposes a node classification method, device and product based on an improved graph Transformer model.

[0007] In a first aspect, the present invention provides a node classification method based on an improved graph Transformer model, comprising the following steps:

[0008] Step 1: Cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k}={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k;

[0009] Step 2: Generate a virtual representative node for each subgraph, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node features;

[0010] Step 3: Input the node feature information into the improved graph Transformer model for encoding calculation; the improved graph Transformer model specifically includes adjusting the sampling method of the self-attention mechanism operation nodes and introducing a similarity-based operation node screening mechanism;

[0011] Step 4: Input the final node vector representation obtained by encoding calculation into the multi-layer perceptron for node classification operation to obtain the classification result.

[0012] Preferably, in step 1, the input graph structure source data containing the nodes to be classified is clustered and partitioned using the METIS algorithm, and the specific implementation includes the following sub-steps:

[0013] Step 1.1: Using a multi-level approach, first construct the graph structure of the initial level, retain the node and edge information of the original graph, then reorder the nodes and edges of each level of the graph, and gradually construct a high-level graph until the preset number of levels or node number threshold is reached;

[0014] Step 1.2: Construct a coarse-grained graph representation by merging nodes and edges, including aggregating nodes according to their degree and similarity to form supernodes, and aggregating edges according to their weights and connectivity to form superedges.

[0015] Step 1.3: Recursively divide the coarse-grained graph into two subgraphs step by step until the preset number of subgraphs is reached, and determine the nodes and edges in each subgraph;

[0016] Step 1.4: Refine and optimize the partitioning results to ensure that the boundaries between each subgraph are minimized; optimize the boundary nodes across subgraphs through optimization algorithms, adjust the node attribution to reduce the number of cross-subgraph edges, and optimize the nodes within the subgraph to improve the connectivity and compactness of the subgraph;

[0017] Step 1.5: Convert the partitioning result back to the representation of the original graph structure; output the subgraph number to which each node belongs, and the information of the nodes and edges contained in each subgraph; the output format includes the correspondence between the node number and the subgraph number, as well as the node list and edge list in each subgraph.

[0018] Preferably, the specific implementation of step 2 includes the following sub-steps:

[0019] Step 2.1: For subgraph G i , introduce a unique virtual node VN i , the virtual node and the subgraph G i Establish connections among all nodes in the

[0020] Step 2.2: Input the subgraph with virtual nodes added into the multi-layer simple graph convolutional neural network for propagation calculation. During the propagation calculation process, information is exchanged with all nodes in the subgraph. According to the formula Calculate the output of virtual node features at each layer of the simple graph convolutional neural network, and use the output of the last layer as the virtual representative node feature of the subgraph; is the feature vector of node u in layer l, |G i | is the subgraph G i The number of nodes in W (l) is the weight matrix of the lth layer, and σ is the activation function.

[0021] Preferably, in step 3, the improved graph Transformer model includes a neighbor node sampling module, a similarity-based node screening module, a position encoding module, a multi-head self-attention calculation module and a feedforward network module;

[0022] The neighbor node sampling module is used to obtain a set of neighbor node feature information corresponding to each node participating in subsequent self-attention calculation by sampling feature information of other nodes in the same partition and feature information of virtual representative nodes in other partitions for each node;

[0023] The node screening module based on similarity is used to further screen neighbor nodes with high similarity to the target node; first, by calculating the cosine similarity between node feature information, the similarity score between each pair of nodes is output, and then the neighbor nodes with similarity higher than the threshold are selected according to the set threshold to participate in subsequent calculations;

[0024] The position encoding module encodes the position information between nodes and adds it to the input node embedding representation and the filtered neighbor node embedding representation set. The node embedding representation after adding the position encoding is input to the multi-head self-attention calculation module.

[0025] The multi-head self-attention calculation module includes a plurality of parallel self-attention calculation heads, each of which performs a linear transformation on the input feature information to generate a query Q, a key K and a value V vector, and then calculates the dot product of the query Q vector and the key K vector, and scales the dot product result by dividing it by the square root of the dimension of the key K vector, and then normalizes the scaled dot product result through a Softmax function and converts it into a probability distribution form, and multiplies the normalized attention weight by the value V vector to obtain a weighted output; finally, the outputs of all self-attention calculation heads are spliced ​​together to form a comprehensive output vector O;

[0026] The feedforward network module is used to further process the output of the multi-head self-attention calculation module; firstly, the first residual connection and layer normalization operation are performed, the output vector O of the multi-head self-attention calculation module is added to the query vector Q inputted initially at the element level, and then normalized in the feature dimension; then the feedforward network layer is inputted, which includes two linear transformation layers and an activation layer therebetween, the first linear transformation layer transforms the input features, the activation layer introduces nonlinearity, and the second linear transformation layer generates output; finally, the second residual connection and layer normalization are performed, the output of the feedforward network layer is added to the input of the feedforward network layer at the element level, and the output is normalized in the feature dimension. .

[0027] As a preferred method, in step 3, firstly according to the formula Calculate the cosine similarity between the node and other nodes in the same partition and other partition virtual representative nodes; where x u and x v represent the feature vectors of nodes u and v respectively, ||x u || and ||x v || represents the Euclidean norm of a vector;

[0028] Then the nodes are sorted according to the similarity, and only a certain proportion of nodes with the highest similarity are selected to participate in the subsequent self-attention calculation.

[0029] As a preferred method, in step 4, the final node vector representation obtained by encoding calculation is input into the multi-layer perceptron for node classification operation. According to the formula Calculate the probability that node u belongs to category c; where y u is the true category of node u, C is the total number of categories, W c and b c are the weight vector and bias term associated with category c, h u Represents the final vector representation of the node obtained by encoding calculation.

[0030] In a second aspect, the present invention provides a node classification system based on an improved graph Transformer model, comprising the following modules:

[0031] The clustering and partitioning module is used to cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k}={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k;

[0032] A subgraph virtual representative node feature acquisition module is used to generate a virtual representative node for each subgraph, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node feature;

[0033] An encoding calculation module is used to input node feature information into an improved graph Transformer model for encoding calculation; the improved graph Transformer model specifically includes adjusting the sampling method of the operation nodes of the self-attention mechanism and introducing an operation node screening mechanism based on similarity;

[0034] The node classification module is used to input the final node vector representation obtained by encoding calculation into the multi-layer perceptron to perform node classification operation and obtain the classification result.

[0035] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the node classification method based on the improved graph Transformer model when executing the program.

[0036] In a fourth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the node classification method based on the improved graph Transformer model.

[0037] Compared with the prior art, the beneficial effects of the invention include:

[0038] (1) The present invention aims to optimize the self-attention calculation mechanism. By introducing graph data clustering partitioning based on the METIS algorithm, adjusting the sampling method of the self-attention mechanism operation nodes, and introducing the similarity-based operation node screening mechanism, the number of nodes involved in the self-attention calculation is reduced, thereby greatly reducing the computational complexity of the model;

[0039] (2) The present invention reduces the interference of irrelevant information by calculating node features based on a simple graph convolutional neural network and virtual partitions of virtual nodes, thereby improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The technical solution of this invention is further described below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of describing the technical solution. For those skilled in the art, other drawings and the intention of the invention can also be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a method according to an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the principle of calculating the characteristics of a virtual partition representative node according to an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of the improved Transformer model structure of an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0046] Please see Figure 1 and Figure 2 The present invention provides a node classification method based on an improved graph Transformer model, which is characterized by comprising the following steps:

[0047] Step 1: Cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k}={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k;

[0048] In one embodiment, information is collected by various sensors, crawlers and other equipment, facilities and technologies to obtain graph structure source data containing nodes to be classified. The graph structure source data includes graph structure information and node feature information of each node in the graph. The source data is preprocessed to construct a graph structure data set G = (V, E), where V = {v1, v2, ..., v m}、E={e1,e2,…,e n} represent the node set and edge set respectively.

[0049] In one implementation, the METIS algorithm is used to cluster and partition the input graph structure source data containing the nodes to be classified, and the specific implementation includes the following sub-steps:

[0050] Step 1.1: Use a multi-level method to decompose the graph into multiple levels and reorder the nodes and edges. Specifically, first build the graph structure of the initial level, retain the node and edge information of the original graph, then reorder the nodes and edges of each level of the graph, and gradually build a high-level graph until the preset number of levels or node number threshold is reached.

[0051] Step 1.2: Construct a coarse-grained graph representation by merging nodes and edges, including aggregating nodes according to their degrees and similarities to form supernodes, and aggregating edges according to their weights and connectivity to form superedges.

[0052] Step 1.3: Recursively divide the coarse-grained graph into two subgraphs step by step until the preset number of subgraphs is reached, and determine the nodes and edges in each subgraph.

[0053] Step 1.4: Refine and optimize the partitioning results to ensure that the boundaries between each subgraph are minimized. Use the optimization algorithm to adjust the ownership of nodes, reduce the number of edges across subgraphs, and improve the partitioning quality. Specifically, it includes optimizing the boundary nodes across subgraphs, adjusting the ownership of nodes to reduce the number of edges across subgraphs, and optimizing the nodes within the subgraphs to improve the connectivity and compactness of the subgraphs.

[0054] Step 1.5: Convert the partitioning result back to the representation of the original graph structure. Output the subgraph number to which each node belongs, and the information of the nodes and edges contained in each subgraph. The output format includes the correspondence between the node number and the subgraph number, as well as the node list and edge list in each subgraph.

[0055] Step 2: Generate a virtual representative node for each subgraph, perform propagation calculations through a simple graph convolutional neural network, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node features;

[0056] Please see Figure 3 In one implementation, the specific implementation of step 2 includes the following sub-steps:

[0057] Step 2.1: For subgraph G i , introduce a unique virtual node VN i , the virtual node and the subgraph G i Establish connections among all nodes in the

[0058] Step 2.2: Input the subgraph with virtual nodes added into the multi-layer simple graph convolutional neural network for propagation calculation. During the propagation calculation process, information is exchanged with all nodes in the subgraph. According to the formula Calculate the output of virtual node features at each layer of the simple graph convolutional neural network, and use the output of the last layer as the virtual representative node feature of the subgraph; is the feature vector of node u in layer l, |G i | is the subgraph G i The number of nodes in W (l) is the weight matrix of the lth layer, and σ is the activation function.

[0059] Step 3: Input the node feature information into the improved graph Transformer model for encoding calculation. The improved graph Transformer model includes adjusting the sampling method of the self-attention mechanism operation nodes and introducing a similarity-based operation node screening mechanism.

[0060] Please see Figure 4 In one embodiment, the improved graph Transformer model includes a neighbor node sampling module, a similarity-based node screening module, a position encoding module, a multi-head self-attention calculation module and a feedforward network module;

[0061] The neighbor node sampling module is used to obtain a set of neighbor node feature information corresponding to each node participating in subsequent self-attention calculation by sampling feature information of other nodes in the same partition and feature information of virtual representative nodes in other partitions for each node;

[0062] The similarity-based node screening module is used to further screen neighbor nodes with high similarity to the target node. First, the cosine similarity between node feature information is calculated to output the similarity score between each pair of nodes, and then neighbor nodes with similarity higher than the threshold are selected according to the set threshold to participate in subsequent calculations;

[0063] The position encoding module encodes the position information between nodes and adds it to the input node embedding representation and the filtered neighbor node embedding representation set. The node embedding representation after adding the position encoding is input to the multi-head self-attention calculation module;

[0064] The multi-head self-attention calculation module includes multiple parallel self-attention calculation heads, each of which first performs a linear transformation on the input feature information to generate query (Q), key (K) and value (V) vectors, then calculates the dot product of the query (Q) vector and the key (K) vector, and then scales the dot product result by dividing it by the square root of the value (K) vector dimension, and then normalizes the scaled dot product result through the Softmax function and converts it into a probability distribution form to represent the importance of the query (Q) corresponding to each key (K), and then multiplies the normalized attention weight with the value (V) vector to obtain a weighted output. Finally, the outputs of all self-attention calculation heads are spliced ​​together to form a comprehensive output (O) vector;

[0065] The feedforward network module is used to further process the output of the multi-head self-attention calculation module. First, the first residual connection and layer normalization operation are performed, and the output (O) vector of the multi-head self-attention calculation module is added to the query (Q) vector initially input at the element level, and then normalized in the feature dimension; then the feedforward network layer is input, which includes two linear transformation layers and an activation layer therebetween. The first linear transformation layer transforms the input features, the activation layer introduces nonlinearity, and the last linear transformation layer generates the output; finally, the second residual connection and layer normalization are performed, and the output of the feedforward network layer is added to the input of the feedforward network layer at the element level, and then normalized in the feature dimension before output.

[0066] In one embodiment, first, according to the formula The cosine similarity between the computation node and other nodes in the same partition and other partition virtual representative nodes; where X u and X v represent the feature vectors of nodes u and v respectively, ||x u || and ||x v || represents the Euclidean norm of a vector;

[0067] Then, the nodes are sorted according to the similarity, and only a certain proportion of nodes with the highest similarity are selected to participate in the subsequent self-attention calculation. Specifically, for node u, the top 80% of other nodes in the same partition with the highest similarity and the top 20% of virtual representative nodes in other partitions are selected as its corresponding neighbor nodes. Then, the feature information of each node and the feature information of the corresponding neighbor node are input into the multi-head self-attention calculation module to obtain a new node vector representation. Then, the obtained node vector representation is sequentially subjected to residual connection operation and normalization operation, and then input into the feedforward network module.

[0068] Step 4: Input the final node vector representation obtained by encoding calculation into the multi-layer perceptron for node classification operation to obtain the classification result.

[0069] In one embodiment, the final node vector representation obtained by encoding calculation is input into a multi-layer perceptron for node classification operation. Calculate the probability that node u belongs to category c; where y u is the true category of node u, C is the total number of categories, W c and b c are the weight vector and bias term associated with category c, h u Represents the final vector representation of the node obtained by encoding calculation.

[0070] This embodiment also provides a node classification system based on the improved graph Transformer model, including the following modules:

[0071] The clustering and partitioning module is used to cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k}={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k;

[0072] A subgraph virtual representative node feature acquisition module is used to generate a virtual representative node for each subgraph, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node feature;

[0073] An encoding calculation module is used to input node feature information into an improved graph Transformer model for encoding calculation; the improved graph Transformer model specifically includes adjusting the sampling method of the operation nodes of the self-attention mechanism and introducing an operation node screening mechanism based on similarity;

[0074] The node classification module is used to input the final node vector representation obtained by encoding calculation into the multi-layer perceptron to perform node classification operation and obtain the classification result.

[0075] This embodiment also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the node classification method based on the improved graph Transformer model when executing the program.

[0076] This embodiment also provides a computer program product, including a computer program, which implements the node classification method based on the improved graph Transformer model when executed by a processor.

[0077] The node classification method based on the improved graph Transformer model proposed in the present invention aims to solve the problems of high computational complexity faced by the existing graph Transformer model when processing large graph data through a series of innovative designs, so as to maintain or even improve the performance of the model while reducing the computational complexity of the model's self-attention. Specifically, this model focuses on optimizing the self-attention calculation mechanism, by introducing graph data clustering partitioning based on the METIS algorithm, adjusting the sampling method of the self-attention mechanism operation nodes, and introducing a similarity-based operation node screening mechanism, thereby reducing the number of nodes involved in the self-attention calculation, thereby greatly reducing the computational complexity of the model, and by reducing the interference of irrelevant information by representing the node feature calculation based on a simple graph convolutional neural network and virtual nodes, thereby improving the performance of the model.

[0078] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0079] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A node classification method based on an improved graph Transformer model, characterized in that: The following steps are involved: Step 1: Cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k }={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k; Step 2: Generate a virtual representative node for each subgraph, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node features; Step 3: Input the node feature information into the improved graph Transformer model for encoding calculation; the improved graph Transformer model specifically includes adjusting the sampling method of the self-attention mechanism operation nodes and introducing a similarity-based operation node screening mechanism; Step 4: Input the final node vector representation obtained by encoding calculation into the multi-layer perceptron for node classification operation to obtain the classification result.

2. The node classification method based on the improved graph Transformer model according to claim 1, characterized in that: In step 1, the METIS algorithm is used to cluster and partition the input graph structure source data containing the nodes to be classified. The specific implementation includes the following sub-steps: Step 1.1: Using a multi-level approach, first construct the graph structure of the initial level, retain the node and edge information of the original graph, then reorder the nodes and edges of each level of the graph, and gradually construct a high-level graph until the preset number of levels or node number threshold is reached; Step 1.2: Construct a coarse-grained graph representation by merging nodes and edges, including aggregating nodes according to their degree and similarity to form supernodes, and aggregating edges according to their weights and connectivity to form superedges. Step 1.3: Recursively divide the coarse-grained graph into two subgraphs step by step until the preset number of subgraphs is reached, and determine the nodes and edges in each subgraph; Step 1.4: Refine and optimize the partitioning results to ensure that the boundaries between each subgraph are minimized; optimize the boundary nodes across subgraphs through the optimization algorithm, adjust the node attribution to reduce the number of cross-subgraph edges, and optimize the nodes within the subgraph to improve the connectivity and compactness of the subgraph; Step 1.5: Convert the partitioning result back to the representation of the original graph structure; output the subgraph number to which each node belongs, and the information of the nodes and edges contained in each subgraph; the output format includes the correspondence between the node number and the subgraph number, as well as the node list and edge list in each subgraph.

3. The node classification method based on the improved graph Transformer model according to claim 1, characterized in that: The specific implementation of step 2 includes the following sub-steps: Step 2.1: For subgraph G i , introduce a unique virtual node VN i , the virtual node and the subgraph G i Establish connections among all nodes in the Step 2.2: Input the subgraph with virtual nodes added into the multi-layer simple graph convolutional neural network for propagation calculation. During the propagation calculation process, information is exchanged with all nodes in the subgraph. According to the formula Calculate the output of virtual node features at each layer of the simple graph convolutional neural network, and use the output of the last layer as the virtual representative node feature of the subgraph; is the feature vector of node u in layer l, |G i | is the subgraph G i The number of nodes in W (l) is the weight matrix of the lth layer, and σ is the activation function.

4. The node classification method based on the improved graph Transformer model according to claim 1, characterized in that: In step 3, the improved graph Transformer model includes a neighbor node sampling module, a similarity-based node screening module, a position encoding module, a multi-head self-attention calculation module and a feedforward network module; The neighbor node sampling module is used to obtain a set of neighbor node feature information corresponding to each node participating in subsequent self-attention calculation by sampling feature information of other nodes in the same partition and feature information of virtual representative nodes in other partitions for each node; The node screening module based on similarity is used to further screen neighbor nodes with high similarity to the target node; first, by calculating the cosine similarity between node feature information, the similarity score between each pair of nodes is output, and then the neighbor nodes with similarity higher than the threshold are selected according to the set threshold to participate in subsequent calculations; The position encoding module encodes the position information between nodes and adds it to the input node embedding representation and the filtered neighbor node embedding representation set. The node embedding representation after adding the position encoding is input to the multi-head self-attention calculation module. The multi-head self-attention calculation module includes a plurality of parallel self-attention calculation heads, each of which performs a linear transformation on the input feature information to generate a query Q, a key K and a value V vector, and then calculates the dot product of the query Q vector and the key K vector, and scales the dot product result by dividing it by the square root of the dimension of the key K vector, and then normalizes the scaled dot product result through a Softmax function and converts it into a probability distribution form, and multiplies the normalized attention weight by the value V vector to obtain a weighted output; finally, the outputs of all self-attention calculation heads are spliced ​​together to form a comprehensive output vector O; The feedforward network module is used to further process the output of the multi-head self-attention calculation module; firstly, the first residual connection and layer normalization operation are performed, the output vector O of the multi-head self-attention calculation module is added to the query vector Q inputted initially at the element level, and then normalized in the feature dimension; then the feedforward network layer is inputted, which includes two linear transformation layers and an activation layer therebetween, the first linear transformation layer transforms the input features, the activation layer introduces nonlinearity, and the second linear transformation layer generates output; finally, the second residual connection and layer normalization are performed, the output of the feedforward network layer is added to the input of the feedforward network layer at the element level, and the output is normalized in the feature dimension. .

5. The node classification method based on the improved graph Transformer model according to claim 1, characterized in that: In step 3, first according to the formula Calculate the cosine similarity between the node and other nodes in the same partition and other partition virtual representative nodes; where x u and x υ represent the feature vectors of nodes u and v respectively, ||x υ || and ||x υ || represents the Euclidean norm of a vector; Then the nodes are sorted according to the similarity, and only a certain proportion of nodes with the highest similarity are selected to participate in the subsequent self-attention calculation.

6. The node classification method based on the improved graph Transformer model according to any one of claims 1 to 5, characterized in that: In step 4, the final node vector representation obtained by encoding is input into the multi-layer perceptron for node classification operation. According to the formula Calculate the probability that node u belongs to category c; where y u is the true category of node u, C is the total number of categories, W c and b c are the weight vector and bias term associated with category c, h u Represents the final vector representation of the node obtained by encoding calculation.

7. A node classification system based on an improved graph Transformer model, characterized in that: Includes the following modules: The clustering and partitioning module is used to cluster and partition the input graph structure source data containing the nodes to be classified, divide the original large graph into several small subgraphs, and remap and arrange the node and edge data to obtain a series of subgraph data {G1, G2, …, G k }={(V1,E1),(V2,E2),…,(V k ,E k )}; where V i Represents the subgraph G i All internal nodes, E i Represents the subgraph G i All internal edges, 1≤i≤k; A subgraph virtual representative node feature acquisition module is used to generate a virtual representative node for each subgraph, aggregate the information of each subgraph into the virtual node, and obtain the subgraph virtual representative node feature; An encoding calculation module is used to input node feature information into an improved graph Transformer model for encoding calculation; the improved graph Transformer model specifically includes adjusting the sampling method of the operation nodes of the self-attention mechanism and introducing an operation node screening mechanism based on similarity; The node classification module is used to input the final node vector representation obtained by encoding calculation into the multi-layer perceptron to perform node classification operation and obtain the classification result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the node classification method based on the improved graph Transformer model as described in any one of claims 1 to 6 is implemented.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the node classification method based on the improved graph Transformer model as described in any one of claims 1 to 6 is implemented.

Citation Information

Cited By

  • Distance perception-based graph division retrieval method and system and readable storage medium

    CN119988694A