An adaptive gated graph neural network method and system for heterogeneous graph learning

Abstract

The application discloses a kind of self-adapting gated graph neural network methods and systems for heterophily graph learning, belong to graph machine learning and graph data mining technical field.The method is first respectively constructed node own feature encoder and neighborhood information encoder, and the independent extraction of node own information and neighborhood information is carried out;Then, two kinds of representations are mapped to a unified feature space, and two kinds of information are adaptively fused according to the dimension based on a learnable gating vector;Further, by simultaneously applying classification constraints to node own representation, neighborhood representation and fusion representation, a hybrid loss function is constructed to alleviate the false convergence problem caused by the inconsistent convergence speed of different encoding branches.The method can adaptively adjust the retention ratio of node own features and neighborhood information according to the homophily or heterophily of graph data, and has good robustness, interpretability and generalization ability in both heterophily graph and homophily graph scenarios.

Description

Technical Field

[0001] This invention relates to the fields of graph machine learning and graph data mining, and in particular to an adaptive gated graph neural network method and system for learning heterogeneous graphs. Background Technology

[0002] Graph-structured data is widely found in scenarios such as social networks, bioinformatics, recommender systems, knowledge graphs, and transportation networks. In recent years, graph neural networks have achieved good results in tasks such as node classification, graph classification, and link prediction by aggregating neighbor node information through message passing mechanisms. Existing methods such as graph convolutional networks, graph attention networks, and GraphSAGE mostly assume that graphs have strong isomatch, that is, connected nodes have high similarity in labels or attributes. Therefore, neighborhood aggregation can usually enhance node representation.

[0003] However, in many real-world scenarios, graph structures often exhibit significant heterogeneity, meaning that adjacent nodes may belong to different categories or have significantly different attributes. For example, in web networks, citation networks, and some knowledge graphs, the neighboring nodes connected to a node may not be semantically consistent with it. In such cases, traditional graph neural networks, by directly aggregating neighborhood messages, are prone to introducing noisy information inconsistent with the current node into the representation learning process, weakening the node's own attributes and resulting in a significant decrease in classification performance.

[0004] Existing improved methods for heterogeneous graphs typically enhance their adaptability to heterogeneous scenarios by expanding the aggregation range, constructing higher-order neighborhoods, or modifying message passing functions, but they still have the following shortcomings: 1) It fails to explicitly distinguish the different roles of node characteristics and neighborhood information, making it difficult to adaptively determine the importance of the two types of information based on specific graph scenarios; 2) During training, there are significant differences in the convergence speed of different coding branches. If only the final fused representation is used for supervision, the gating mechanism may prematurely favor the faster-converging branch, thus inhibiting the effective learning of the other branch. 3) Existing methods lack interpretability regarding how the model internally adjusts the degree of retention of the two types of information on different datasets. Summary of the Invention

[0005] The purpose of this invention is to provide an adaptive gated graph neural network method and system for learning heterogeneous graphs, so as to overcome the problems of traditional graph neural networks in the prior art, such as over-reliance on neighborhood information in heterogeneous graphs, difficulty in dynamically balancing node features and neighborhood information, and the tendency to spurious convergence during multi-branch training.

[0006] To achieve the above objectives, this invention provides an adaptive gated graph neural network method for learning heterogeneous graphs, comprising the following steps: S1. Obtain the graph data to be trained, and represent the graph data as graph structure data containing a set of nodes, a set of edges, a node feature matrix, and node labels; S2. Construct a node self-feature encoder and a neighborhood information encoder. The node self-feature encoder is used to extract the node's own attribute representation, and the neighborhood information encoder is used to aggregate the neighborhood structure information representation connected to the node. S3. Map the node's own attribute representation and neighborhood structure information representation to a common feature space of the same dimension to obtain the node's own representation and neighborhood representation; S4. Generate gated vectors based on learnable vectors, and use the gated vectors to perform dimension-wise adaptive fusion of the node's own representation and neighborhood representation to obtain the final fused representation of the node; S5. Construct classification branches based on the node's own representation, neighborhood representation, and final fused representation, and jointly train the model using a hybrid loss function; S6. Use the trained model to perform classification prediction on the target node.

[0007] Preferably, in S2, the node's own feature encoder uses a multilayer perceptron (MLP), and the neighborhood information encoder uses a graph convolutional network (GCN); the propagation process of the neighborhood information encoder is represented as follows: ; in, This represents the adjacency matrix after adding self-loops. for The corresponding degree matrix, Indicates the first Layer input feature matrix, Indicates the first Layer learnable parameters This represents a non-linear activation function.

[0008] Preferably, in S3, the node's own attribute representation and the neighborhood structure information representation have the same output dimension after linear transformation, so as to perform a dimension-by-dimensional fusion operation; the node's own representation is denoted as... The neighborhood is denoted as .

[0009] Preferably, in S4, the gate vector is obtained by performing a cumulative softmax operation on the learnable vector, specifically as follows: ; in, z For learnable vectors, softmax Used to normalize the components of a vector to a range of 0 to 1; cumsumUsed to perform cumulative summation on the normalized components to generate a monotonically increasing gated vector. .

[0010] Preferably, in S4, the final fused representation of the nodes It can be obtained through the following formula: ; Where ⊙ represents the Hadamard product, Used to control the node's own representation Representation of Neighborhood The retention ratio across different feature dimensions.

[0011] Preferably, in S5, the hybrid loss function is composed of a weighted sum of the node's own representation classification loss, the neighborhood representation classification loss, and the fusion representation classification loss, and is expressed as: ; in, , and For loss weight hyperparameters, Representation based on the node's own representation Classification loss, Representation based on neighborhood Classification loss, The representation is based on the final fusion representation. Classification loss.

[0012] Preferably, the , and Both use the cross-entropy loss function; among them, the classification loss based on the final fused representation is expressed as: ; in, Represents a set of nodes. Represents a node v The true label, The representation is based on the final fusion representation. The obtained predicted label distribution; Classification loss based on node self-representation and neighborhood representation and The calculation formulas are as follows: ; ; in, and These represent the predicted label distributions obtained based on the node's own representation and its neighborhood representation, respectively.

[0013] This invention also provides an adaptive gated graph neural network system for heterogeneous graph learning, comprising: The graph data input module is used to obtain graph structure data containing nodes, edges, node features, and node labels; The dual-branch encoding module is used to extract the node's own attribute representation and the neighborhood structure information representation, respectively. The unified mapping module is used to map two types of representations to a feature space of the same dimension; The gated fusion module is used to generate learnable gated vectors and perform dimension-adaptive fusion of the two types of representations; The joint training module is used to construct a hybrid loss function based on the node's own representation, neighborhood representation, and fused representation, and to complete the model training. The prediction output module is used to output the classification results of the target node.

[0014] The present invention also provides a computer program product that, when run on a computer, causes the computer to execute the adaptive gated graph neural network method for heterogeneous graph learning.

[0015] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned adaptive gated graph neural network method for heterogeneous graph learning.

[0016] Therefore, the above-mentioned adaptive gated graph neural network method and system for heterogeneous graph learning has the following beneficial effects: 1) It can explicitly separate and independently model the node's own features and neighborhood information, avoiding excessive interference from neighborhood noise on node representation in heterogeneous graph scenarios; 2) By using learnable gating vectors to adaptively fuse the two types of information in a dimension-wise manner, the information retention ratio is dynamically adjusted according to different graph data and different task scenarios, thereby improving the model's compatibility with heterogeneous and homogeneous graphs. 3) By introducing a hybrid loss function consisting of node self-branch, neighborhood branch, and fusion branch, the false convergence problem caused by the difference in convergence speed among different encoders is alleviated, thereby improving training stability and final accuracy. 4) Gated vectors are interpretable and can reflect the model’s preference for node information and neighborhood information on different datasets, which facilitates the analysis of the model’s decision-making basis.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the overall structure of the adaptive gated graph neural network method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of generating gated vectors using cumulative softmax in an embodiment of the present invention; Figure 3 This is a schematic diagram of the distribution of gated vectors learned on different isomatch graph datasets according to an embodiment of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the preceding element covers the following element and its equivalents, but do not exclude the presence of other elements.

[0021] Example 1 like Figure 1 As shown, This embodiment provides an adaptive gated graph neural network method for learning heterogeneous graphs, and its overall architecture is as follows: Figure 1 As shown, it mainly includes three parts: dual-branch coding, adaptive gating fusion, and hybrid loss training. The specific steps are as follows.

[0022] S1. Obtain the graph data to be trained and represent the graph data as graph G=(V,E,X,Y), where V represents the set of nodes, E represents the set of edges, X represents the node feature matrix, and Y represents the set of node labels.

[0023] In this method, the graph data can be citation networks, webpage networks, knowledge graph subgraphs, or other graph structure data with node attributes and edge relationships. To verify the generality of this method, both identical and dissimilar graph datasets are selected for training and testing, such as the Cora, PubMed, CiteSeer, Texas, Squirrel, and Chameleon datasets.

[0024] S2. Construct a node self-feature encoder and a neighborhood information encoder, which are used to extract the node's own attribute representation and aggregate the neighborhood structure information representation connected to the node, respectively.

[0025] In this embodiment, the node self-feature encoder uses a multilayer perceptron (MLP) to perform a nonlinear transformation on the input node features to obtain the node's own attribute representation. Since the MLP does not perform graph neighborhood aggregation, it is more conducive to preserving the discriminative information in the original node attributes.

[0026] The neighborhood information encoder employs a graph convolutional network (GCN) to aggregate neighboring node information on a graph structure. The propagation process of this encoder is represented as follows:

[0027] in, The adjacency matrix after adding self-loops, for The corresponding degree matrix, For the first Layer input feature matrix, For the first Layer learnable parameters This is a non-linear activation function. Through this propagation process, the neighborhood structure information representation of the node is obtained.

[0028] S3. Map the node's own attribute representation and neighborhood structure information representation to a common feature space of the same dimension to obtain the node's own representation and neighborhood representation.

[0029] In this embodiment, the MLP output and GCN output are mapped to the same dimension d through a linear transformation layer to obtain nodes. v The corresponding node's own representation and neighborhood representation Unifying the two types of representations to the same feature dimension facilitates the execution of dimension-wise weighted control in the subsequent fusion stage.

[0030] S4, such as Figure 2 As shown, learnable gated vectors are generated, and the node's own representation and neighborhood representation are adaptively fused by dimension to obtain the final fused representation of the node.

[0031] In this embodiment, learnable vectors are first defined. z and execute it. softmax Operations to obtain the sum of elements The normalized vector; then the normalized result is processed. cumsum Accumulated summation generates a monotonically increasing gate vector. ,Right now: ; Using gated vectors Representation of the node itself and neighborhood representation Dimensional fusion is performed to obtain the final fused representation. : ; Where ⊙ represents the Hadamard product, Used to control the node's own representation Representation of Neighborhood The retention ratio across different feature dimensions.

[0032] Compared to direct concatenation or scalar weighted summation, the dimension-wise gating method used in this embodiment allows the model to retain more node-specific attributes in some dimensions and more neighborhood information in others, thereby improving the flexibility of representation learning in heterogeneous graph scenarios. Especially in heterogeneous graphs, adjacent nodes may differ significantly from the central node; in this case, the gating vector can adaptively increase the retention of node-specific features. Conversely, in homogeneous graphs, the gating vector can increase the utilization of neighborhood information.

[0033] S5. Classification branches are constructed based on the node's own representation, neighborhood representation, and final fused representation, respectively, and trained using a hybrid loss function.

[0034] In this embodiment, after obtaining , and Then, prediction results based on the node's own representation, prediction results based on the neighborhood representation, and prediction results based on the fused representation are obtained through classifiers, respectively. The corresponding classification losses are defined as follows: , and Both methods employ the cross-entropy loss function. The classification loss of the fusion representation is expressed as: ;in, Represents a set of nodes. Represents a node v The true label, The representation is based on the final fusion representation. The resulting predicted label distribution.

[0035] Classification loss based on node self-representation and neighborhood representation and The calculation formulas are as follows: ; ; in, and These represent the predicted label distributions obtained based on the node's own representation and its neighborhood representation, respectively.

[0036] Furthermore, a hybrid loss function is constructed: ; in, , and This is a hyperparameter used to balance the three parts of the loss. In this embodiment, , and Set to 1, 100, and 1 respectively. By simultaneously supervising the node's own branch, neighborhood branch, and fusion branch, the gating mechanism can be prevented from prematurely favoring a branch in the early stages of training simply because that branch converges faster, thus alleviating the problem of false convergence.

[0037] In this embodiment, the validity analysis of the experimental results was performed based on the constructed graph dataset, including the following:

[0038] 1) Comparative analysis with existing technologies.

[0039] The model constructed by this method is compared with existing graph learning methods, including MLP methods that only utilize the features of the nodes themselves, classic graph neural network methods GCN, GAT, and GraphSAGE for homographs, and MixHop, GEOM-GCN, and GPRGNN methods designed for heterograph scenarios.

[0040] The results show that the proposed AG2NN method outperforms the aforementioned comparative methods in classification accuracy on heterogeneous image datasets such as Texas, Squirrel, and Chameleon; meanwhile, it maintains near-optimal performance on homogeneous image datasets such as CiteSeer, Pubmed, and Cora, indicating that the proposed method can not only effectively handle heterogeneous image scenarios but also take into account representation learning capabilities in homogeneous image scenarios. Specific results are shown in Table 1.

[0041] Table 1: Overall performance comparison of different methods on the same-map and dissimilar-map datasets.

[0042] As shown in Table 1, on the three heterograph datasets (Texas, Squirrel, and Chameleon), the classification accuracy of this model reached 82.46%, 55.07%, and 69.14%, respectively, which is a significant improvement over the baseline method (e.g., significantly better than GraphSAGE's 41.61% and 58.73% on Squirrel and Chameleon). Meanwhile, on the three homograph datasets (Citeceer, Pubmed, and Cora), this model also achieved accuracies of 77.14%, 89.15%, and 87.37%, respectively, placing it in the same tier as high-performing models such as GEOM-GCN and GPRGNN.

[0043] 2) Interpretability analysis of adaptive gating vectors.

[0044] To verify the effectiveness of the gating fusion mechanism in this method, the gating vectors learned on different datasets were visualized and analyzed. The results are as follows: Figure 3As shown in the figure. The analysis results show that on heterogeneous mapping datasets such as Texas and Squirrel, the gate vectors have larger values ​​in many dimensions, indicating that the model tends to preserve the node's own features in this type of scenario; while on homogeneous mapping datasets such as CiteSeer and Cora, the gate vectors have smaller values ​​in many dimensions, indicating that the model tends to utilize neighborhood information in this type of scenario.

[0045] The above phenomena indicate that the adaptive gating mechanism proposed in this method can dynamically adjust the contribution ratio of node features and neighborhood information according to the homo-match or hetero-match of different graph data, thereby improving the final classification effect.

[0046] 3) Ablation experiment analysis of key modules.

[0047] To further verify the effectiveness of the gating fusion mechanism and the hybrid loss training strategy, this embodiment sets up multiple ablation comparison models, including: a model that does not introduce adaptive gating vectors and only uses a simple concatenation method for information fusion; a model that introduces a gating mechanism but does not use a hybrid loss function; and a model that does not use a gating mechanism but retains a multi-loss training strategy.

[0048] The experimental results are shown in Table 2. The complete AG2NN model outperforms various ablation variants on multiple datasets. However, if the fusion loss is removed and only the final fused representation is used for supervision, the model tends to prematurely favor the faster-converging node-specific feature branches, resulting in a significant performance drop on heterogeneous graph datasets such as Squirrel and Chameleon. If no gating mechanism is used and the node-specific representation is directly concatenated with the neighborhood representation, the model struggles to adaptively coordinate the effects of the two types of information according to different scenarios, and its overall performance is also inferior to the complete model.

[0049] Table 2: Performance comparison of different ablation variants of AG2NN on various datasets

[0050] Specifically, as shown in Table 2, the complete AG2NN model achieved the best performance across all datasets. After removing the multi-loss, the model's accuracy on the heterogeneous graph datasets Squirrel and Chameleon plummeted to 27.87% and 43.84%, respectively, significantly lower than the complete model's 55.07% and 69.14%. This clearly demonstrates the crucial role of the multi-loss in mitigating spurious convergence. Furthermore, without the gate mechanism, the model's accuracy on the Chameleon dataset also decreased from 69.14% to 63.09%. The most basic simple concatenation variant (AG2NN-naive) performed the worst, achieving only 26.65% accuracy on Squirrel. These specific data comparisons fully demonstrate the necessity of the module design.

[0051] 4) Adaptability analysis for different graph structure scenarios.

[0052] This embodiment further compares the performance of the method of the present invention in two typical scenarios: identical and dissimilar graphs. (Combined with the performance data in Table 1...) Figure 3 Further analysis of the gating vector distribution: Under heterogeneous mapping conditions (such as Texas and Squirrel), such as Figure 3 As shown, the gating vector has large values ​​in many dimensions. This method suppresses the interference of inconsistent neighborhood messages on the classification results by increasing the retention ratio of the node's own features, thus achieving a significant improvement in classification accuracy as shown in Table 1 (such as an accuracy of 55.07% on the Squirrel dataset); while under the same map conditions (such as CiteSeer and Cora). Figure 3 The gating vectors in the graph have relatively small values ​​across multiple dimensions, indicating that this method can adaptively and effectively utilize the structural prior information brought about by neighborhood aggregation, thereby maintaining near-optimal classification accuracy. The results in the above figures corroborate each other, fully demonstrating that this model has a strong adaptive capability to different graph structure scenarios.

[0053] Therefore, this invention adopts the aforementioned adaptive gated graph neural network method and system for learning heterogeneous graphs. Addressing the performance degradation, inflexible information fusion, and spurious convergence issues of traditional graph neural networks in heterogeneous graph scenarios, this invention employs three core technologies: independent encoding with two branches, dimensional adaptive gating fusion, and three-branch hybrid loss training. This improves classification accuracy on typical heterogeneous graph datasets compared to the best method, while maintaining near-optimal performance on homogeneous graph datasets. It effectively alleviates the spurious convergence problem, and the gating vectors possess good interpretability. The model's versatility, robustness, and generalization ability are significantly enhanced.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An adaptive gated graph neural network method for learning heterogeneous graphs, characterized in that, Includes the following steps: S1. Obtain the graph data to be trained, and represent the graph data as graph structure data containing a set of nodes, a set of edges, a node feature matrix, and node labels; S2. Construct a node self-feature encoder and a neighborhood information encoder. The node self-feature encoder is used to extract the node's own attribute representation, and the neighborhood information encoder is used to aggregate the neighborhood structure information representation connected to the node. S3. Map the node's own attribute representation and neighborhood structure information representation to a common feature space of the same dimension to obtain the node's own representation and neighborhood representation; S4. Generate gated vectors based on learnable vectors, and use the gated vectors to perform dimension-wise adaptive fusion of the node's own representation and neighborhood representation to obtain the final fused representation of the node; S5. Construct classification branches based on the node's own representation, neighborhood representation, and final fused representation, and jointly train the model using a hybrid loss function; S6. Use the trained model to perform classification prediction on the target node.

2. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 1, characterized in that, In S2, the node's own feature encoder uses a multilayer perceptron (MLP), and the neighborhood information encoder uses a graph convolutional network (GCN); the propagation process of the neighborhood information encoder is represented as follows: ； in, This represents the adjacency matrix after adding self-loops. for The corresponding degree matrix, Indicates the first Layer input feature matrix, Indicates the first Layer learnable parameters This represents a non-linear activation function.

3. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 2, characterized in that, In S3, the node's own attribute representation and the neighborhood structure information representation have the same output dimension after linear transformation, so as to perform a dimension-by-dimensional fusion operation; the node's own representation is denoted as... The neighborhood is denoted as .

4. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 3, characterized in that, In S4, the gate vector is obtained by performing a cumulative softmax operation on the learnable vector, specifically as follows: ； in, z For learnable vectors, softmax Used to normalize the components of a vector to a range of 0 to 1; cumsum Used to perform cumulative summation on the normalized components to generate a monotonically increasing gated vector. .

5. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 4, characterized in that, In S4, the final fused representation of the nodes It can be obtained through the following formula: ； Where ⊙ represents the Hadamard product, Used to control the node's own representation Representation of Neighborhood The retention ratio across different feature dimensions.

6. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 1, characterized in that, In S5, the hybrid loss function is composed of a weighted sum of the node's own representation classification loss, the neighborhood representation classification loss, and the fusion representation classification loss, and is expressed as: ； in, , and For loss weight hyperparameters, Representation based on the node's own representation Classification loss, Representation based on neighborhood Classification loss, The representation is based on the final fusion representation. Classification loss.

7. The adaptive gated graph neural network method for heterogeneous graph learning according to claim 6, characterized in that, The , and Both use the cross-entropy loss function; among them, the classification loss based on the final fused representation is expressed as: ;in, Represents a set of nodes. Represents a node v The true label, The representation is based on the final fusion representation. The obtained predicted label distribution; Classification loss based on node self-representation and neighborhood representation and The calculation formulas are as follows: ； ； in, and These represent the predicted label distributions obtained based on the node's own representation and its neighborhood representation, respectively.

8. An adaptive gated graph neural network system for learning heterogeneous graphs, characterized in that, include: The graph data input module is used to obtain graph structure data containing nodes, edges, node features, and node labels; The dual-branch encoding module is used to extract the node's own attribute representation and the neighborhood structure information representation, respectively. The unified mapping module is used to map two types of representations to a feature space of the same dimension; The gated fusion module is used to generate learnable gated vectors and perform dimension-adaptive fusion of the two types of representations; The joint training module is used to construct a hybrid loss function based on the node's own representation, neighborhood representation, and fused representation, and to complete the model training. The prediction output module is used to output the classification results of the target node.

9. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the adaptive gated graph neural network method for heterogeneous graph learning as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the adaptive gated graph neural network method for heterogeneous graph learning as described in any one of claims 1-7.

Images