A graph node classification method based on ensemble learning and graph feature self-attention mechanism

Abstract

This invention relates to a graph node classification method based on a graph feature self-attention mechanism using ensemble learning. First, the original graph network data is input, including the node feature matrix H and the graph adjacency matrix A. The LightGBM method is used to preprocess the dataset to obtain model parameters. Finally, the expanded node feature matrix H is obtained by summarizing the tree model parameters. new ; Combine the graph adjacency matrix A and the expanded node feature matrix H new The input graph features are trained on an attention network, and the network's preference information for nodes and features is obtained after training. Finally, the trained model is used to predict the classification of the graph node dataset on the validation and test sets. This invention utilizes an ensemble learning method and incorporates node attention preferences for features, which not only efficiently leverages the high interpretability of tree models but also enriches the model's expression, allowing node features to contain more information, thus contributing to higher performance on node classification tasks.

Description

Technical Field

[0001] This invention belongs to the field of information mining technology, specifically relating to a graph node classification method based on a graph feature self-attention mechanism of ensemble learning. Background Technology

[0002] In recent years, Generative Neural Networks (GNNs) have been widely applied to various graph analysis tasks, including node-centric tasks (such as node classification and link prediction) and graph-centric tasks (such as graph similarity detection and graph classification). Among these, node classification has become one of the most typical research directions in graph analysis due to its wide range of applications. Specifically, the goal of node classification is to predict the specific class of each unlabeled node in a graph based on graph information. For example, node classification can predict the research topic to which each article belongs in a citation network. In protein-protein interaction networks, each node can be assigned several gene ontology types. Graph neural networks that can be used for node classification are mainly divided into three categories: convolutional mechanisms, attention mechanisms, and autoencoder mechanisms.

[0003] Attention mechanisms are among the most useful architectures in artificial intelligence, including computer vision and graph analysis. Different neighbors should contribute differently to the target node, rather than using constant weights. Furthermore, the number of neighbors for each node is different. The advantage of attention mechanisms is their ability to handle variable inputs and focus on the most relevant parts. Therefore, applying attention mechanisms in node classification is natural. Veliˇckovi′c et al. introduced an attention-based graph convolutional layer to capture the different neighborhood importance of the target node while generating higher hidden representations, and then constructed an arbitrary-depth graph attention network (GAT) by stacking these attention layers. Similarly, Kiran et al. proposed a novel attention-based graph neural network (AGNN) with dynamically adaptive propagation layers, and then used it to handle cases where supervised information is scarce in the original data.

[0004] For node classification tasks, the better the generated node embedding information, the more significant the improvement in classification performance. Therefore, various graph network models are currently researching how to utilize the connections between nodes to generate better node embedding information. Graph Convolutional Networks (GCNs) approximate the aggregation of neighbor node information through adjacency matrix normalization, with the core being the aggregation of node features using the adjacency matrix. Graph Attention Networks (GATs) represent node features by calculating the importance of neighboring nodes, also exploring how to more efficiently aggregate neighbor node information. Later graph network models, including SGCN, GrpahSage, ElsticGNN, and APPNP, all focus on improving message passing and neighbor aggregation methods in graphs.

[0005] While the aforementioned models have achieved good performance, their research focuses on graph structures and lacks attention to feature analysis. For node classification tasks, generating better node embeddings undoubtedly leads to higher algorithm performance. Therefore, extracting and enhancing features separately may help enrich node embeddings and thus improve algorithm performance. Models with strong feature processing capabilities, such as the GBDT series in ensemble learning, the DeepFM series in deep learning, and the latest Transformer series, can utilize not only their outputs but also leaf nodes and some parameter matrix information from the model. These elements can significantly enhance the features of graph nodes, enabling the network to learn better node feature representations and thus achieve better classification results. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a graph node classification method based on ensemble learning and graph feature self-attention mechanism. This method makes great use of some results in the tree model, enhances the feature representation of nodes, and improves the classification ability of the model.

[0007] The technical problem solved by this invention is achieved through the following technical solution:

[0008] A graph node classification method based on graph feature self-attention mechanism using ensemble learning, characterized by the following steps:

[0009] S1. Obtain raw data from a publicly available graph network dataset, including the node feature matrix H and the graph adjacency matrix A;

[0010] S2. The classic LightGBM algorithm model is used to predict the feature matrix H and the graph adjacency matrix A of the nodes.

[0011] S3. Use the LightGBM algorithm from step S2 to integrate the learning model and enhance the node features;

[0012] S4. Merge the tree model parameters summarized in step S3 into the node matrix features to obtain a new node feature matrix H. new That is, merging the feature vectors of all nodes into H. new =[h 1,new ,h 2,new ,...,h n,new ], where n is the number of samples;

[0013] S5. Construct a graph feature self-attention network;

[0014] S6. The node feature matrix H obtained in step S4 new The graph adjacency matrix A is input into the graph feature self-attention network model in step S5 for training;

[0015] S7. Combine the two models in steps S2 and S5;

[0016] S8. Apply the two-stage models in steps S2 and S5 above to the graph node data to calculate the accuracy of node classification.

[0017] Moreover, step S2 is specifically as follows:

[0018] S201. Initialize the decision tree model. Assume there are a total of Tr trees. When iterating the t-th (1 < t ≤ Tr) tree each time, the model f(h) is updated in an additive manner:

[0019] f t (H) = f t-1 (H) + ∈g t (H)

[0020] where: f t-1 is the model constructed in the previous iteration;

[0021] f t is the model constructed in this iteration;

[0022] g t is a weak learner selected from a certain function family H;

[0023] ∈ is the learning rate;

[0024] H is the input feature of the sample;

[0025] Select the weak learner to approximate the negative gradient of the loss function L;

[0026] S202. The prediction of the decision tree model is as follows:

[0027]

[0028] where: h i represents the feature vector of the i-th sample;

[0029] S203. For one of the decision trees, the calculation of the final result depends on which leaf node the input sample is classified into. Let I k be the set of samples at the leaf node where the output result of the k-th tree is located. Therefore, the output result of the k-th tree can be expressed as where: w i represents the value of the leaf node in the I k set; Therefore, the final output of this ensemble learning method is where: α k represents the weight of the k-th weak learner.

[0030] Furthermore, step S3 specifically involves: for each input sample, the weighted sum of the results from each weak learner in the model is obtained, resulting in a total of Tr weak learners. Finally, a Tr+n dimensional feature extension is obtained, where n is the number of categories in the classification task, including the results from the Tr weak learners and the final output of the model. That is, the new features become:

[0031]

[0032] Wherein: the middle part represents the weighted sum of the leaf nodes of each weak learner in the model, forming a feature vector, and the right part represents the classification result represented by one-hot encoding.

[0033] Moreover, step S5 specifically includes:

[0034] S501. When aggregating information at the central node, a first-level attention mechanism is first applied to neighboring nodes.

[0035]

[0036] Where: α ij This represents the attention coefficient of the central node i to its neighbor node j;

[0037] LeakReLU is an activation function;

[0038] a:R F ×R F →R is a single-layer negative feedback neural network that is used to calculate the attention coefficient;

[0039] W∈R F× F is the parameter matrix used for feature transformation, and F is the feature dimension of each node;

[0040] h i ∈R F h j ∈R F These are the feature vectors of the center node i and the neighbor node j, respectively, where 0 ≤ i, j < N, and N represents the number of nodes;

[0041] S502, Replace the existing feature h with the node feature generated in step S3. i =h i-old ||s i , where h i-old Let s be the original input of the node features. i =s i,left ||s i,right =[α 1 s 1 ,α 2 s 2 ,...,αt s t ]||[0,1,...,0];

[0042] S503. First, initialize a hidden vector vi∈R for each feature dimension. F Secondly, a negative feedback neural network with a scalar output is used to calculate the attention score b of each node for each feature dimension. ij =att(h i ||v j ), where: the parameters within the att network are of specification (F+F')×1, and the output is a scalar;

[0043] S504. To make the coefficients easier to compare across different features, the softmax function is used to normalize them across all choices of j:

[0044]

[0045] Where: β ij The normalized attention coefficient;

[0046] S505. Obtain the normalized attention coefficients, which will be used to calculate a linear combination of the features corresponding to them, as the output feature of each node. The feature update for a single node is as follows:

[0047]

[0048] Where: βi∈R F This represents the attention coefficient vector of node i for all features, and it shows how the node's feature information is updated.

[0049] S506. After passing through the graph feature attention network, multi-classification is performed using the softmax function:

[0050] y = softmax(DNN(h) i ,W))

[0051] Where: y∈R class×1 Based on the classification results, the loss function is:

[0052]

[0053] Where: Y ij ∈R n×class For the predicted results;

[0054] n is the number of samples;

[0055] `class` represents the number of categories;

[0056] Z ij ∈Rn×class This is a real label.

[0057] Furthermore, step S8 specifically involves: calculating and evaluating the accuracy rate. Where N pos This represents a positive sample, and N represents the total number of samples.

[0058] The advantages and beneficial effects of this invention are as follows:

[0059] 1. This invention is a graph node classification method based on the graph feature self-attention mechanism of ensemble learning. By combining the parameters in the tree model and the graph feature attention mechanism, it not only makes great use of the high interpretability of the tree model, but also enriches the node's preference information for each feature, thereby further improving the classification accuracy of the graph dataset.

[0060] 2. The graph node classification method based on the graph feature self-attention mechanism of ensemble learning in this invention performs latent vector embedding on all features of the node, so that each feature has more information representation, which can greatly enhance the model's expressive capacity and effectively improve the model performance.

[0061] 3. The graph node classification method based on the graph feature self-attention mechanism of ensemble learning in this invention incorporates the attention preference of each node for each dimension of features, making the model pay more attention to the information that is more important to the model's classification task. This not only enhances the model's expressive power and interpretability, but also improves the model's classification accuracy.

[0062] 4. This invention is a graph node classification method based on the graph feature self-attention mechanism of ensemble learning. It combines ensemble learning and graph attention network to form a two-stage method, which utilizes the feature processing capability of tree model and the node data carrying graph topology information of graph network. This combines the advantages of the two sub-models and helps to improve the classification accuracy of the model.

[0063] 5. The graph node classification method based on the graph feature self-attention mechanism of ensemble learning in this invention has better performance, higher classification accuracy, more accurate extraction of node feature information of graphs, and richer information in the generated node embeddings. It can provide better information completion capabilities for information mining fields including social networks, knowledge graphs, citation networks, etc. Attached Figure Description

[0064] Figure 1 This is a flowchart of the present invention;

[0065] Figure 2The following are visualizations of the classification results of the present invention and the comparison method: (a) is the t-sne visualization of GCN on the Cora dataset, (b) is the t-sne visualization of GCN on the Citeseer dataset, (c) is the t-sne visualization of GAT on the Cora dataset, (d) is the t-sne visualization of GAT on the Citeseer dataset, (e) is the t-sne visualization of the EL-GFAT of the present invention on the Cora dataset, and (f) is the t-sne visualization of the EL-GFAT of the present invention on the Citeseer dataset.

[0066] Figure 3 The images show the iterative convergence graphs of the present invention and the comparison method. (a), (b), and (c) are comparison graphs of the iterative convergence graphs of the present invention's EL-GFAT and GCN on the Cora, Citeseer, and PubMed datasets, respectively. (d), (e), and (f) are comparison graphs of the iterative convergence graphs of the present invention's EL-GFAT and GAT on the Cora, Citeseer, and PubMed datasets, respectively. Detailed Implementation

[0067] The present invention will be further described in detail below through specific embodiments. The following embodiments are merely descriptive and not limiting, and should not be used to limit the scope of protection of the present invention.

[0068] This invention provides a graph node classification method based on a graph feature self-attention mechanism using ensemble learning, comprising:

[0069] S1. Obtain data: Obtain raw data from publicly available graph network datasets.

[0070] S2, LightGBM prediction processing: The feature matrix H of the nodes and the graph adjacency matrix A are predicted using the classic LightGBM model.

[0071] S201. Initialize the decision tree model with a total of t trees. In each iteration t, the model f(h) is updated additively:

[0072] f t (H)=f t-1 (H)+∈g t (H)

[0073] Where f t-1 The model was built in the previous iteration, g t From a certain family of functions The weak learner is selected, where ∈ is the learning rate and H is the input feature of the sample. Selecting a weak learner To approximate the negative gradient of the loss function L;

[0074] S202, The model's predictions are as follows:

[0075]

[0076] Where h i Let represent the feature vector of the i-th sample. The gradient of the current prediction indicates how these predictions should be changed to improve the loss function. Informally, gradient boosting can be considered as performing gradient descent in the function space;

[0077] S203. For one of the decision trees, the final result depends on which leaf node the input sample is classified into. Let I... k Let be the set of samples of the leaf nodes containing the output of the k-th tree. Therefore, the output of the k-th tree can be represented as...

[0078] Where: w i Indicate I k The values ​​of the leaf nodes in the set;

[0079] Therefore, the final output of this ensemble learning method is: Where α k This represents the weight of the k-th weak learner.

[0080] S3, Feature Expansion

[0081] The node features are enhanced by utilizing the ensemble learning model from step S2. For each input sample, the result of a weighted sum of the values ​​from each weak learner in the model is obtained. Since there are a total of Tr weak learners, the final result is a Tr+n dimensional feature expansion (where n is the number of categories in the classification task), including the results of the Tr weak learners and the final model output. That is, the new features become:

[0082]

[0083] The middle part represents the weighted sum of the leaf nodes of each weak learner in the model, forming a feature vector, while the right part represents the classification result represented by one-hot encoding.

[0084] S4. Construct a new feature matrix

[0085] After summarizing the tree model, merge all the node feature vectors into a new feature matrix H. new =[h 1,new ,h 2,new ,...,h n,new ], where n is the number of samples.

[0086] S5. Construct a graph feature self-attention network

[0087] S501. When aggregating information at the central node, a first-level attention mechanism is first applied to neighboring nodes.

[0088]

[0089] Where α ij This represents the attention coefficient of node i to its neighbor j. LeakReLU is an activation function, where a = R F ×R F →R is a single-layer negative feedback neural network that serves to calculate the attention coefficient; W∈R F×F This is the parameter matrix used for feature transformation, where F is the feature dimension of each node; h i ∈R F h j ∈R F These are the feature vectors of the center node i and the neighbor node j, respectively, where 0 ≤ i, j < N, and N represents the number of nodes;

[0090] S502, Replace the existing feature h with the node feature generated in step S3. i =h i-old ||s i , where h i-old Let s be the original input of the node features. i =s i,left ||s i,right =[α 1 s 1 ,α 2 s 2 ,...,α t s t ]||[0,1,...,0];

[0091] S503. First, initialize a hidden vector vi∈R for each feature dimension. F Secondly, a negative feedback neural network with a scalar output is used to calculate the attention score b of each node for each feature dimension. ij =att(h i ||v j ), where the parameters of the att network are sized as (F+F')×1, and the output is a scalar.

[0092] S504. To make the coefficients easier to compare across different features, we use the softmax function to normalize them over all choices of j:

[0093]

[0094] Where β ij The normalized attention coefficient;

[0095] S505. Once the normalized attention coefficients are obtained, they will be used to calculate a linear combination of the features corresponding to them, which will serve as the output features for each node. The feature update for a single node is as follows:

[0096]

[0097] Where βi∈R F This represents the attention coefficient vector of node i for all features. In summary, we can obtain the update method for node feature information.

[0098] S506. After passing through the graph feature attention network, multi-classification is performed using the softmax function:

[0099] y = softmax(DNN(h) i ,W))

[0100] Where y∈R class×1 Based on the classification results, the loss function is:

[0101]

[0102] Where Y ij ∈R n×class The prediction result is given, where n is the number of samples, class is the number of classification categories, and Z is the number of categories. ij ∈R n×class This is a real label.

[0103] S6, Model Training

[0104] The newly obtained feature matrix H new The adjacency matrix A is input into the S5 model for training.

[0105] S7. Merge Model: Merge the two sub-models S2 and S5 into a single whole;

[0106] S8. Node classification and accuracy calculation: Apply the above two-stage model to the graph node data to evaluate the node calculation accuracy. Where N pos This represents a positive sample, and N represents the total number of samples.

[0107] With the development of graph networks, the accuracy requirements for node classification tasks are becoming increasingly stringent. However, most existing models fail to adequately mine node feature information and do not fully utilize node feature information. This invention first expands the features by combining highly interpretable model parameters of an ensemble learning model, and then enhances the node feature representation by incorporating node attention preferences for features. This greatly utilizes some results from the tree model, enhances the feature representation of nodes, and improves the model's classification ability.

[0108] Example 1

[0109] The advantages of this invention will be illustrated below by comparing its accuracy with that of existing graph network methods:

[0110] Experimental conditions

[0111] The equipment used in the experiment was a machine with a 64-core E5-2682 v4@2.50GHz processor, 128GB of memory and a TiTAN Xp GPU. The programming platform was PyCharm. All experiments in this paper were run on this machine.

[0112] The datasets used in the experiment were Cora, Citeseer and PubMed citation datasets. Only 20 nodes were used for training in each category, and the remaining 1000 nodes were used for the testing phase. The specific dataset parameters are shown in Table 1.

[0113] Table 1 Node Classification Dataset

[0114] Dataset Number of nodes Number of sides Feature number category Training / Test Nodes Cora 2708 5278 1433 7 140 / 1000 Citeseer 3327 4552 3703 6 120 / 1000 PubMed 19,717 44,324 500 3 60 / 1000

[0115] Experimental content

[0116] In addition to the classic GCN, GAT, GraphSage, and APPNP, this invention also includes seven other methods for comparison: Attention-based Graph Neural Networks (AGNN), Elastic Graph Neural Networks (ElasticGNN), and Multi-View Graph Convolutional Networks with Attention Mechanisms (MAGCN-3Views).

[0117] The comparative experiments included the seven comparative methods proposed in the previous paragraph and the proposed method EL-GFAT. For each experiment, we calculated the average performance and standard deviation for 10 runs. For all methods, the hyperparameters were tuned from the following search space: 1) learning rate: {0.05, 0.01, 0.005}; 2) weight decay: {5e-4, 5e-5, 5e-6}; 3) dropout: {0.5, 0.6, 0.7, 0.8}. For APPNP, the propagation step size K was tuned from {5, 10}; where the parameters were tuned from {0, 0.1, 0.2, 0.3, 0.5, 0.8, 1.0}. For ElasticGNNs, the propagation step size K was tuned from {5, 10}, and the hyperparameters were tuned from {0, 3, 6, 9}. As for other details, the learning rate decayed every 40 epochs, the network parameters were initialized with a Gaussian distribution, the early stopping measure was 20 epochs, and the Adam optimizer was used in all experiments.

[0118] Table 2 shows the node classification accuracy results of this invention compared with the above 7 algorithms.

[0119]

[0120]

[0121] As can be seen from Table 2, the present invention has higher accuracy than existing graph network node classification methods, achieving accuracies of 83.3±0.8, 71.9±0.8 and 81.2±0.4 on the Cora, Citeseer and PubMed datasets, respectively.

[0122] Example 2

[0123] The advantages of this invention will be illustrated below by comparing its performance in visualizing classification results with that of the classic GAT and GCN methods:

[0124] Experimental conditions

[0125] The equipment used in the experiment was a machine with a 64-core E5-2682 v4@2.50GHz processor, 128GB of memory and a TiTAN Xp GPU. The programming platform was PyCharm. All experiments in this paper were run on this machine.

[0126] The experiments used three citation datasets: Cora, Citeseer, and PubMed. Only 20 nodes were used for training in each category, and the remaining 1000 nodes were used for testing. Specific dataset parameters are shown in Table 1.

[0127] Experimental content

[0128] To more clearly display the learning results, this embodiment will perform two-dimensional visualization of the hidden layer on t-SNE. The t-distributed random neighbor embedding (t-SNE) method is used to map the output representation in Z to a two-dimensional space. Each dot in the graph represents a node in the graph, and the color of the dot represents the node's label. Graphing all methods would be redundant; therefore, this section selects some typical methods for comparison, including the t-SNE visualization performance of GCN, GAT, and the EL-GFAT of this invention on the Cora and Citeseer datasets, respectively. Figure 2 (a) and Figure 2 (b) shows the t-SNE visualization results of GAT on the Cora and Citeseer datasets, respectively. Figure 2 (c) and Figure 2 (d) shows the t-SNE visualization results of GCN on the Cora and Citeseer datasets, respectively. Figure 2 (e) and Figure 2(f) shows the t-SNE visualization results of the EL-GFAT of this invention on the Cora and Citeseer datasets, respectively.

[0129] from Figure 2 (a) and Figure 2 As shown in (b), GAT performs well in overall segmentation on the Cora dataset, but there is some overlap with the purple and green categories in the yellow-labeled categories. On the Citeseer dataset, the overall category segmentation is less clear compared to Cora, especially in the yellow and purple-labeled categories, which have significant overlap with other categories, consistent with the accuracy ranking of the datasets.

[0130] from Figure 2 (c) and Figure 2 As shown in (d), on the Cora dataset, the categories marked in purple are mixed to some extent with those in light green and yellow, while other categories are clearly separated. On the Citeseer dataset, only the categories marked in yellow are scattered throughout the entire image, resulting in poor classification performance, but the other categories perform well.

[0131] from Figure 2 (e) and Figure 2 As shown in (f), the EL-GFAT method of this invention achieves good results on the Cora dataset where the categories marked in yellow partially overlap with those marked in purple, while the remaining categories are well-separated. On the Citeseer dataset, although the categories corresponding to yellow partially mix with those marked in purple, the other categories are well-separated, resulting in good performance.

[0132] Example 3

[0133] The advantages of this invention will be illustrated below by comparing its performance in iterative convergence with that of the classic GAT and GCN methods:

[0134] Experimental conditions

[0135] The equipment used in the experiment was a machine with a 64-core E5-2682 v4@2.50GHz processor, 128GB of memory and a TiTAN Xp GPU. The programming platform was PyCharm. All experiments in this paper were run on this machine.

[0136] The experiments used three citation datasets: Cora, Citeseer, and PubMed. Only 20 nodes were used for training in each category, and the remaining 1000 nodes were used for testing. Specific dataset parameters are shown in Table 1.

[0137] Experimental content

[0138] This embodiment includes iterative graphs of three methods—GAT, GCN, and the EL-GFAT method of this invention—on three datasets, showing the changes in loss and accuracy on the training and validation sets as a function of the number of iterations. Here, the epoch is set to 200.

[0139] Figure 3 (a), (b), and (c) represent the accuracy convergence and loss convergence graphs of GCN and the EL-GFAT of this invention on the training and validation sets of the Cora, Citeseer, and PubMed datasets, respectively. Figure 3 (d), (e), and (f) represent the accuracy convergence and loss convergence graphs of GCN and the EL-GFAT of this invention on the training and validation sets of the Cora, Citeseer, and PubMed datasets, respectively.

[0140] pass Figure 3 (a) As can be seen in the accuracy graph on the left side of the Cora dataset, the training accuracy is clearly higher than the validation accuracy. Secondly, regardless of whether it's validation or training, the EL-GFAT method outperforms the GAT method. It's worth noting that the two curves are very similar, which is because EL-GFAT is an improvement on GAT, and the similarity in the curves has some interpretive value. In the loss convergence on the right side, the two models generally show the same trend, but this cannot be used as a standard to judge the performance.

[0141] pass Figure 3 (b) As can be seen on the Citeseer dataset, in the accuracy iteration curves (left), except for the first 25 epochs on the training set where GAT outperforms EL-GFAT, EL-GFAT consistently leads GAT. Validation accuracy plateaus after 50 epochs, with EL-GFAT maintaining its lead. In the loss iteration curves (right), EL-GFAT initially decreases faster in validation loss, while GAT shows better iteration performance after 150 epochs. However, in the final result, EL-GFAT achieves the best training set fit.

[0142] pass Figure 3 (c) It can be seen that on the PubMed dataset, in the accuracy iteration curves (left), except for the first 80 epochs on the validation set where GAT outperforms EL-GFAT, the accuracy on both the training and validation sets is higher for EL-GFAT, and remains stable and leading after 150 epochs. In the loss iteration curves (right), the training and validation losses of EL-GFAT decrease more slowly than those of GAT before 150 epochs, and then decrease slowly thereafter.

[0143] pass Figure 3(d) As can be seen on the Cora dataset, in the accuracy iteration curves (left), GCN's accuracy is higher than EL-GFAT's only in the first 20 epochs on the training set. EL-GFAT's accuracy remains higher and stabilizes after 100 epochs. In the loss iteration curves (right), the two methods are largely similar, with GCN showing slightly better convergence on the training set and EL-GFAT showing slightly worse convergence on the validation set.

[0144] pass Figure 3 (e) It can be seen that on the Citeseer dataset, in the accuracy iteration curves in the left figure, the EL-GFAT method outperforms the GCN method throughout both the training and validation sets, and remains stable after 50 epochs. In the loss iteration curves in the right figure, the loss of the EL-GFAT method converges and remains stable at 120 epochs, while the loss of GCN remains unstable within 200 epochs.

[0145] pass Figure 3 (f) As can be seen on the PubMed dataset, in the accuracy iteration curves (left), the GCN method outperforms the EL-GFAT method on both the training and validation sets before 20 epochs. However, after that, the EL-GFAT method maintains its lead and stabilizes after 100 epochs. In the loss iteration curves (right), EL-GFAT converges faster than GCN on the validation set and remains stable after 100 epochs. In contrast, GCN only gradually stabilizes around 200 epochs.

[0146] Although embodiments and drawings of the present invention have been disclosed for illustrative purposes, those skilled in the art will understand that various substitutions, variations and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the scope of the present invention is not limited to the contents disclosed in the embodiments and drawings.

Claims

1. A graph node classification method based on graph feature self-attention mechanism of ensemble learning, characterized in that: The method includes the following steps: S1. Obtain raw data from publicly available graph network datasets, including the feature matrices of nodes. And graph adjacency matrix ; S2. The classic LightGBM algorithm model is used to analyze the feature matrix of the nodes. And graph adjacency matrix Perform predictive processing; S3. Use the LightGBM algorithm from step S2 to integrate the learning model and enhance the node features; S4. Merge the tree model parameters summarized in step S3 into the node matrix features to obtain a new node feature matrix. That is, merging the feature vectors of all nodes into ,in n The number of samples; S5. Construct a graph feature self-attention network; S6. The node feature matrix obtained in step S4 And graph adjacency matrix The input is fed into the graph feature self-attention network model in step S5 for training; S7. Merge the two models from steps S2 and S5; S8. Apply the two-stage model of steps S2 and S5 above to the graph node data to calculate the accuracy of node classification. Step S5 specifically involves: S501. When aggregating information at the central node, a first-level attention mechanism is first applied to neighboring nodes. ； in: This represents the central node. i For neighboring nodes j Attention coefficient; LeakReLU is an activation function; a:R F ×R F →R is a single-layer negative feedback neural network that is used to calculate the attention coefficient; W∈R F×F It is the parameter matrix used for feature transformation. It is the feature dimension of each node; h i ∈R F , h j ∈R F These are the central nodes. i and neighboring nodes j eigenvectors, , Indicates the number of nodes; S502, Replace the existing features with the node features generated in step S3. ,in As the original input for node features, define ; S503. First, initialize a hidden vector for each feature dimension. vi ∈R F Secondly, a negative feedback neural network with scalar output is used to calculate the attention score of each node for each feature dimension. ,in: att Network intrinsic parameter specifications are The output is a scalar; S504. To make coefficients easier to compare across different features, the softmax function is used across all features. j Normalize them in the selection: ； in: The normalized attention coefficient; S505. Obtain the normalized attention coefficients, which will be used to calculate a linear combination of the features corresponding to them, as the output feature of each node. The feature update for a single node is as follows: ； in: βi ∈R F This represents the attention coefficient vector of node i for all features, and it shows how the node's feature information is updated. S506. After passing through the graph feature attention network, multi-classification is performed using the softmax function: ； in: y∈ R class×1 Based on the classification results, the loss function is: ； Where: Y ij ∈ R n×class For the predicted results; n The number of samples; class The number of categories; Z ij ∈ R n×class This is a real label.

2. The graph node classification method based on ensemble learning and graph feature self-attention mechanism according to claim 1, characterized in that: Step S2 specifically involves: S201. Initialize the decision tree model, assuming a total of... Tr Trees, in each iteration the first (1< t ≤ Tr When there are 10 trees, the model Update in an additive manner: ； in: It is a model built in the previous iteration; This is the model built in this iteration; From a certain family of functions The weak learner selected in the selection; It is the learning rate; These are the input features of the sample; Select weak learners To approximate the loss function The negative gradient; S202, The predictions of the decision tree model are as follows: ； in: Indicates the first i Feature vectors of each sample; S203. For one of the decision trees, the final result depends on which leaf node the input sample is classified into. Let... For the first The set of samples in the leaf nodes where the output of the tree is located, therefore the first tree... The output of the tree can be represented as ,in: express The values ​​of the leaf nodes in the set; therefore, the final output of this ensemble learning method is ,in: Indicates the first k The weights of the weak learner.

3. The graph node classification method based on ensemble learning and graph feature self-attention mechanism according to claim 1, characterized in that: Step S3 specifically involves: for each input sample, the weighted sum of the results of each weak learner in the model is obtained, totaling [number of samples]. Tr A weak learner will eventually yield... Tr + n dimensional feature extension, where n The number of categories for the classification task, including Tr The results of the weak learners and the final output of the model, i.e., the new features, will become: ； Wherein: the middle part represents the weighted sum of the leaf nodes of each weak learner in the model, forming a feature vector, and the right part represents the classification result represented by one-hot encoding.

4. The graph node classification method based on ensemble learning and graph feature self-attention mechanism according to claim 1, characterized in that: Step S8 specifically involves: calculating and evaluating the accuracy rate. ,in Indicates a positive sample. N This represents the total number of samples.

Images