A graph representation learning method based on sparse and distant neighbor node enhancement

Abstract

The present invention provides a graph representation learning method based on sparse and distant neighbor node enhancement, which relates to the field of data enhancement technology. According to the acquired graph structure data set to be enhanced, the present invention obtains the initial node feature matrix, adjacency matrix and original classification labels, and divides the training set and the test set; the node features of the training set are enhanced by feature enhancement technology to obtain a sparse enhanced graph structure data set; then, in combination with the node mixing technology, the features and classification labels of intra-class nodes and inter-class nodes are mixed to increase the reachability of distant neighbor nodes, and a distant neighbor enhanced graph structure data set is obtained; the distant neighbor enhanced graph structure data set is input into the graph neural network model for graph representation learning and training, so as to obtain the optimal node representation and the trained graph representation learning model; finally, the model is tested and evaluated to obtain an evaluated model. The present invention solves the shortcomings of traditional graph neural networks in processing sparse nodes and distant neighbor node information aggregation.

Description

Technical Field

[0001] The present invention relates to the technical field of graph neural network data enhancement, and in particular to a graph representation learning method based on sparse and distant neighbor node enhancement. Background Art

[0002] With the rapid development of data science and artificial intelligence, Graph Neural Network (GNN) has become one of the core technologies for processing graph structured data. Graph Neural Networks have been widely used in social networks, recommendation systems, chemical molecular analysis and other fields. However, in practical applications, sparse nodes (i.e. nodes with fewer connections) and distant neighbor nodes (nodes that are far away and difficult to transmit information) are often encountered. These nodes are often difficult to effectively classify or identify through traditional graph neural network models. Due to the limited information propagation of sparse nodes and distant neighbor nodes, the model performs poorly when processing graph data, affecting the overall classification accuracy and the generalization ability of the model.

[0003] In recent years, researchers have proposed a variety of data enhancement methods and tried to improve the performance of graph neural networks through various data enhancement techniques. However, these methods still face challenges in practical applications, especially when the graph structure is complex and the node features vary greatly. Traditional enhancement methods are difficult to effectively solve the problem of feature propagation and aggregation of sparse and distant nodes.

[0004] Therefore, a graph representation learning method that can enhance sparse and distant neighbor nodes is proposed, which has important application value and helps to improve the performance of graph neural networks in various complex graph data. Summary of the invention

[0005] The present invention aims to provide a graph representation learning method based on sparse and distant neighbor node enhancement, so as to perform data node enhancement on complex graph structure data with sparse and distant neighbor node distribution, obtain the optimal node representation, and lay the foundation for the subsequent fast and accurate node classification.

[0006] In order to solve the above technical problems, the present invention is implemented through the following technical solutions:

[0007] A graph representation learning method based on sparse and distant neighbor node enhancement, comprising:

[0008] S1, according to the acquired graph structure data set to be enhanced, obtain the initial node feature matrix, adjacency matrix and original classification label, and divide them into a training set and a test set; wherein the initial node feature matrix is ​​used to describe the feature information of the node, and the adjacency matrix is ​​used to represent the connection relationship between nodes;

[0009] S2, according to the training set, using feature enhancement technology to enhance the input node features, and combining the initial node feature matrix of the training set to obtain a graph structure feature matrix after enhancing the sparse nodes, thereby obtaining a sparse enhanced graph structure data set after enhancing the sparse nodes;

[0010] S3, according to the sparse enhanced graph structure dataset, combined with the node mixing technology, by mixing the features and classification labels of the intra-class nodes and the inter-class nodes to increase the reachability of the distant neighbor nodes, a graph structure dataset after enhancing the distant neighbor nodes is obtained, that is, the distant neighbor enhanced graph structure dataset;

[0011] S4, inputting the distant neighbor enhanced graph structure data set into the graph neural network model for graph representation learning and training to optimize the graph representation of the data set, thereby obtaining the optimal node representation and the trained graph representation learning model based on sparse and distant neighbor node enhancement;

[0012] S5, input the test set into the trained graph representation learning model based on sparse and distant neighbor node enhancement for test evaluation, and obtain an evaluated graph representation learning model; input the newly acquired sparse graph structure data set into the evaluated graph representation learning model for node enhancement, and obtain the graph structure data set with optimal node representation for subsequent classification tasks.

[0013] Preferably, the feature enhancement technology generates and enhances the initial node feature space of the input training set through the CVAE model, specifically:

[0014] The initial node features of the training set are input into the encoder of the CVAE model for processing. The input initial node features are subjected to multi-layer nonlinear transformation of the multi-layer perceptron MLP to extract the features, and pass through two fully connected layers to obtain the mean vector and variance vector of the features, i.e., latent variables;

[0015] The latent variables are decoded and mapped back to the original feature space through the decoder of the CVAE model, and the initial node features of the input training set are combined with the latent variables to reconstruct node features with richer feature expressions, thereby obtaining the sparse enhanced graph structure dataset.

[0016] Preferably, it also includes: using a consistent regularization loss function on the obtained sparse enhanced graph structure data set Perform optimization training to ensure that the reconstructed node features remain consistent in the feature space by calculating the differences between multiple enhanced feature distributions; wherein the consistency regularization loss function The formula is:

[0017] ；

[0018] in, is the mixing ratio generated based on Beta distribution; N is the number of samples; is the reconstruction node feature; is the average feature after temperature scaling, obtained by scaling the standard deviation of the latent variable.

[0019] Preferably, the node mixing technology is:

[0020] According to the sparse enhanced graph structure data set, in the forward propagation process, the initial node feature matrix and the adjacency matrix of the sparse enhanced graph structure data set are inferred and predicted to obtain a pseudo-label classification result;

[0021] The feature interpolation method is used to generate enhanced node features for the sparse enhanced graph structure dataset, and the mixed ratio generated by the Beta distribution is applied to combine the indexes of the original labeled nodes and unlabeled nodes in the training set and the pseudo-label classification results. , select intra-class nodes and inter-class nodes for feature mixing. The formula for feature mixing is as follows:

[0022] ;

[0023] in, Represents the node features after mixing; and Respectively represent the selected intra-class or inter-class node features; Represents the mixture ratio generated based on Beta distribution;

[0024] According to the mixed node features, high-similarity mixed nodes are selected based on the sampled neighbor distribution and the similarity matrix to generate an intra-class mixed feature matrix , inter-class mixed features And the mixed adjacency matrix , thereby obtaining a data set with enhanced feature aggregation capability of distant neighbor nodes, namely the distant neighbor enhanced graph structure data set.

[0025] Preferably, it also includes: in the node mixing process, using the KL divergence mixing loss function for the distant neighbor enhanced graph structure data set to adjust and optimize the mixing result, specifically:

[0026] According to the intra-class mixed features , inter-class mixed features And the mixed adjacency matrix Perform label classification prediction to obtain mixed node labels;

[0027] The original classification label and the mixed node label are input into the KL divergence mixed loss function; wherein the KL divergence mixed loss function The formula is as follows:

[0028] ;

[0029] in, represents the intra-class mixed loss The weight hyperparameters of represents the inter-class mixing loss The weight hyperparameters of represents the intra-class mixed loss, that is, the KL divergence mixed loss of the original classification label corresponding to the intra-class mixed feature and the mixed node label; It represents the inter-class mixing loss, that is, the KL divergence mixing loss of the original classification label corresponding to the inter-class mixing feature and the mixed node label.

[0030] Preferably, the process of using the graph neural network model for graph representation learning and training is specifically as follows:

[0031] Step 1: After the initial node feature matrix of the input distant neighbor enhanced graph structure data set is extracted through the first graph convolution layer of the graph neural network model, it is normalized through the batch normalization layer; then a nonlinear change is performed through the ReLU activation function to help the model better learn complex features; the data output by the ReLU activation function is input into the second graph convolution layer for feature aggregation, and after downsampling, it is passed through the batch normalization layer again to stabilize the model training;

[0032] Step 2: After the output features of the first graph convolution layer and the output features of the second graph convolution layer are concatenated in the channel dimension, the features are further processed through a multi-layer neural network. The features after each layer of convolution are further extracted through residual connection or aggregation, and the feature representation ability of the node is progressively enhanced layer by layer. The features of distant neighbor nodes are propagated and strengthened through aggregation and random walk of neighbor node features.

[0033] Step 3: The features output from each layer are spliced ​​again and processed by multiple bridge modules of the graph neural network model to gradually reduce the spatial dimension of the features and perform feature compression and integration; the bridge module includes a convolutional layer, a batch normalization layer, and an activation function;

[0034] Step 4: Input the node features obtained in step 3 into the fully connected layer to further compress the channel dimension to reduce the number of parameters and integrate the features; process the compressed features through the batch normalization layer and the nonlinear change of the second activation function to obtain the final features.

[0035] Preferably, it also includes: performing node classification prediction according to the last feature and updating the classification result; and calculating the cross entropy loss of feature enhancement and node mixing respectively through back propagation during the training process. and , the final cross entropy loss function The expression is as follows:

[0036]

[0037] in, is the weight hyperparameter of the feature enhancement method, is the weight hyperparameter of the node mixing method; is the consistency regularization loss; is the cross entropy loss of feature enhancement, that is, the cross entropy loss between the classification prediction result and the original classification label after the feature enhancement operation; is the KL divergence mixed loss function; It is the cross entropy loss of node mixing, that is, the cross entropy loss between the classification prediction result and the original classification label after the node mixing operation.

[0038] The present invention also provides a graph representation learning device based on sparse and distant neighbor node enhancement, comprising:

[0039] An acquisition unit is used to obtain an initial node feature matrix, an adjacency matrix and original classification labels according to the acquired graph structure data set to be enhanced, and divide them into a training set and a test set; wherein the initial node feature matrix is ​​used to describe the feature information of the node, and the adjacency matrix is ​​used to represent the connection relationship between the nodes;

[0040] A feature enhancement unit, used to enhance the input node features by using feature enhancement technology according to the training set, and obtain a graph structure feature matrix after enhancing the sparse nodes in combination with the initial node feature matrix of the training set, thereby obtaining a sparse enhanced graph structure data set after enhancing the sparse nodes;

[0041] A node mixing unit is used to obtain a graph structure dataset after enhancing distant neighbor nodes, i.e., a distant neighbor enhanced graph structure dataset, by mixing features and classification labels of intra-class nodes and inter-class nodes according to the sparse enhanced graph structure dataset and combining the node mixing technology to increase the reachability of distant neighbor nodes;

[0042] A learning and training unit, used for inputting the distant neighbor enhanced graph structure data set into a graph neural network model for graph representation learning and training, so as to optimize the graph representation of the data set, thereby obtaining an optimal node representation and a trained graph representation learning model based on sparse and distant neighbor node enhancement;

[0043] The model evaluation unit is used to input the test set into the trained graph representation learning model based on sparse and distant neighbor node enhancement for test evaluation, so as to obtain an evaluated graph representation learning model; input the newly acquired sparse graph structure data set into the evaluated graph representation learning model for node enhancement, so as to obtain a graph structure data set with optimal node representation for subsequent classification tasks.

[0044] The present invention also provides a graph representation learning device based on sparse and distant neighbor node enhancement, including a processor and a memory, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement a graph representation learning method based on sparse and distant neighbor node enhancement as described above.

[0045] The present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the graph representation learning method based on sparse and distant neighbor node enhancement is implemented as described above.

[0046] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0047] The method of the present invention uses feature enhancement technology to enhance the features of the original sparse nodes; adopts node mixing technology to increase the accessibility of distant neighbor nodes in graph structure data, and combines with consistency regularization loss function to optimize the feature expression of sparse nodes and distant neighbor nodes in graph structure data, effectively improving the performance of graph representation learning models when processing sparse graph structures and data sets with large differences in node features. By enhancing the feature propagation capabilities of sparse nodes and distant neighbor nodes, the present invention solves the shortcomings of traditional graph neural network models in processing sparse nodes and distant neighbor node information aggregation, thereby improving the overall classification prediction performance of subsequent classification models for multiple complex graph structure data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 A schematic diagram of a graph representation learning method based on sparse and distant neighbor node enhancement provided in Example 1 of the present invention.

[0050] Figure 2 A flowchart for implementing the training of a graph representation learning model with enhanced sparse and distant neighbor nodes provided in Example 1 of the present invention.

[0051] Figure 3 This is a structural block diagram of the GNN graph neural network module provided in Example 1 of the present invention.

[0052] Figure 4 This is a comparison chart of the classification results of the test data set provided in Example 1 of the present invention after different enhancement strategies are used.

[0053] Figure 5 A schematic diagram of a graph representation learning device based on sparse and distant neighbor node enhancement provided in Embodiment 2 of the present invention.

[0054] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention claimed for protection, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] Embodiment 1

[0057] Embodiment 1 of the present invention provides a graph representation learning method based on sparse and distant neighbor node enhancement, which can be implemented by a graph representation learning device based on sparse and distant neighbor node enhancement (hereinafter referred to as graph representation learning device), and in particular, executed by one or more processors in the graph representation learning device.

[0058] In this embodiment, the graph representation learning device may be an electronic device equipped with a processor, the processor having a computer program of the graph representation learning method based on sparse and distant neighbor node enhancement and the computer program can be executed, such as a computer, a smart phone, a smart tablet, a workstation, etc., which is not limited here.

[0059] In this embodiment, graph structure data is a data structure composed of nodes (or vertices) and edges, where nodes represent entities or objects, such as entities, people, places, etc., and edges represent the relationship or connection between these entities or objects. Graph structure data can be used for social network analysis to reveal the community structure, influence propagation and key nodes in social networks. For example, social media platforms such as Facebook and Twitter use graph algorithms to recommend content, advertisements and friends; it can be used to build recommendation systems to provide users with personalized recommendations by analyzing the relationship between users and the relationship between items, such as Amazon, Taobao and other companies use graph structure data and algorithms in shopping recommendation systems; it can be used to analyze bioinformatics data such as genomes, protein interaction networks and metabolic networks; it can be used to optimize traffic networks to provide more efficient traffic planning and route recommendations by analyzing the connection between roads and traffic flow; it can be used to analyze risk propagation and association relationships in banking systems; it can be used to build knowledge graphs to organize knowledge into graph structures to better represent the relationship between entities and semantic information; and so on.

[0060] The embodiment of the present invention aims to provide a graph representation learning method based on sparse and distant neighbor node enhancement to enhance the node representation of complex graph structure data with sparse and distant neighbor node distribution, thereby achieving further rapid and accurate node classification of the data and improving the application effect of the model in the fields of social network analysis, recommendation system, etc.

[0061] like Figure 1 As shown, a graph representation learning method based on sparse and distant neighbor node enhancement includes steps S1 to S5.

[0062] S1, according to the acquired graph structure data set to be enhanced, obtain the initial node feature matrix, adjacency matrix and original classification labels, and divide them into training set and test set; wherein, the initial node feature matrix is ​​used to describe the feature information of the node, and the adjacency matrix is ​​used to represent the connection relationship between nodes.

[0063] In this embodiment, according to the acquired graph structure data to be enhanced, an initial node feature matrix and an adjacency matrix are constructed, and a training set, a test set, and a validation set are divided. The specific steps include A1 to A2.

[0064] Step A1: construct a graph structure dataset, including an initial node feature matrix to describe the feature information of the node, and an adjacency matrix to represent the connection relationship between nodes;

[0065] The initial node feature matrix is ​​obtained by performing feature extraction on the multi-view dataset;

[0066] Based on the Euclidean distance, generating node edges for the nearest neighbors of each node of the initial feature matrix to obtain an initial adjacency graph;

[0067] Using a pruning strategy on the initial adjacency graph to obtain a key adjacency graph to improve the quality of the adjacency graph;

[0068] The key adjacency graph is normalized to obtain the adjacency matrix.

[0069] Step A2: According to the original classification labels, a fixed number of samples of each category are randomly selected as the training set, and then the validation set and test set are extracted from the remaining data;

[0070] S2, according to the training set, the input node features are enhanced using feature enhancement technology, and combined with the initial node feature matrix of the training set to obtain the graph structure feature matrix after enhancing the sparse nodes, thereby obtaining a sparse enhanced graph structure data set after enhancing the sparse nodes.

[0071] like Figure 2 As shown in the figure, the complete flow chart of the graph neural network (GNN) model in this embodiment. The figure clearly shows the entire processing process from input node features and edges to the final classification result. As can be seen in the figure, the input node features are first extracted through multiple layers of graph convolution layers, and then nonlinearly transformed through activation functions such as ReLU to capture the complex patterns of the data. The output of each layer will aggregate and process the features until the classification result is finally generated through the fully connected layer.

[0072] In this embodiment, the training set data is first loaded, and the features of the input sparse nodes are enhanced using feature enhancement technology; wherein the CVAE model is used to generate and enhance the feature space of the sparse nodes, and the specific implementation steps are as follows:

[0073] CVAE, the full name of which is Conditional Variational Autoencoder, combines the advantages of convolutional neural network (CNN) and variational autoencoder (VAE) and is a directional graph generation model. CVAE includes encoder, latent space sampler and decoder. The encoder part usually adopts a convolutional neural network structure to map the input sparse node features (i.e. conditional features) to distribution parameters (mean and standard deviation) in the latent space. The latent space sampler samples latent variables from these parameters. The decoder uses these latent variables and conditional features to reconstruct node features.

[0074] Specifically, in this embodiment, the encoder in the CVAE model is used to process the input sparse node features (i.e., the initial node features of the training set), and the input sparse node features are subjected to multi-layer nonlinear transformation through a multi-layer perceptron (MLP) to extract the features, and the mean vector and variance vector of the features are generated through two fully connected layers to obtain the latent variables;

[0075] The sampled latent variables are decoded by the decoder, and the conditional features are combined with the latent variables to generate the reconstructed features of the sparse nodes, that is, the graph structure features after enhancing the sparse nodes, and then the sparse enhanced graph structure dataset after enhancing the sparse nodes is obtained.

[0076] During the training process of the feature enhancement step, a consistency regularization loss is added , by calculating the differences between multiple feature distributions generated by feature enhancement operations, we ensure that the generated reconstructed features remain consistent.

[0077] The consistency regularization loss function The formula is:

[0078] ；

[0079] in, is the mixing ratio generated based on Beta distribution; N is the number of samples; is the reconstruction node feature; is the average feature after temperature scaling, which is obtained by scaling the standard deviation of the latent variable. The temperature scaling technique is used to adjust the smoothness of the feature distribution.

[0080] The consistency regularization loss can be calculated by various methods, such as maximum mean difference (MMD), KL divergence, JS divergence, Wasserstein distance, etc. Choose an appropriate distance metric (such as Euclidean distance, cosine similarity, etc.) to calculate the difference between the average feature after temperature scaling and each reconstructed feature.

[0081] This difference measures the deviation of the reconstructed feature from the average feature and reflects the degree of feature consistency.

[0082] S3, based on the sparse enhanced graph structure dataset, combined with the node mixing technology, by mixing the features and classification labels of intra-class nodes and inter-class nodes to increase the reachability of distant neighbor nodes, a graph structure dataset after enhancing distant neighbor nodes is obtained, namely, a distant neighbor enhanced graph structure dataset.

[0083] In this embodiment, the node mixing technology is combined to mix the features and labels of the intra-class and inter-class nodes to increase the reachability of distant neighbor nodes, and finally obtain a graph structure dataset with distant neighbor enhancement. The specific implementation is as follows:

[0084] First, a feature-enhanced dataset (i.e., the sparse enhanced graph structure dataset) is loaded and a node hybrid model is created. In the forward propagation process, the node features and adjacency matrix in the sparse enhanced graph structure dataset are first inferred and predicted to obtain the node classification result, i.e., the pseudo-label classification result.

[0085] Then, intra-class mixing and inter-class mixing are performed. Intra-class mixing refers to mixing the node features of the same category (including nodes marked with original classification labels, unlabeled nodes and pseudo-labeled nodes), classification labels and adjacency matrices in the sparse enhanced graph structure dataset. Inter-class mixing refers to mixing the node features of different categories (including nodes marked with original classification labels, unlabeled nodes and pseudo-labeled nodes) and classification labels in the sparse enhanced graph structure dataset.

[0086] Both hybrids use feature interpolation to generate enhanced node features. Beta distribution is applied to generate the hybrid ratio based on the indexes of the original labeled nodes and unlabeled nodes in the training set and the pseudo-label classification results. , which is used to mix features within and between classes. The specific formula for feature mixing is as follows:

[0087] ;

[0088] in, Represents the node features after mixing; and Respectively represent the selected intra-class or inter-class node features; Represents the mixture ratio generated based on the Beta distribution.

[0089] Then, appropriate (e.g., high-similarity) mixed nodes are selected by sampling neighbor distribution and similarity matrix to generate intra-class mixed features. , inter-class mixed features , and the mixed adjacency matrix , the original classification labels and the mixed node classification results are input into the KL divergence loss function to adjust and optimize the mixed results to obtain the KL divergence mixed loss function , the specific formula is as follows:

[0090] ;

[0091] in, represents the intra-class mixed loss The weight hyperparameters of represents the inter-class mixing loss The weight hyperparameters of represents the intra-class mixed loss, that is, the KL divergence mixed loss of the original classification label corresponding to the intra-class mixed feature and the mixed node label; It represents the inter-class mixing loss, that is, the KL divergence mixing loss of the original classification label corresponding to the inter-class mixing feature and the mixed node label.

[0092] In this embodiment, KL divergence mixing is selected for optimization, and other existing metric loss functions or combined loss functions may also be selected for optimization training, such as mean square error loss, root mean square error, Huber loss, etc., which are not limited here.

[0093] In graph neural networks, neighbor sampling is an important technique used to reduce the amount of computation and maintain the relevance between nodes. First, determine the sampling strategy, which includes random sampling and importance-based sampling; then select the number of samplings to apply neighbor sampling. During the forward propagation of the graph neural network, the neighbors of the target node are sampled, and the sampled neighbor nodes are used for subsequent similarity calculations.

[0094] The similarity matrix is ​​used to represent the similarity relationship between nodes and is one of the important feature representations in graph neural networks. A variety of similarity metrics can be used, such as Euclidean distance, Manhattan distance, cosine similarity, etc., to calculate the similarity metric; then, a similarity matrix is ​​constructed based on the calculated similarity metric values. Each element in the matrix represents the similarity between two nodes. Analyze the similarity matrix to find node pairs with high similarity.

[0095] Finally, a data set with enhanced feature aggregation capability of distant neighbor nodes is obtained, namely, the distant neighbor enhanced graph structure data set.

[0096] S4, inputting the distant neighbor enhanced graph structure data set into the graph neural network model for graph representation learning and training to optimize the graph representation of the data set, thereby obtaining the optimal node representation and a trained graph representation learning model based on sparse and distant neighbor node enhancement.

[0097] Using graph structure data enhanced with sparse nodes and distant neighbor nodes, graph representation learning is performed through graph neural networks. During the training process, the graph representation is optimized by enhancing the features of the input sparse nodes and modeling the relationship between distant neighbor nodes; and the validation set is used to obtain the optimal node representation, which is used as the best model for downstream tasks (such as classification).

[0098] like Figure 3 As shown in the figure, the graph neural network includes a Dropout layer, a GNNConv layer, a ReLU layer and an Output layer.

[0099] The Dropout layer plays an important role in preventing overfitting in graph neural networks. It forces the network to learn more robust feature representations by randomly discarding the output of some neurons during the training process. This randomness helps the model generalize to new data and improves the stability and accuracy of the model. Specifically, the Dropout layer randomly ignores certain neurons in the network with a certain probability (such as 0.5), so that these neurons do not participate in forward propagation and backpropagation in the current training iteration. During the testing phase, the Dropout layer is turned off and the output of all neurons is retained, but in order to ensure that the input and output expectations are consistent, the output of the neurons is usually scaled.

[0100] GNNConv (graph convolution layer) is a core component in graph neural networks, used to process graph structured data. It updates the representation of the current node by aggregating the information of neighboring nodes. Graph convolution operations can extract feature representations in graph structured data, thereby achieving more efficient and accurate analysis and processing of graph structured data. In graph neural networks, GNNConv layers are often used for tasks such as node classification, graph classification, and link prediction. By stacking multiple GNNConv layers, the model can learn more complex graph structure features.

[0101] The ReLU (Rectified Linear Unit) layer is a commonly used activation function layer and also plays an important role in graph neural networks. It sets the part of the input value less than 0 to 0 and the part greater than 0 unchanged. This nonlinear transformation enables the model to learn more complex feature representations. The ReLU layer has the advantages of simple calculation and fast training speed, and alleviates the problem of gradient disappearance to a certain extent. In graph neural networks, the ReLU layer usually follows the GNNConv layer to introduce nonlinear characteristics.

[0102] Output layer is used to output the final result.

[0103] The specific steps include C1 to C5.

[0104] Step C1: The initial node feature X of the input distant neighbor enhanced graph structure dataset is first extracted through the first graph convolution layer. After the first convolution layer, the data is normalized through the batch normalization layer, and then nonlinear changes are introduced through the ReLU activation function to help the model better learn complex features. The data output by the first activation function is further aggregated through the second graph convolution layer and downsampled, and then passed through the batch normalization layer again to stabilize the model training.

[0105] Step C2: The output features of the first graph convolution layer are concatenated with the output features of the second graph convolution layer in the channel dimension, and the features are further processed through a multi-layer neural network. The features after each convolution layer are further extracted through residual connection or aggregation, and the feature representation ability of the node is progressively enhanced layer by layer. The model will propagate and strengthen the features of distant neighbor nodes through aggregation and random walk of neighbor node features.

[0106] Step C3: The features output from each layer are spliced ​​again and processed through multiple bridge modules, including convolutional layers, batch normalization layers, and activation functions. Through these modules, the model gradually reduces the spatial dimension of the features and performs feature compression and integration.

[0107] Step C4: Finally, the node features obtained in step C3 are input into the fully connected layer to further compress the channel dimension, reduce the number of parameters and integrate the features. The compressed features are processed again through the batch normalization layer, and finally the nonlinear effect is introduced through the activation function to obtain the final features, further improving the robustness and performance of the model.

[0108] Step C5: The final feature output by the model is used for node classification task and the predicted category is output. During the training process, the cross entropy loss (CrossEntropyLoss) of feature enhancement and node mixing is calculated through back propagation. and , the final loss function is as follows:

[0109]

[0110] in, is the weight hyperparameter of the feature enhancement method; is the weight hyperparameter of the node mixing method, is the cross entropy loss of feature enhancement, that is, the cross entropy loss between the classification prediction result and the original classification label after the feature enhancement operation; is the KL divergence mixed loss function; It is the cross entropy loss of node mixing, that is, the cross entropy loss between the classification prediction result and the original classification label after the node mixing operation.

[0111] In this embodiment, the cross entropy loss function is selected to optimize the training of the final node classification result. Other existing loss functions may also be selected for optimization training, such as mean square error loss, root mean square error, Huber loss, etc., which are not limited here.

[0112] Finally, the model is optimized to ensure good classification results on various node features, and the validation set is used to select and save the best performing model and parameters for subsequent classification test evaluation.

[0113] S5, input the test set into the trained graph representation learning model based on sparse and distant neighbor node enhancement for test evaluation, and obtain an evaluated graph representation learning model; input the newly acquired sparse graph structure data set into the evaluated graph representation learning model for node enhancement, and obtain the graph structure data set with optimal node representation for subsequent classification tasks.

[0114] In this embodiment, the model aggregates and propagates the node features in the test set through the graph convolution layer, outputs the predicted category of each node, and converts the predicted category output by the model into The true label of the test set Compare and evaluate the classification performance of the model on unknown data through various evaluation indicators (such as accuracy, MAR, MAP, MAF, etc.), ensuring that the model not only performs well on training data, but also achieves high accuracy on unseen test data. If the test evaluation indicators meet the predetermined standards, the model is used as the final data enhancement model in the actual classification task.

[0115] Furthermore, the test set is used to test the trained graph representation learning model based on sparse and distant neighbor nodes enhancement, evaluate the performance of the model on unknown data, verify its processing ability for sparse nodes and distant neighbor nodes, and obtain accurate classification results through the classification algorithm.

[0116] like Figure 4As shown, in this embodiment, a public graph neural network real data set is used for testing, and training and testing are performed under the same hardware environment. The training data sets used include five single-view data sets widely used for graph neural network node classification tasks, namely ACM, CiteSeer, Cora_ML, Twitch_DE and DBLP. Among them, the ACM data set contains 3025 nodes and 13843 edges, and the node feature dimension is 1870; the CiteSeer data set contains 3327 nodes and 4732 edges, and the node feature dimension is 3703; the Cora_ML data set contains 2995 nodes and 16516 edges, and the node feature dimension is 2879; the Twitch_DE data set contains 9498 nodes and 153138 edges, and the node feature dimension is 3177; the DBLP data set contains 17716 nodes and 105734 edges, and the node feature dimension is 334. During the training process, a standard graph construction method is used to represent the relationship between nodes through an adjacency matrix, and a graph neural network model is used for feature aggregation and classification. This example uses a common evaluation indicator: classification accuracy (ACC) to evaluate the node classification performance of the model.

[0117] Figure 4 This is a comparison of the accuracy of node classification using different graph neural networks under different graph structure dataset enhancement algorithm strategies. The relevant information of this experiment is as follows:

[0118] (1) The network datasets used in the experiment are:

[0119] ACM dataset: a network dataset of academics from the Association for Computing Machinery;

[0120] CiteSeer dataset: a network dataset cited by CiteSeer;

[0121] CoraML dataset: Cora network dataset used for machine learning (ML stands for Machine Learning).

[0122] TwitchDE dataset: a German social network dataset.

[0123] DBLP dataset: a network dataset of academic literature in the Digital Bibliography & Library Project.

[0124] (2) The types of graph neural network models used in the experiment are:

[0125] GCN model: Graph Convolutional Network, is a semi-supervised learning algorithm specifically for graph structured data. It combines the knowledge of convolutional neural network (CNN) and graph theory to perform effective feature extraction and node classification on graph data.

[0126] GAT model: Graph Attention Network is a graph neural network model based on the attention mechanism. It enhances the model's ability to capture important information in the graph structure by introducing attention coefficients between nodes.

[0127] APPNP model: Approximate Personalized Propagation of Neural Predictions, is an open source project based on PyTorch. It originates from an important research paper published at ICLR 2019 - "Prediction Post-Propagation: Graph Neural Networks Meet Personalized PageRank". Its core lies in its innovative node information propagation mechanism, which combines the graph convolutional network (GCN) with the personalized PageRank algorithm to solve the problem of neighborhood restrictions in the traditional GNN model.

[0128] Graphsage model: Graph Sample and Aggregated, is a graph neural network model for graph node embedding learning. The core idea is to generate node embedding by learning how to aggregate local domain information without relying on the structure of the entire graph.

[0129] (3) The data set processing strategies used in the experiment are:

[0130] Original: original dataset.

[0131] LAGNN: Local Augmentation Graph Neural Network, an efficient data augmentation framework that aims to solve the problem of limited expressiveness of graph neural networks (GNNs) when processing local neighborhood information.

[0132] DropMessage: It is a random deletion method for graph neural networks (GNNs) that operates directly on the message passing level, which can reduce sample variance in a more fine-grained manner and maintain information diversity.

[0133] NodeMixup: Node data enhancement technology, which fuses the information of labeled and unlabeled nodes through linear interpolation to generate new training samples.

[0134] MixAug: the method of the present invention.

[0135] It can be seen from the experimental results that based on different graph neural network models, the classification evaluation results of the graph structure data set enhanced by the method of the present invention (i.e., Mixaug in the figure) are overall better than other algorithms in terms of classification accuracy (ACC) indicators compared with other enhancement strategies.

[0136] In summary, compared with the prior art, the present invention has the following beneficial effects:

[0137] The method of the present invention uses feature enhancement technology to enhance the features of the original sparse nodes; adopts node mixing technology to increase the accessibility of distant neighbor nodes in graph structure data, and combines with the consistency regularization loss function to optimize the feature expression of sparse nodes and distant neighbor nodes in graph structure data, effectively improving the performance of the graph neural network model when processing sparse graph structures and data sets with large differences in node features. By enhancing the feature propagation capabilities of sparse nodes and distant neighbor nodes, the present invention solves the shortcomings of traditional graph neural networks in processing sparse nodes and distant neighbor node information aggregation, thereby improving the overall classification prediction performance of subsequent classification models for multiple complex graph structure data sets.

[0138] Embodiment 2

[0139] like Figure 5 As shown, the second embodiment of the present invention further provides a graph representation learning device based on sparse and distant neighbor node enhancement, comprising:

[0140] An acquisition unit is used to obtain an initial node feature matrix, an adjacency matrix and original classification labels according to the acquired graph structure data set to be enhanced, and divide them into a training set and a test set; wherein the initial node feature matrix is ​​used to describe the feature information of the node, and the adjacency matrix is ​​used to represent the connection relationship between the nodes;

[0141] A feature enhancement unit, used to enhance the input node features by using feature enhancement technology according to the training set, and obtain a graph structure feature matrix after enhancing the sparse nodes in combination with the initial node feature matrix of the training set, thereby obtaining a sparse enhanced graph structure data set after enhancing the sparse nodes;

[0142] A node mixing unit is used to obtain a graph structure dataset after enhancing distant neighbor nodes, i.e., a distant neighbor enhanced graph structure dataset, by mixing features and classification labels of intra-class nodes and inter-class nodes according to the sparse enhanced graph structure dataset and combining the node mixing technology to increase the reachability of distant neighbor nodes;

[0143] A learning and training unit, used for inputting the distant neighbor enhanced graph structure data set into a graph neural network model for graph representation learning and training, so as to optimize the graph representation of the data set, thereby obtaining an optimal node representation and a trained graph representation learning model based on sparse and distant neighbor node enhancement;

[0144] The model evaluation unit is used to input the test set into the trained graph representation learning model based on sparse and distant neighbor node enhancement for test evaluation, so as to obtain an evaluated graph representation learning model; input the newly acquired sparse graph structure data set into the evaluated graph representation learning model for node enhancement, so as to obtain a graph structure data set with optimal node representation for subsequent classification tasks.

[0145] Embodiment 3

[0146] The third embodiment of the present invention also provides a graph representation learning device based on sparsity and distant neighbor node enhancement, which includes a memory and a processor, wherein the memory stores a computer program, and the computer program can be executed by the processor to implement the graph representation learning method based on sparsity and distant neighbor node enhancement as described above.

[0147] Embodiment 4

[0148] The fourth embodiment of the present invention also provides a computer-readable storage medium, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a device where the computer-readable storage medium is located, the graph representation learning method based on sparse and distant neighbor node enhancement as described above is implemented.

[0149] In several embodiments provided in the embodiments of the present invention, it should be understood that the disclosed apparatus and method can also be implemented in other ways. The apparatus and method embodiments described above are merely schematic. For example, the flowcharts in the accompanying drawings show the possible architecture, functions and operations of the apparatus, method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0150] In addition, the functional modules in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0151] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code. It should be noted that in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0152] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0153] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0154] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0155] The "first\second" mentioned in the embodiments is only to distinguish similar objects, and does not represent a specific order for the objects. It is understandable that the "first\second" can be interchanged with the specific order or sequence where permitted. It should be understood that the objects distinguished by "first\second" can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0156] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A graph representation learning method based on sparse and distant neighbor node enhancement, characterized in that: include: S1, according to the acquired graph structure data set to be enhanced, obtain the initial node feature matrix, adjacency matrix and original classification labels, and divide them into a training set and a test set; wherein the initial node feature matrix is ​​used to describe the feature information of the node, and the adjacency matrix is ​​used to represent the connection relationship between nodes; S2, using feature enhancement technology to enhance the node features of the input training set, and combining the initial node feature matrix of the training set to obtain a graph structure feature matrix after enhancing the sparse nodes, thereby obtaining a sparse enhanced graph structure data set after enhancing the sparse nodes; S3, according to the sparse enhanced graph structure dataset, combined with the node mixing technology, by mixing the features and classification labels of the intra-class nodes and the inter-class nodes to increase the reachability of the distant neighbor nodes, a graph structure dataset after enhancing the distant neighbor nodes is obtained, that is, the distant neighbor enhanced graph structure dataset; wherein the node mixing technology is specifically: According to the sparse enhanced graph structure data set, in the forward propagation process, the initial node feature matrix and the adjacency matrix of the sparse enhanced graph structure data set are inferred and predicted to obtain a pseudo-label classification result; The feature interpolation method is used to generate enhanced node features for the sparse enhanced graph structure dataset, and the mixed ratio generated by the Beta distribution is applied to combine the indexes of the original labeled nodes and unlabeled nodes in the training set and the pseudo-label classification results. , select intra-class nodes and inter-class nodes for feature mixing. The formula for feature mixing is as follows: ； in, Represents the node features after mixing; and Respectively represent the selected intra-class node features and inter-class node features; Represents the mixture ratio generated based on Beta distribution; According to the mixed node features, high-similarity mixed nodes are selected based on the sampled neighbor distribution and the similarity matrix to generate an intra-class mixed feature matrix , inter-class mixed features And the mixed adjacency matrix , thereby obtaining a data set with enhanced feature aggregation capability of distant neighbor nodes, namely, the distant neighbor enhanced graph structure data set; S4, inputting the distant neighbor enhanced graph structure data set into the graph neural network model for graph representation learning and training to optimize the graph representation of the data set, thereby obtaining the optimal node representation and the trained graph representation learning model based on sparse and distant neighbor node enhancement; S5, input the test set into the trained graph representation learning model based on sparse and distant neighbor node enhancement for test evaluation, and obtain an evaluated graph representation learning model; input the newly acquired sparse graph structure data set into the evaluated graph representation learning model for node enhancement, and obtain the graph structure data set with optimal node representation for subsequent classification tasks.

2. According to claim 1, a graph representation learning method based on sparse and distant neighbor node enhancement is characterized in that The feature enhancement technology generates and enhances the initial node feature space of the input training set through the CVAE model, specifically: The initial node features of the training set are input into the encoder of the CVAE model for processing. The input initial node features are subjected to multi-layer nonlinear transformation of the multi-layer perceptron MLP to extract the features, and pass through two fully connected layers to obtain the mean vector and variance vector of the features, i.e., latent variables; The latent variables are decoded and mapped back to the original feature space through the decoder of the CVAE model, and the initial node features of the training set are combined with the latent variables to reconstruct node features with richer feature expressions, thereby obtaining the sparse enhanced graph structure dataset.

3. According to claim 2, a graph representation learning method based on sparse and distant neighbor node enhancement is characterized in that , further comprising: applying a consistent regularization loss function to the obtained sparse enhanced graph structure data set Perform optimization training to ensure that the reconstructed node features remain consistent in the feature space by calculating the differences between multiple enhanced feature distributions; wherein the consistency regularization loss function The formula is: ； in, is the mixing ratio generated based on Beta distribution; N is the number of samples; is the reconstruction node feature; is the average feature after temperature scaling, obtained by scaling the standard deviation of the latent variable.

4. According to claim 1, a graph representation learning method based on sparse and distant neighbor node enhancement is characterized in that , and further comprising: in the node mixing process, using the KL divergence mixing loss function for the distant neighbor enhanced graph structure data set to adjust and optimize the mixing result, specifically: According to the intra-class mixed features , inter-class mixed features And the mixed adjacency matrix Perform label classification prediction to obtain mixed node labels; The original classification label and the mixed node label are input into the KL divergence mixed loss function; wherein the KL divergence mixed loss function The formula is as follows: ； in, represents the intra-class mixed loss The weight hyperparameters of represents the inter-class mixing loss The weight hyperparameters of represents the intra-class mixed loss, that is, the KL divergence mixed loss of the original classification label corresponding to the intra-class mixed feature and the mixed node label; It represents the inter-class mixing loss, that is, the KL divergence mixing loss of the original classification label corresponding to the inter-class mixing feature and the mixed node label.

5. According to claim 4, a graph representation learning method based on sparse and distant neighbor node enhancement is characterized in that ,The process of using the graph neural network model for graph representation learning and training is as follows: Step 1: After extracting features from the initial node feature matrix of the input distant neighbor enhanced graph structure dataset through the first graph convolution layer of the graph neural network model, the matrix is ​​normalized through a batch normalization layer; then a nonlinear change is performed through a ReLU activation function to help the model better learn complex features; The data output by the ReLU activation function is input into the second graph convolution layer for feature aggregation, and after downsampling, it passes through the batch normalization layer again to stabilize the model training; Step 2: After the output features of the first graph convolution layer and the output features of the second graph convolution layer are concatenated in the channel dimension, the features are further processed through a multi-layer neural network. The features after each layer of convolution are further extracted through residual connection or aggregation, and the feature representation ability of the node is progressively enhanced layer by layer. The features of distant neighbor nodes are propagated and strengthened through aggregation and random walk of neighbor node features. Step 3: The features output from each layer are spliced ​​again and processed by multiple bridge modules of the graph neural network model to gradually reduce the spatial dimension of the features and perform feature compression and integration; the bridge module includes a convolutional layer, a batch normalization layer, and an activation function; Step 4: Input the node features obtained in step 3 into the fully connected layer to further compress the channel dimension to reduce the number of parameters and integrate the features; process the compressed features through the batch normalization layer and the nonlinear change of the second activation function to obtain the final features.

6. A graph representation learning method based on sparse and distant neighbor node enhancement according to claim 5, characterized in that , and also includes: performing node classification prediction based on the last feature and updating the classification result; and calculating the cross entropy loss of feature enhancement and node mixing respectively through back propagation during the training process and , the final cross entropy loss function The expression is as follows: ； in, is the weight hyperparameter of the feature enhancement method; is the weight hyperparameter of the node mixing method; is the consistency regularization loss; is the cross entropy loss of feature enhancement, that is, the cross entropy loss between the classification prediction result and the original classification label after the feature enhancement operation; is the KL divergence mixed loss function; It is the cross entropy loss of node mixing, that is, the cross entropy loss between the classification prediction result and the original classification label after the node mixing operation.

Images