Heterogeneous graph node classification method based on double-layer variational auto-encoder

Abstract

The invention provides a heterogeneous graph node classification method based on a double-layer variational auto-encoder, which effectively improves the node classification effect of a heterogeneous graph under the condition of feature deficiency and inaccuracy. The method comprises the following steps: firstly, determining the meaning of a heterogeneous graph and the definition of inaccurate and missing features, and initializing the missing features through one-hot coding; secondly, mapping the high-dimensional features into a low-dimensional space; secondly, representing the features in a node-level mode and an attribute-level mode; in the encoder, a double-layer variational auto-encoder is used for generating embedding for the features of the two modes; in the decoder, through an inner product form, the embedding of the two modes is restored respectively; then, aligning the restored attribute-level representation with the initial feature through a multi-layer perceptron; and finally, applying the generated features to a downstream classification task. According to the method, the cost of time, space and manual annotation is remarkably reduced, good generalization ability is achieved, and the classification effect under the condition of feature missing and inaccuracy in the heterogeneous graph is improved.

Description

Technical Field

[0001] The present invention relates to the field of graph data mining and self-supervised learning, and in particular to a heterogeneous graph node classification method based on a double-layer variational autoencoder in the field of heterogeneous graphs. Background Art

[0002] Heterogeneous Information Networks (HINs) are information networks that contain various types of nodes and edges. In real-world scenarios, HINs can effectively model data complexity, which provides rich semantics and a comprehensive view of the data. In recent years, heterogeneous graph neural networks (HGNNs) have received extensive attention and have been widely used in related fields such as social networks, recommendation systems, and knowledge graphs. In order to conduct in-depth analysis of HINs, many heterogeneous graph models have been proposed to learn the representation of nodes and successfully complete downstream tasks such as node classification and link prediction.

[0003] Due to the difficulty of collecting labels, it is crucial to learn from data in unlabeled scenarios. Self-supervised learning is a method of learning through the characteristics of the data itself. Self-supervised learning on graphs is generally divided into contrastive learning and generative methods. Contrastive learning uses pre-designed data augmentation to obtain relevant views of a given graph and adopts contrastive loss to learn representations that maximize the consistency between positive views while minimizing the disagreement between negative views. However, the success of contrastive learning relies heavily on the augmentation of informative data, which has been shown not to generalize across datasets.

[0004] The purpose of generative methods is to use input graphs for self-supervision and recovery of input data. Generative methods are highly flexible and can be applied to a variety of data types and tasks. By learning the latent distribution, the generative model generates new data samples with high generalization ability. This flexibility enables generative methods to play a role in different fields and scenarios, such as natural language processing, image generation, data augmentation, etc. In addition, generative methods are highly robust and have strong resistance to noise and outliers. This enables generative methods to have good performance in practical applications and maintain stable performance in complex environments with noise and outliers. Generative design can automatically adjust the design scheme according to different needs and conditions to meet the needs of different scenarios. This strong adaptability and flexibility enables generative methods to maintain effective performance in different environments and conditions. These advantages make generative methods have broad application prospects and important research value in various fields and scenarios.

[0005] Existing technology is insufficient

[0006] Although heterogeneous graphs have received extensive attention, there are two major challenges that are easily overlooked in most approaches:

[0007] ⅰ) The cost of obtaining dataset labels is high, and there is a problem of label scarcity.

[0008] ii) Node attributes in the original dataset are usually incomplete:

[0009] Due to high cost and privacy issues, it is difficult to collect the attributes of all nodes. Take the academic data ACM dataset as an example: the heterogeneous graph modeled by ACM consists of three types of nodes: papers, authors, and topics. The attributes of paper nodes are derived from the keywords in their titles, while the other two types of nodes lack attributes. Studies have shown that the characteristics of authors and research objects play a vital role in learning the embedding of heterogeneous graphs. Therefore, the completion of missing attributes is a problem worthy of attention.

[0010] ⅲ) The node attributes in the original dataset often contain noise:

[0011] Inaccurate node attributes can cause the spread of wrong information, which can adversely affect the performance of the model. For example, in datasets such as ACM, the attributes of paper nodes are often extracted from the bag-of-words representation of their keywords. However, this may be noisy. For example, words that do not help express the topic may be included, or some words may be mislabeled. According to the message passing mechanism of graph neural networks, the representation of a node is obtained by aggregating the information of its neighboring nodes. If the original node attributes are inaccurate, the noise will propagate to the node's neighboring nodes and degrade the performance of the model. Therefore, it is important to mitigate the impact of inaccurate attributes in the graph. Summary of the invention

[0012] The purpose of the present invention is to provide a heterogeneous graph node classification method based on a double-layer variational autoencoder, which can solve the problem of missing and inaccurate features in heterogeneous graphs. The present invention significantly reduces the cost of manual annotation, improves the speed and efficiency of model training, and has better generalization ability and adaptability. It can be applied to multi-classification tasks and obtain good classification results.

[0013] The specific technical solution for achieving the purpose of the present invention is:

[0014] A heterogeneous graph node classification method based on a double-layer variational autoencoder, the method comprising the following steps:

[0015] Step 1: Obtain a heterogeneous graph dataset, and construct a training set and a test set according to the ratio of 8:2. The heterogeneous graph consists of nodes with attributes and nodes without attributes. For nodes with attributes, manually collected attribute features are directly used, and for nodes without attributes, the data is preprocessed.

[0016] Step 2: Build a heterogeneous graph node classification model based on a two-layer variational autoencoder;

[0017] Step 3: Iteratively train the heterogeneous graph node classification model based on the double-layer variational autoencoder constructed in step 2 to obtain an embedded representation;

[0018] Step 4: Use the training set to iteratively train the Support Vector Machine (SVM) classifier to obtain the node classification results of the heterogeneous graph test set;

[0019] In step 1, the data preprocessing specifically includes:

[0020] 1.1: Filter all types of nodes without attributes and count the number of nodes under each type;

[0021] 1.2: Feature initialization: For each type of node without attributes, use one-hot encoding to generate the feature matrix of each type;

[0022] 1.3: For nodes with attributes, the manually collected attributes are directly used as their feature matrix, and the feature matrix of nodes with attributes and the feature matrix of nodes without attributes are used as initial features. At this time, the initial features exist in a high-dimensional space;

[0023] In step 2, a heterogeneous graph node classification model based on a two-layer variational autoencoder is constructed, which is divided into three parts: dimensionality reduction, inference model, and generation model. Specifically, it includes:

[0024] 2.1: Dimensionality reduction: Multilayer Perceptron (MLP) is used for dimensionality reduction. Given that the feature dimensions of each type of node are different, a layer of MLP network equal to the number of node types is used to reduce the dimensions of different types of nodes to the same dimension. Low-dimensional features reduce the time complexity of the model and reduce the model's memory requirements.

[0025] 2.2: Inference model: The inference model consists of a node-level variational autoencoder and an attribute-level variational autoencoder;

[0026] For node-level variational autoencoders:

[0027] 2.2.1: Concatenate the low-dimensional representations of all types of nodes obtained in 2.1 to obtain a feature matrix with the number of behavior nodes and columns as feature dimensions;

[0028] 2.2.2: Build a heterogeneous graph network: Based on the different relationships of the heterogeneous graph, build an attention network under each relationship, input the spliced ​​feature matrix into the graph attention network, and obtain the first layer of embedded representation;

[0029] 2.2.3: Build a double-layer Gaussian distribution latent space: Sample the Gaussian noise distribution matrix from the standard Gaussian distribution, and concatenate it with the embedded representation obtained in 2.2.2 to obtain the concatenated matrix of the embedding matrix and the noise matrix; input it into the attention network of the second layer of heterogeneous graph to obtain the node-level feature mean and variance matrices respectively;

[0030] For attribute-level variational autoencoders:

[0031] 2.2.4: Concatenate the low-dimensional representations of all types of nodes in 2.1 and transpose them to obtain a feature matrix with behavioral feature dimensions and the number of nodes as columns;

[0032] 2.2.5: Build a multi-layer perceptron: Use two layers of MLP; input the feature matrix of 2.2.4 into the first layer of MLP to obtain the embedding representation of the first layer;

[0033] 2.2.6: Build a double-layer Gaussian distribution latent space: Sample the Gaussian noise distribution matrix from the standard Gaussian distribution, and concatenate it with the embedded representation obtained in 2.2.5 to obtain the concatenated matrix of the embedding matrix and the noise matrix; input it into the second-layer MLP network to obtain the feature mean and variance matrices at the attribute level respectively;

[0034] 2.3: Generative model: The generative model consists of a structure decoder and a feature decoder;

[0035] For the structure decoder:

[0036] 2.3.1: The structural decoder is used to restore the structural information; after reparameterizing the node-level mean and variance obtained in 2.2.3, a continuous and smooth latent space representation is obtained, and the reparameterized mean matrix of all types of nodes is obtained;

[0037] 2.3.2: Using the reparameterized mean matrix of all types of nodes obtained in 2.3.1, perform inner product calculations on all different node types and control the calculated values ​​between 0 and 1 through the sigmoid activation function to obtain the node-node adjacency relationship matrix of two types of nodes, i.e., one relationship, and restore the structural information in the heterogeneous graph;

[0038] For the feature decoder:

[0039] 2.3.3: The feature decoder is used to restore the low-dimensional feature space in 2.1; after re-parameterizing the attribute-level mean and variance obtained in 2.2.6, a continuous and smooth latent space representation is obtained, and the re-parameterized mean matrix of the attribute is obtained;

[0040] 2.3.4: Use the attribute reparameterized mean matrix obtained in 2.3.3 to perform inner product calculation with the node-level feature mean matrix obtained in 2.2.3 and use the tanh activation function to control the calculated value between -1 and 1 to obtain the low-dimensional feature matrix of "node-attribute" and restore the low-dimensional feature space in the heterogeneous graph;

[0041] 2.3.5: Align initial features: Filter the low-dimensional representations of attribute node types in 2.3.4, map them to the high-dimensional feature space in the initial 1.3 through MLP, and align the initial features;

[0042] Step 3 iteratively trains the heterogeneous graph node classification model based on the double-layer variational autoencoder constructed in step 2, specifically including:

[0043] 3.1: Define the loss function of the heterogeneous graph node classification model;

[0044] 3.1.1: For the node-level variational autoencoder, the loss function is , which consists of cross entropy (reconstruction loss) and KL divergence, aims to constrain the distribution of the latent space and extract structural information;

[0045] 3.1.2: For attribute-level variational autoencoders, the loss function is , which consists of mean square loss (reconstruction loss) and KL, aims to constrain the distribution of the latent space and extract semantic information from the low-dimensional feature space;

[0046] 3.1.3: For 2.3.5 feature alignment, the loss function is ,Use mean square error to align high-dimensional original attributes, the purpose is to strengthen the feature information of the original attributed nodes;

[0047] 3.1.4: The total loss function of the heterogeneous graph node classification model is: ,in, and is an adjustable model loss balance parameter;

[0048] 3.2: Input the initialization data set in step 1 to the model, and iteratively train the heterogeneous graph node classification model using the loss function in 3.1;

[0049] The step 4 specifically includes:

[0050] 4.1: Use the representation in step 3 corresponding to the training set in step 1 to train the SVM classifier and obtain the classification results of the test set nodes in step 1.

[0051] Compared with the existing methods, the present invention has the following advantages:

[0052] 1) Significantly reduce the amount of annotation and improve efficiency: Through self-supervised learning and generative methods, the present invention does not require manually annotated labels, predefined meta-paths, and specific view enhancement methods, reducing dependence on manual operations and business knowledge experts, thereby reducing labor costs.

[0053] 2) Improve training efficiency: By mapping high-dimensional features to low dimensions and optimizing low-dimensional representation, the time and space complexity is reduced to linear, thereby improving training efficiency.

[0054] 3) Generalization and adaptability: The present invention fills in missing features and denoises inaccurate features. It is applicable to data sets in various fields, improves the generalization and adaptability of the model and the effect of heterogeneous graph classification models, and shows superior performance over traditional machine learning models and other graph models. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the present invention;

[0056] Figure 2 It is a structural diagram of the heterogeneous graph node classification model of the present invention;

[0057] Figure 3 Schematic diagram of the performance comparison between the present invention and existing methods on the DBLP dataset. DETAILED DESCRIPTION

[0058] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0059] See also Figure 1 and Figure 2 , the present invention specifically includes:

[0060] 1 Heterogeneous Graph Data Construction

[0061] Data definition: Heterogeneous Graph is a relational graph data, which is characterized by containing multiple types of nodes and edges. These different types of nodes and edges have different characteristics or attributes.

[0062] Missing features: Due to high cost and privacy issues, it is difficult to collect the attributes of all nodes. Take the benchmark ACM dataset as an example: the heterogeneous graph modeled by ACM consists of three types of nodes: paper, author, and topic. The attributes of the paper node are derived from the keywords in its title, while the other two types of nodes lack attributes.

[0063] Inaccurate features: Data features collected manually often have inaccurate features. For example, in datasets such as ACM, the attributes of paper nodes are usually extracted from the bag-of-words representation of their keywords. However, some words that are not helpful in expressing the topic may be included, or some words may be mislabeled.

[0064] Data preprocessing

[0065] Based on the above definition, the present invention needs to perform an operation to initialize the data. Heterogeneous graph data contains attribute nodes and non-attribute nodes. For nodes with attributes, manually collected attributes are used as feature matrices; for nodes without attributes, all types of nodes without attributes are screened and the number of nodes under each type is counted, and unique hot encoding is used to generate feature matrices of each type. The feature matrices of nodes with attributes and nodes without attributes are used as initial features.

[0066] For example, the ACM dataset contains 7167 attribute-free nodes of type A (Author) and 60 attribute-free nodes of type S (Subject). Unit diagonal matrices of 7167 dimensions and 60 dimensions are established for the two node types respectively, and are used together with the feature matrix of attributed nodes of type P (Paper) as the initial features of the data.

[0067] The nodes of the heterogeneous graph data ACM are divided into training set and test set according to the ratio of 8:2.

[0068] 2 Data Dimensionality Reduction

[0069] Through a fully connected network (MLP1), the initial features obtained in step 1 are reduced to a lower dimension (64, 128). Low-dimensional features reduce the time complexity of the model and reduce the model's memory requirements.

[0070] 3. Two-layer variational graph autoencoder:

[0071] The two-layer variational graph autoencoder (Semi-Implicit Graph Variational Auto-Encoders, SIG-VAE) extends the variational graph autoencoders (VGAE) to model graph data. The SIG-VAE hierarchical variation framework adopts a randomly generated node embedding model to learn implicit posteriors while maintaining simple optimization. This hierarchical structure not only provides a more flexible graph generation model, but also captures the properties of nodes on the graph. SIG-VAE can solve problems such as long tails and skew peaks that VGAE could not solve before. Compared with VGAE, the potential representation of the graph derived by SIG-VAE is more interpretable.

[0072] Reasoning Model:

[0073] Node level:

[0074] Input: Concatenate the low-dimensional representations of all types of nodes in step 2 to obtain a feature matrix with the number of behavior nodes and columns as feature dimensions.

[0075] Training: Using the graph neural network (HGNN1), based on the different relationships of the heterogeneous graph, we build an attention network under each relationship, input the concatenated feature matrix into the graph attention network, and obtain the first-layer embedded representation.

[0076] A Gaussian noise distribution matrix is ​​sampled from a standard Gaussian distribution, and concatenated with the embedding representation matrix obtained in the previous step to obtain a concatenated matrix of the embedding matrix and the noise matrix. This matrix is ​​input into the attention network of the heterogeneous graph to obtain the mean and variance matrices, which are used to fit the standard normal distribution in the subsequent variational autoencoder latent space.

[0077] Attribute level:

[0078] Input: Concatenate the low-dimensional representations of all types of nodes in step 2 and transpose them to obtain a feature matrix with behavioral feature dimensions and the number of nodes as columns.

[0079] Training: Use a multi-layer perceptron (MLP2) to input the feature matrix obtained in the previous step into the multi-layer perceptron to obtain the embedding representation of the first layer.

[0080] A Gaussian noise distribution matrix is ​​sampled from the standard Gaussian distribution, and is concatenated with the embedding representation matrix obtained in the previous step to obtain a concatenated matrix of the embedding matrix and the noise matrix. This matrix is ​​input into the second layer network to obtain the mean and variance matrices, which are used to fit the standard normal distribution in the subsequent variational autoencoder latent space.

[0081] 4 Generate model:

[0082] The inner product decoder restores the structural information and low-dimensional feature information of the heterogeneous graph.

[0083] Node level:

[0084] Restore structural information: After reparameterizing the node-level mean and variance obtained in step 3, a continuous and smooth latent space representation is obtained. Using the obtained matrices of all types of nodes, inner product calculations are performed on each of them and the values ​​are controlled between 0 and 1 through the sigmoid activation function, thereby obtaining a "node-node" adjacency relationship matrix under a certain relationship and restoring the edge information in the heterogeneous graph.

[0085] Attribute level:

[0086] Restore low-dimensional feature information: After reparameterizing the attribute-level mean and variance obtained in step 3, the obtained matrix representation is inner-producted with the node-level matrix representation obtained in step 3 to obtain a low-dimensional representation matrix of "node-feature". Use the tanh activation function to activate the obtained matrix, align the low-dimensional representation in step 2, and restore the low-dimensional feature information.

[0087] Abstract features: Use graph neural network (HGNN2) to abstract the restored low-dimensional feature information to obtain features with more semantic information.

[0088] 5 Restore the attribute node features:

[0089] Use a multi-layer perceptron (MLP3) to filter node representations that meet the attribute conditions, such as nodes of type P (Paper) in the ACM dataset, map them to the original high-dimensional feature space, and align the initial features.

[0090] 6. Model Optimization:

[0091] For the side information and low-dimensional feature information in step 4, reconstruction loss and KL divergence are used, that is, The loss calculation is performed to restore the initial useful information with the maximum probability and constrain the distribution of the latent space. For the high-dimensional features of the attributed nodes obtained in step 5, they are aligned with the original features through the mean square error.

[0092] Cross Entropy

[0093] Cross-Entropy is a method commonly used to measure the difference between two probability distributions, especially in machine learning and deep learning, to evaluate the performance of classification models. It comes from the concept of information theory and is used to quantify the difference between the predicted distribution and the actual distribution.

[0094] The present invention uses cross entropy as the edge reconstruction loss. The link prediction task is to predict whether there is a link between nodes, and the model output is often the probability of the link existing. When the graph neural network is used for link prediction, the model will output a probability value between 0 and 1 for each pair of nodes, indicating the possibility of a link between the pair of nodes. Cross entropy can handle this probabilistic output well because it is a loss function defined based on probability distribution.

[0095] KL divergence

[0096] KL divergence (Kullback-Leibler Divergence), also known as relative entropy, is used to measure the difference between two probability distributions. By minimizing KL divergence, the present invention makes the variational distribution as close as possible to the true posterior distribution. This helps the model learn a more meaningful latent space representation.

[0097] Mean square error

[0098] Mean Squared Error (MSE) is a commonly used loss function that measures the difference between the predicted value and the true value and is often used in regression tasks. The alignment of the generated features with the original features in the present invention is a regression problem. The mean squared error can perceive outliers and help the model find the optimal parameters so that the predicted value is as close to the true value as possible.

[0099] 7 Classification Model Evaluation

[0100] Dataset Partitioning

[0101] The ratio of training set to test set is 8:2, and stratified sampling is used to ensure that the training set and test set can cover samples of each category.

[0102] Evaluation results

[0103] The present invention "heterogeneous graph node classification method based on double-layer variational autoencoder" is used to train all data and obtain embedded representation. Then, the embedded representation of the training set is trained using the SVM classifier, and the classification result is obtained using the test set.

[0104] In the heterogeneous graph node classification method based on the two-layer variational autoencoder, hyperparameters play a vital role in the classification model training. The hyperparameter settings directly affect the performance and convergence speed of the two-layer variational autoencoder. The important hyperparameters are introduced as follows:

[0105] 1) lr, model learning rate, the value range is {0.001, 0.005, 0.01, 0.05}

[0106] 2) Dropout, neuron dropout rate, the value range is {0.0, 0.3}

[0107] 3) K, the number of multi-head attention heads, ranging from {1, 2, 4, 8}

[0108] 4) d, the dimension of the embedding space, the value range is {32, 64, 128, 256}

[0109] 5) 𝜆1, 𝜆2, weight parameters of the loss function, ranging from [0,1]

[0110] The present invention uses a 32G V100 GPU to complete all experiments. Taking the node classification task as an example, Macro-F1 and Micro-F1 are used to evaluate the classification effect. The two indicators comprehensively consider the Precision and Recall of the model. Macro-F1 focuses on the performance of each category, and Micro-F1 calculates all categories as a whole. After training, the present invention is evaluated on 80% of the labeled training samples in the SVM classifier. On the ACM dataset, the double-layer variational autoencoder is tested using the test set, Macro-F1=93.49%, Micro-F1=93.38%. It can be seen that the present invention has good performance in multiple evaluation indicators.

[0111] Comparative experiment

[0112] In order to explore the efficiency of the present invention in data training, the heterogeneous graph node classification method of the double-layer variational autoencoder (Ours) is compared with the comparative learning model HGCA to observe the performance results on the DBLP dataset. Figure 3 As shown. It is found that the heterogeneous graph node classification method proposed in the present invention converges quickly and shows consistent performance growth. On the DBLP dataset, it is nearly four times faster than the contrastive learning HGCA. This experiment shows that the heterogeneous graph node classification method based on the double-layer variational autoencoder not only has excellent experimental results, but also has good performance.

Claims

1. A heterogeneous graph node classification method based on a two-layer variational autoencoder, characterized in that: The steps include: Step 1: Obtain a heterogeneous graph dataset, and construct a training set and a test set according to the ratio of 8:

2. The heterogeneous graph consists of nodes with attributes and nodes without attributes. For nodes with attributes, manually collected attribute features are directly used, and for nodes without attributes, the data is preprocessed. Step 2: Build a heterogeneous graph node classification model based on a two-layer variational autoencoder; Step 3: Iteratively train the heterogeneous graph node classification model based on the double-layer variational autoencoder constructed in step 2 to obtain an embedded representation; Step 4: Use the training set to iteratively train the support vector machine classifier SVM to obtain the node classification results of the heterogeneous graph test set; where: Step 1 is to pre-process the data, specifically including: 1.1: Filter all types of nodes without attributes and count the number of nodes under each type; 1.2: Feature initialization: For each type of node without attributes, use one-hot encoding to generate the feature matrix of each type; 1.3: For nodes with attributes, the manually collected attributes are directly used as their feature matrix, and the feature matrix of nodes with attributes and the feature matrix of nodes without attributes are used as initial features. At this time, the initial features exist in a high-dimensional space; Step 2 describes the construction of a heterogeneous graph node classification model based on a two-layer variational autoencoder, which specifically includes: 2.1: Dimensionality reduction: Multi-layer perceptron (MLP) is used for dimensionality reduction. Given that the feature dimensions of each type of node are different, a layer of MLP network equal to the number of node types is used to reduce the dimensions of different types of nodes to the same dimension. 2.2: Inference model: The inference model consists of a node-level variational autoencoder and an attribute-level variational autoencoder. For node-level variational autoencoders: 2.2.1: Concatenate the low-dimensional representations of all types of nodes obtained in 2.1 to obtain a feature matrix with the number of behavior nodes and columns as feature dimensions; 2.2.2: Build a heterogeneous graph network: Based on the different relationships of the heterogeneous graph, build an attention network under each relationship, input the spliced ​​feature matrix into the graph attention network, and obtain the first layer of embedded representation; 2.2.3: Build a double-layer Gaussian distribution latent space: Sample the Gaussian noise distribution matrix from the standard Gaussian distribution, and concatenate it with the embedded representation obtained in 2.2.2 to obtain the concatenated matrix of the embedding matrix and the noise matrix; input it into the attention network of the second layer of heterogeneous graph to obtain the node-level feature mean and variance matrices respectively; For attribute-level variational autoencoders: 2.2.4: Concatenate the low-dimensional representations of all types of nodes in 2.1 and transpose them to obtain a feature matrix with behavioral feature dimensions and the number of nodes as columns; 2.2.5: Build a multi-layer perceptron: Use two layers of MLP; input the feature matrix of 2.2.4 into the first layer of MLP to obtain the embedding representation of the first layer; 2.2.6: Build a double-layer Gaussian distribution latent space: Sample the Gaussian noise distribution matrix from the standard Gaussian distribution, and concatenate it with the embedded representation obtained in 2.2.5 to obtain the concatenated matrix of the embedding matrix and the noise matrix; input it into the second-layer MLP network to obtain the feature mean and variance matrices at the attribute level respectively; 2.3: Generative model: The generative model consists of a structure decoder and a feature decoder; For the structure decoder: 2.3.1: The structural decoder is used to restore the structural information; after reparameterizing the node-level mean and variance obtained in 2.2.3, a continuous and smooth latent space representation is obtained, and the reparameterized mean matrix of all types of nodes is obtained; 2.3.2: Using the reparameterized mean matrix of all types of nodes obtained in 2.3.1, perform inner product calculations on all different node types and control the calculated values ​​between 0 and 1 through the sigmoid activation function to obtain the node-node adjacency relationship matrix of two types of nodes, i.e., one relationship, and restore the structural information in the heterogeneous graph; For the feature decoder: 2.3.3: The feature decoder is used to restore the low-dimensional feature space in 2.1; After reparameterizing the attribute-level mean and variance obtained in 2.2.6, a continuous and smooth latent space representation is obtained, and the reparameterized mean matrix of the attribute is obtained; 2.3.4: Use the attribute reparameterized mean matrix obtained in 2.3.3 to perform inner product calculation with the node-level feature mean matrix obtained in 2.2.3 and use the tanh activation function to control the calculated value between -1 and 1 to obtain the low-dimensional feature matrix of "node-attribute" and restore the low-dimensional feature space in the heterogeneous graph; 2.3.5: Align initial features: Filter the low-dimensional representations of attribute node types in 2.3.4, map them to the high-dimensional feature space in the initial 1.3 through MLP, and align the initial features; Step 3 iteratively trains the heterogeneous graph node classification model based on the double-layer variational autoencoder constructed in step 2, specifically including: 3.1: Define the loss function of the heterogeneous graph node classification model; 3.1.1: For node-level variational autoencoders, the loss function is L_edge, which consists of cross entropy and KL divergence. The purpose is to constrain the distribution of the latent space and extract structural information; 3.1.2: For the attribute-level variational autoencoder, the loss function is L_attr, which consists of mean square loss and KL, the purpose is to constrain the distribution of the latent space and extract the semantic information of the low-dimensional feature space; 3.1.3: For feature alignment in 2.3.5, the loss function is L_rmse, and the mean square error is used to align high-dimensional original attributes in order to enhance the feature information of the original attributed nodes; 3.1.4: The total loss function of the heterogeneous graph node classification model is: L=L_edge+λ_1 L_attr+λ_2 L_rmse, where, λ_1 and λ_2 are adjustable model loss balance parameters; 3.2: Input the initialization data set in step 1 to the model, and iteratively train the heterogeneous graph node classification model using the loss function in 3.1; The step 4 specifically includes: 4.1: Use the representation in step 3 corresponding to the training set in step 1 to train the SVM classifier and obtain the classification results of the test set nodes in step 1.

Images