Scalable hidden equidistant manifold citation network node classification method and system
By introducing scalable hidden isometric manifold autoencoder and flow model in deep learning, combining metapathic paths and position features, the problem of deep learning destroying geometric structures in the process of mining citation network nodes is solved, and the accuracy of node classification is improved Sex and robustness.
Patent Information
- Application Number
- CN202410954619.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Deep learning will destroy the geometric structure of the original space in the process of mining citation network nodes, and will not be able to capture the manifold structure between samples, resulting in a decrease in the accuracy and robustness of node classification.
Using scalable hidden isometric manifold autoencoder, by introducing stream models and scaling isometric regularization terms, learning data manifolds and maintaining potential spatial coordinates of geometry, combining metapathic paths and position feature extraction, the total loss function is constructed to optimize Model parameters.
Effectively capture the manifold structure between samples, improves the accuracy and robustness of the citation network node classification, and solves the problem of model performance degradation in classification tasks of high-dimensional data.
Smart Images

Figure CN118964702A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of heterogeneous graph processing, and relates to a citation network node classification method and system for scalable implicit isometric manifolds. Background Art
[0002] The problem of citation network node classification has always been a hot topic and a difficult problem in research. The research background of citation network node classification mainly stems from the complex relational data in the real world, which usually contains multiple types of nodes and edges to form a citation network. Real-world graphs are always heterogeneous, and the recent success of heterogeneous graph masked autoencoders in representation learning can be largely traced back to the addition of a regularization term. This regularized autoencoder "constrains" the representation to prevent overfitting the data while producing a minimalistic generative model. In response to the problem that deep learning mining information between data may destroy the geometric structure of the original space and fail to capture the manifold structure between samples, the regularized autoencoder should in principle learn not only the data manifold, but also a set of geometry-preserving coordinates of the latent representation space, which means that the latent space representation should try to maintain the relative distances and angles on the data manifold. In order to learn the data manifold and a set of more isometric latent space coordinates that preserve geometry, thereby retaining the original structural information between the data, a scalable isometric manifold heterogeneous graph mask autoencoder is established. The scalable isometric regularization term conforms to the coordinate invariance and will not change the regularization term loss due to the change of the node feature coordinates. The idea of the flow model is introduced to further flatten the latent space learned by the autoencoder and map it to a more isometric representation space without causing any further loss of reconstruction accuracy.
[0003] In machine learning, isometric learning is a mapping between two spaces that preserves distances and angles everywhere. For a linear mapping between two vector spaces with inner products, isometric learning preserves the inner product everywhere. Moreover, most regularization terms do not conform to coordinate invariance, and the regularization term loss will change due to changes in node feature coordinates.
[0004] Before the present invention, deep learning would destroy the geometric structure of the original space in the process of mining citation network nodes, and could not capture the manifold structure between samples. Currently, most citation network node classifications use heterogeneous graph mask autoencoders without adding regularization terms, and the regularization terms do not conform to coordinate invariance, resulting in the inability to capture the manifold structure between samples, thereby affecting the training of scalable hidden isometric manifold autoencoders and the accuracy of citation network node classification methods. Summary of the invention
[0005] Purpose of the invention: The purpose of the present invention is to overcome the problem that deep learning mining of information between data will destroy the geometric structure of the original space and cannot capture the manifold structure between samples, and to provide a citation network node classification method and system for scalable hidden isometric manifolds to solve the problem that the citation network nodes cannot capture the manifold structure between samples during feature extraction, resulting in serious degradation of model performance, improve the accuracy and robustness of high-dimensional data classification tasks, and while solving complex high-dimensional data, retain the original structural information between data.
[0006] Technical solution: To achieve the above-mentioned invention object, the present invention adopts the following technical solution: a citation network node classification method of scalable hidden isometric manifold, comprising the following steps:
[0007] Given a citation network graph dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, Φ is the set of meta-paths; meta-path φ∈Φ is a path connecting different types of nodes with different types of edges; construct an encoder f E , RealNVP flow model i and its inverse model i -1 、MLP model f MLP and decoder f D Scalable Hidden Isometric Manifold Autoencoder;
[0008] Using dynamic masking rate to obtain the adjacency matrix of masked meta-paths Will and node attribute X input encoder f E , get the potential node embedding Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix The loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path
[0009] A subset of nodes of target node type t Sampling is performed, and a learnable mask mark [M] is used to mask each attribute of the subset to obtain a masked attribute matrix Will Input encoder f E , get the potential node embedding Through the flow model i Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the recovered node attribute Z is obtained and compared with the target attribute X using a scaled cosine error. v and reconstruct the attribute Z v , and get the loss of target attribute recovery
[0010] Use Mp2vec to extract meta-path-aware position features P, and then The decoder obtains the position feature P′ predicted by the model, and uses a scaled cosine error to compare the target position feature P and the predicted position feature P′ to obtain the position feature prediction loss;
[0011] Construct a scaled isometry regularization term, combined with and The loss constitutes the total loss, and the weight parameters of the scalable hidden isometric manifold autoencoder are optimized through the back-propagation algorithm;
[0012] The pre-trained scalable hidden isometric manifold autoencoder is used to obtain the hidden layer features, which are then converted into a new set of potential coordinates through the flow model i for training the classifier. Finally, the nodes of the test set citation network are classified to obtain the classification results.
[0013] Furthermore, the combined loss of the edge reconstruction based on the meta-path Specifically: the dynamic masking rate increases linearly with the period, and the adjacency matrix of the masked meta-path is obtained
[0014]
[0015] in, is the dynamic mask rate, A φ is the adjacency matrix based on meta-paths.
[0016] Will and node attribute X input encoder f E , get the potential node embedding The specific formula is as follows:
[0017]
[0018] Will have of Send to decoder f D , generate the decoded node embedding H φ . , using H φTo reconstruct the adjacency matrix, the specific formula is as follows:
[0019]
[0020] in, is the reconstructed meta-path-based adjacency matrix, and σ is the sigmoid activation function. A scaled cosine error is used to compare the target of each node and reconstruction The specific formula is as follows:
[0021]
[0022] in, is the loss of the meta-path φ, γ1 is the scaling factor, |A φ | is the number of adjacency matrices based on meta-paths, For each node target and reconstruction In order to merge each element path φ∈Φ A semantic-level attention vector q is introduced to automatically learn the importance score s of each meta-path φ The scores of all meta-paths are normalized using the softmax function to obtain the weight of the meta-path φ, denoted as α φ , the specific formula is as follows:
[0023]
[0024] Where W is the weight matrix and b is the bias vector. The loss of each meta-path is fused to obtain the combined loss of edge reconstruction based on the meta-path
[0025]
[0026] Furthermore, the loss of target attribute recovery Specifically: a subset of nodes of target node type t To perform sampling, a learnable mask tag [M] is used to mask each attribute of the subset. Accordingly, for each Masking attribute matrix Node attributes in It can be expressed as:
[0027]
[0028] Among them, x [M] Represents the node x in the subset v Mask, i is a reversible RealNVP flow model, Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the restored node attribute Z is obtained, which is specifically expressed as follows:
[0029]
[0030] Compare the target attribute X using a scaled cosine error v and reconstruct the attribute Z v , and get the loss of target attribute recovery
[0031]
[0032] Among them, γ2 is the scaling factor, is the number of mask nodes, <X v ,Z v > is the target attribute X v and reconstruct the attribute Z v is the inner product of , ‖·‖ is the vector bi-norm.
[0033] Furthermore, the position feature prediction The loss is as follows: use Mp2vec to extract the meta-path-aware position feature P, and Input decoder, decoder consists of i -1 Stream model and MLP model composition Then The decoder obtains the position features predicted by the model ± ′ , specifically expressed as follows:
[0034]
[0035] Among them, ± ′ Represents the predicted node position features.
[0036] Use a scaled cosine error to compare the target position feature P and the predicted position feature ± ′ , get the position feature prediction Loss:
[0037]
[0038] Among them, γ3 is the scaling factor, is the number of nodes, <P v ,c′ v >For each node, the target position feature P v and the predicted position feature P′ v is the inner product of , ‖·‖ is the vector bi-norm.
[0039] Furthermore, the scaling isometry regularization term is specifically: in the case of mapping between Riemann manifolds, f MLP : For smooth mapping, is a local coordinate The Riemannian manifold, whose Riemann metric is a local coordinate The Riemannian manifold of f MLP The Jacobian matrix of the differential express:
[0040]
[0041] Among them, c is the scaling factor, and m and n are the dimensions of the node.
[0042] Construct a scaled isometry regularizer:
[0043]
[0044]
[0045] in, Is The eigenvalue calculated by eigenvalue decomposition, τ(λ)=(λ-1) 2 , is the data distribution obtained by the encoder, It conforms to the standard normal distribution, and the second term is a regularization term to prevent i from diverging.
[0046] Furthermore, the total loss is the combined loss of the edge reconstruction of the meta-paths Target attribute recovery loss Position feature prediction loss And the scaled isometry regularization term, expressed as:
[0047]
[0048] Among them, α and β are hyperparameters for adjusting the loss of each part. is the number of nodes.
[0049] Furthermore, the training classifier is specifically as follows: the citation network training set is input into the pre-trained scalable hidden isometric manifold autoencoder to obtain hidden layer features, and then the hidden layer features are converted into a set of new potential coordinates through the flow model i, and then input into the classification module to obtain the category feature vector, and finally input into the softmax classifier to obtain the predicted label y pred , calculate ypred and the true label y label The loss between , using cross entropy as the loss function To train the model. The specific formula is as follows:
[0050]
[0051] Based on the same inventive concept, the present invention provides a citation network node classification system of scalable hidden isometric manifold, comprising:
[0052] Preprocessing module, given a citation network dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, Φ is the set of meta-paths; meta-path φ∈Φ is a path connecting different types of nodes with different types of edges; construct an encoder f E , RealNVP flow model i and its inverse model i -1 、MLP model f MLP and decoder f D Scalable Hidden Isometric Manifold Autoencoder;
[0053] A combination module for edge reconstruction based on meta-paths, used to obtain a combination loss for edge reconstruction based on meta-paths; an adjacency matrix based on masked meta-paths is obtained using a dynamic masking rate Will and node attribute X input encoder f E , get the potential node embedding Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix The loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path
[0054] The target attribute recovery module is used to obtain the loss of target attribute recovery; for a subset of nodes of target node type t Sampling is performed, and a learnable mask mark [M] is used to mask each attribute of the subset to obtain a masked attribute matrix Will Input encoder f E , get the potential node embedding Through the flow model i Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the recovered node attributes Z are obtained and compared with the target attribute matrix X using a scaled cosine error. v and reconstruct the attribute matrix Z v , and get the loss of target attribute recovery
[0055] The position feature prediction module is used to obtain the loss of position feature prediction; Mp2vec is used to extract the meta-path-aware position feature P, and then After the decoder, the position feature P predicted by the model is obtained ′ , using a scaled cosine error to compare the target position feature P and the predicted position feature P ′ , get the position feature prediction loss;
[0056] Scaling isotropic regularization module, constructing scaling isotropic regularization, combined with and The loss constitutes the total loss, and the weight parameters of the scalable hidden isometric manifold autoencoder are optimized through the back-propagation algorithm;
[0057] The classification model training module uses the pre-trained scalable hidden isometric manifold autoencoder to obtain the hidden layer features, and then converts the hidden layer features into a new set of potential coordinates through the flow model i for training the classifier;
[0058] And the testing module uses the classification model to classify the test set citation network nodes and obtain the classification results.
[0059] Based on the same inventive concept, the present invention provides a computer system comprising a memory, a processor, and a computer program / instruction stored in the memory and executable on the processor, wherein when the computer program / instruction is executed by the processor, the steps of the citation network node classification method of the scalable implicit isometric manifold are implemented.
[0060] Based on the same inventive concept, the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the citation network node classification method of the scalable implicit isometric manifold.
[0061] Beneficial effects: The advantages and effects of the present invention are to overcome the problem that deep learning mining of information between data will destroy the geometric structure of the original space and cannot capture the manifold structure between samples, and provide a citation network node classification method and system for scalable hidden isometric manifolds to solve the problem that the citation network nodes cannot capture the manifold structure between samples during feature extraction, resulting in serious degradation of model performance, improve the accuracy and robustness of high-dimensional data classification tasks, and while solving complex high-dimensional data, retain the original structural information between data.
[0062] The present invention adopts scalable hidden isometric manifolds for citation network node classification. The model combines manifold learning in machine learning and autoencoders in popular deep learning into a network framework, introduces coordinate invariance and flow model ideas, uses autoencoders to learn data manifolds and a set of more isometric latent space coordinates that maintain geometry, uses pre-trained scalable hidden isometric manifold autoencoders to obtain hidden layer features, and then converts the hidden layer features into a set of new latent coordinates through flow model i for training classifiers, and finally uses the classification model to classify the test set citation network nodes to obtain classification results. This method effectively solves the problem that the current deep learning will destroy the geometric structure of the original space in the process of mining citation network nodes, thereby capturing the manifold structure between sample nodes and improving the accuracy of the citation network node classification method. Therefore, the method of the present invention can achieve better performance in heterogeneous graph node classification tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0063] Figure 1 Detailed framework diagram of an embodiment of the present invention. In the figure, nodes a∈{a1,…,a b} is the author, and the node p∈{p1,…,p e} is a paper, and the node s∈{s1,…,s g} is the topic of the paper. p is divided into three categories: database, wireless communication, and data mining. In the figure, the adjacency matrix A and feature matrix X of p are input into the scalable hidden isometric manifold autoencoder for training. The pre-trained scalable hidden isometric manifold autoencoder is used to obtain the hidden layer features of p. The hidden layer features are then converted into a set of new potential coordinates through the flow model i for training the classifier. Finally, the node p is classified in the test set citation network to obtain the classification results. DETAILED DESCRIPTION
[0064] The scheme of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0065] like Figure 1As shown, an embodiment of the present invention discloses a citation network node classification method of scalable hidden isometric manifold. The present invention first develops a meta-path mask with a dynamic masking rate, which is used to reconstruct the adjacency matrix composed of the edges of the meta-path; secondly, the dynamic masking rate is used to learn node attribute recovery and explore the content information involved in the node attributes, so that the model can focus on the target node type; then a position feature prediction strategy is designed to integrate the position information of each node; then a scalable isometric regularization term is constructed, and finally the total loss on the citation network dataset is defined by the above four parts of loss to train the scalable hidden isometric manifold autoencoder. The present invention effectively solves the problem that the current deep learning will destroy the geometric structure of the original space in the process of mining citation network nodes, thereby capturing the manifold structure between sample nodes and improving the accuracy of citation network node classification. Therefore, the method of the present invention can achieve better performance in the task of citation network node classification.
[0066] The embodiment of the present invention discloses a citation network node classification method of a scalable implicit isometric manifold, which specifically includes the following steps:
[0067] (1) Given a citation network graph dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, and Φ is the set of meta-paths. A meta-path φ∈Φ is a path connecting different types of nodes with different types of edges, that is, in, etc. are different types of nodes, etc. are different types of edges, and l is the path length. The citation network dataset is divided into training set, validation set and test set. All nodes in the training set are labeled to train the scalable hidden isometric manifold autoencoder, and then perform the node classification task; the scalable hidden isometric manifold autoencoder consists of an encoder network f E (·), RealNVP flow model i(·) and the inverse model i of the flow model -1 (·), MLP model and decoder f D (·)constitute;
[0068] (2) First, we use the dynamic mask rate The adjacency matrix of the masked meta-path is obtained Then, we will and node attribute X input encoder f E , get the potential node embedding Afterwards, it will have of Send to decoder f D, generate the decoded node embedding H φ , then we use H φ To reconstruct the adjacency matrix Afterwards, the loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path
[0069] (3) Secondly, we use dynamic mask rate to learn node attribute recovery and explore the content information involved in node attributes, so that the model can focus on the target node type. We select a subset of nodes of the target node type t. Then, we use a learnable mask tag [M] to mask each attribute of the subset to obtain the masked attribute matrix Will Input encoder f E , get the potential node embedding Will Transformed into a new set of latent coordinates Next, we will add A and Send to decoder f D In the above example, the recovered node attribute Z is obtained, and then a scaled cosine error is used to compare the target attribute X v and reconstruct the attribute Z v , and get the loss of target attribute recovery
[0070] (4) Then, in order to integrate the location information of each node, a location feature prediction strategy is designed. We use Mp2vec to extract the meta-path-aware location feature P, which generates node sequences through random walks based on meta-paths and uses the Skip-Gram model to learn the embedding representation of each node. Each node will be mapped to a low-dimensional space. These vector representations can be regarded as the location features of the node. Then After the decoder, the position feature P predicted by the model is obtained ′ Then, a scaled cosine error is used to compare the target position feature P and the predicted position feature P ′ , get the position feature prediction loss;
[0071] (5) Construct a scalable isometry regularization term, which is composed of the total loss of the above four parts, and optimize the weight parameters of the scalable hidden isometry manifold autoencoder through the back propagation algorithm;
[0072] (6) Use the pre-trained scalable hidden isometric manifold autoencoder to obtain the hidden layer features, and then convert the hidden layer features into a new set of latent coordinates through the flow model i for training the classifier. Finally, perform node classification on the test set citation network to obtain the classification results.
[0073] The combined loss of the meta-path-based edge reconstruction in step (2) Specifically:
[0074] The dynamic masking rate increases linearly with the period, and the adjacency matrix of the masked meta-path is obtained
[0075]
[0076] in, represents the dynamic mask rate, which ranges from [0.5, 0.8] and increases linearly with the period in steps of 0.005. φ is the adjacency matrix based on meta-paths.
[0077] Then, we will and node attribute X input encoder f E , get the potential node embedding The specific formula is as follows:
[0078]
[0079] Then, we will have of Send to decoder f D , generate the decoded node embedding H φ Afterwards, we use H φ To reconstruct the adjacency matrix, the specific formula is as follows:
[0080]
[0081] in, is the reconstructed meta-path-based adjacency matrix, and σ is the sigmoid activation function. Next, we will use a scaled cosine error to compare the target of each node and reconstruction The specific formula is as follows:
[0082]
[0083] in, is the loss of the meta-path φ, γ1 is the scaling factor, |A φ | is the number of adjacency matrices based on meta-paths, For each node target and reconstruction The inner product of , ‖·‖ is the vector binorm, indicating the length of the vector. In order to merge each element path φ∈Φ We first introduce a semantic-level attention vector q to automatically learn the importance score s for each meta-path.φ Then, we use the softmax function to normalize the scores of all meta-paths and obtain the weight of the meta-path φ, denoted as α φ , the specific formula is as follows:
[0084]
[0085] Where W is the weight matrix and b is the bias vector. Then, we fuse the losses of each meta-path to obtain the combined loss of edge reconstruction based on the meta-path.
[0086]
[0087] The loss of target attribute recovery in step (3) Specifically: we have a subset of nodes of target node type t Then, we use a learnable mask tag [M] to mask each attribute of the subset. Accordingly, for each Masking attribute matrix Node attributes in It can be expressed as:
[0088]
[0089] Among them, x [M] Represents the node x in the subset v Mask, i is a reversible RealNVP flow model, Transformed into a new set of latent coordinates Next, we will add A and Send to decoder f D In the above example, the restored node attribute Z is obtained, which is specifically expressed as follows:
[0090]
[0091] Then, a scaled cosine error is used to compare the target attribute X v and reconstruct the attribute Z v , and get the loss of target attribute recovery
[0092]
[0093] Among them, γ2 is the scaling factor, is the number of mask nodes, <X v ,Z v > is the target attribute X v and reconstruct the attribute Z v is the inner product of , ‖·‖ is the vector bi-norm, indicating the length of the vector.
[0094] Position feature prediction in step (4) Specifically, we use Mp2vec to extract the meta-path-aware position feature P, which generates node sequences through random walks based on meta-paths and uses the Skip-Gram model to learn the embedding representation of each node. Each node is mapped to a low-dimensional space. These vector representations can be regarded as the position features of the nodes. The decoder consists of i -1 Flow model and MLP model composition Then After the decoder, the position feature P predicted by the model is obtained ′ , specifically expressed as follows:
[0095]
[0096] Among them, P ′ Represents the predicted node position feature. Then, a scaled cosine error is used to compare the target position feature P and the predicted position feature P′ to obtain the position feature prediction Loss:
[0097]
[0098] Among them, γ3 is the scaling factor, is the number of nodes, <P v ,P′ v >For each node, the target position feature P v and the predicted position feature P′ v is the inner product of , ‖·‖ is the vector bi-norm, indicating the length of the vector.
[0099] The scaling isometry regularization term in step (5) is: In the case of mapping between Riemannian manifolds, f MLP : For smooth mapping, is a local coordinate The Riemannian manifold, whose Riemann metric is a local coordinate The Riemannian manifold, whose Riemann metric f MLP The Jacobian matrix of the differential express:
[0100]
[0101] Where c is the scaling factor.
[0102] Construct a scaled isometry regularizer:
[0103]
[0104] in, Is The eigenvalues calculated by eigenvalue decomposition, where c is the scaling factor, τ(λ) = (λ-1) 2 , is the data distribution obtained by the encoder, It conforms to the standard normal distribution, and the second term is a regularization term to prevent i from diverging.
[0105] Furthermore, in step (5), the total loss is the combined loss of the edge reconstruction of the meta-path Target attribute recovery loss Position feature prediction loss And the scaled isometry regularization term, expressed as:
[0106]
[0107] Among them, α and β are hyperparameters for adjusting the loss of each part. is the number of nodes.
[0108] In step (6), the classifier is trained by: inputting the citation network training set into the pre-trained scalable hidden isometric manifold autoencoder to obtain the hidden layer features, then converting the hidden layer features into a new set of potential coordinates, inputting them into the classification module to obtain the category feature vector, and finally inputting them into the softmax classifier to obtain the predicted label y pred , calculate y pred and the true label y label The loss between , using cross entropy as the loss function To train the model. The specific formula is as follows:
[0109]
[0110] To demonstrate the effectiveness of the present invention, the present invention is compared with existing methods on different heterogeneous graph datasets.
[0111] As shown in Table 1, the table shows the node classification of the present invention (abbreviated as SLIMHG in English) under 4 heterogeneous graph data sets. The DBLP data set contains 4 types of nodes, author, paper, term and venue; the Freebase data set contains 4 types of nodes, movie, actor, director and producer; the ACM data set contains 3 types of nodes, paper, author and conference subject; the Aminer data set contains 4 types of nodes, author, paper, conference and reference. The comparison algorithm data above the table comes from the relevant author's articles. All experimental results are the average of 10 runs. The comparison algorithms in the table are all relatively classic network models. Specifically, we use 20, 40, and 60 labeled nodes for each class as training sets, 1,000 nodes each as validation sets and test sets, and use node classification performance Micro-F1, Macro-F1, and AUC as classification evaluation indicators, where Micro-F1 and Macro-F1 are the weighted harmonic means of precision and recall, and AUC is the area under the ROC curve, which can quantitatively reflect the model performance measured based on the ROC curve.
[0112] Table 1 Comparison of indicators (means) on different heterogeneous graph datasets at 20% label rate
[0113]
[0114]
[0115] Table 2 Comparison of indicators (means) on different heterogeneous graph datasets at 40% label rate
[0116]
[0117] Table 3 Comparison of indicators (means) on different heterogeneous graph datasets at 60% label rate
[0118]
[0119]
[0120] It can be seen from Tables 1-3 that, under the same label rate, SLIMHG has better node classification performance than other network models. By comparing the experimental results of the same data set at different label rates, it can be found that the classification performance of SLIMHG is not greatly affected by the label rate of the data set, which also indirectly reflects that the model has a high applicability to the label rate of the selected data set and has a certain stability. This shows the effectiveness of the present invention for heterogeneous graph node classification tasks.
[0121] Based on the same inventive concept, the present invention provides a heterogeneous graph node classification system with scalable hidden isometric manifolds, including: given a citation network dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, Φ is the set of meta-paths; meta-path φ∈Φ is a path connecting different types of nodes with different types of edges; construct an encoder f E , RealNVP flow model i and its inverse model i -1 、MLP model f MLP and decoder f D A scalable hidden isometric manifold autoencoder based on the edge reconstruction of the meta-path, which is used to obtain the combined loss of the edge reconstruction based on the meta-path; an adjacency matrix based on the masked meta-path is obtained by using a dynamic masking rate Will and node attribute X input encoder f E , get the potential node embedding Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix The loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path The target attribute recovery module is used to obtain the loss of target attribute recovery; for a subset of nodes of target node type t Sampling is performed, and a learnable mask mark [M] is used to mask each attribute of the subset to obtain a masked attribute matrix Will Input encoder f E , get the potential node embedding Through the flow model i Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the recovered node attributes Z are obtained and compared with the target attribute matrix X using a scaled cosine error. v and reconstruct the attribute matrix Z v , and get the loss of target attribute recovery The position feature prediction module is used to obtain the loss of position feature prediction; Mp2vec is used to extract the meta-path-aware position feature P, and then The decoder obtains the position feature P′ predicted by the model, and uses a scaled cosine error to compare the target position feature P with the predicted position feature P′ to obtain the position feature prediction The loss; scaling isometry regularization module, constructing scaling isometry regularization, combined with and The loss constitutes the total loss, and the weight parameters of the scalable latent isometric manifold autoencoder are optimized through the back-propagation algorithm; the classification model training module uses the pre-trained scalable latent isometric manifold autoencoder to obtain the hidden layer features, and then converts the hidden layer features into a set of new potential coordinates through the flow model i for training the classifier; and the testing module uses the classification model to classify the test set citation network nodes to obtain the classification results.
[0122] Based on the same inventive concept, an embodiment of the present invention provides a computer system, including a memory, a processor, and a computer program / instruction stored in the memory and executable on the processor. When the computer program / instruction is executed by the processor, the steps of the citation network node classification method of the scalable implicit isometric manifold are implemented.
[0123] Based on the same inventive concept, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the citation network node classification method of the scalable implicit isometric manifold.
Claims
1. A citation network node classification method based on a scalable implicit isometric manifold, characterized in that: The steps include: Given a citation network graph dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, Φ is the set of meta-paths; meta-path φ∈Φ is a path connecting different types of nodes with different types of edges; the construction includes the encoder f E , RealNVP flow model i and its inverse model i -1 、MLP model f MLP and decoder f D Scalable Hidden Isometric Manifold Autoencoder; Using dynamic masking rate to obtain the adjacency matrix of masked meta-paths Will and node attribute X input encoder f E , get the potential node embedding Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix The loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path A subset of nodes of target node type t Sampling is performed, and a learnable mask mark [M] is used to mask each attribute of the subset to obtain a masked attribute matrix Will Input encoder f E , get the potential node embedding Through the flow model i Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the recovered node attribute Z is obtained and compared with the target attribute X using a scaled cosine error. v and reconstruct the attribute Z v , and get the loss of target attribute recovery Use Mp2vec to extract meta-path-aware position features P, Through the flow model i -1 and the MLP model f MLP The decoder constructed obtains the position feature P′ predicted by the model, and uses a scaled cosine error to compare the target position feature P with the predicted position feature P′ to obtain the position feature prediction loss; Construct a scaled isometry regularization term, combined with and The loss constitutes the total loss, and the weight parameters of the scalable hidden isometric manifold autoencoder are optimized through the back-propagation algorithm; The pre-trained scalable hidden isometric manifold autoencoder is used to obtain the hidden layer features, which are then converted into a new set of potential coordinates through the flow model i for training the classifier. Finally, the nodes of the test set citation network are classified to obtain the classification results.
2. The citation network node classification method of scalable implicit isometric manifold according to claim 1, characterized in that: The combined loss of the meta-path based edge reconstruction Specifically: The dynamic masking rate increases linearly with the period, and the adjacency matrix of the masked meta-path is obtained in, is the dynamic mask rate, A φ is the adjacency matrix based on meta-path; Will and node attribute X input encoder f E , get the potential node embedding The specific formula is as follows: Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix, the specific formula is as follows: in, is the reconstructed meta-path-based adjacency matrix, σ is the sigmoid activation function; Use a scaled cosine error to compare the target of each node and reconstruction The specific formula is as follows: in, is the loss of the meta-path φ, γ1 is the scaling factor, |A φ | is the number of adjacency matrices based on meta-paths, For each node target and reconstruction The inner product of , ‖·‖ is the vector binorm; A semantic-level attention vector q is introduced to automatically learn the importance score s of each meta-path φ , use the softmax function to normalize the scores of all meta-paths to obtain the weight of the meta-path φ, denoted as α φ , the specific formula is as follows: Where W is the weight matrix and b is the bias vector; The loss of each meta-path is fused to obtain the combined loss of edge reconstruction based on the meta-path 3. The citation network node classification method of scalable implicit isometric manifold according to claim 1, characterized in that: The target attribute is restored at a loss Specifically: A subset of nodes of target node type t Sampling is performed, using a learnable mask tag [M] to mask each attribute of the subset. For each Masking attribute matrix Node attributes in It is expressed as: Among them, x [M] Represents the node x in the subset v Mask, i is a reversible RealNVP flow model, Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the restored node attribute Z is obtained, which is specifically expressed as follows: Compare the target attribute X using a scaled cosine error v and reconstruct the attribute Z v , and get the loss of target attribute recovery Among them, γ2 is the scaling factor, is the number of mask nodes, <X v ,Z v > is the target attribute X v and reconstruct the attribute Z v is the inner product of , ‖·‖ is the vector bi-norm.
4. The citation network node classification method of scalable implicit isometric manifold according to claim 1, characterized in that: The position feature prediction The loss is as follows: use Mp2vec to extract the meta-path-aware position feature P, and Input decoder, the decoder consists of i -1 Flow model and MLP model composition Then The position feature P′ predicted by the model is obtained through the decoder, which is specifically expressed as follows: Among them, P′ represents the predicted node position feature; Use a scaled cosine error to compare the target position feature P and the predicted position feature P′ to obtain the position feature prediction Loss: Among them, γ3 is the scaling factor, is the number of nodes, <P v ,P′ v >For each node, the target position feature P v and the predicted position feature P′ v is the inner product of , ‖·‖ is the vector bi-norm.
5. The citation network node classification method of scalable implicit isometric manifold according to claim 4, characterized in that: The scaling isometry regularization term is specifically: in the case of mapping between Riemann manifolds, f MLP : For smooth mapping, is a local coordinate The Riemannian manifold of is a local coordinate The Riemannian manifold of f MLP The Jacobian matrix of the differential express: Among them, c is the scaling factor, m and n are the dimensions of the node; Construct a scaled isometry regularizer: in, Is The eigenvalue calculated by eigenvalue decomposition, τ(λ)=(λ-1) 2 , is the data distribution obtained by the encoder, It conforms to the standard normal distribution, and the second term is a regularization term to prevent i from diverging.
6. The citation network node classification method of scalable implicit isometric manifold according to claim 5, characterized in that: The total loss is the combined loss of the edge reconstruction of the meta-paths Target attribute recovery loss Position feature prediction loss And the scaled isometry regularization term, expressed as: Among them, α and β are hyperparameters for adjusting the loss of each part. is the number of nodes.
7. The citation network node classification method of scalable implicit isometric manifold according to claim 1, characterized in that: The training classifier is specifically as follows: the citation network training set is input into the pre-trained scalable hidden isometric manifold autoencoder to obtain hidden layer features, and then the hidden layer features are converted into a set of new potential coordinates through the flow model i, and then input into the classification module to obtain the category feature vector, and finally input into the softmax classifier to obtain the predicted label y pred , calculate y pred and the true label y label The loss between , using cross entropy as the loss function To train the model, the specific formula is as follows:
8. A citation network node classification method system with scalable hidden isometric manifold, characterized in that: include: Preprocessing module, given a citation network dataset is the set of nodes of the graph, ε is the set of edges of the graph, A is the adjacency matrix, is the set of types of nodes in the graph, T ε is the set of edge types of the graph, X is the node feature matrix of the graph, Φ is the set of meta-paths; meta-path φ∈Φ is a path connecting different types of nodes with different types of edges; the construction includes the encoder f E , RealNVP flow model i and its inverse model i -1 、MLP model f MLP and decoder f D Scalable Hidden Isometric Manifold Autoencoder; A combination module for edge reconstruction based on meta-paths, used to obtain a combination loss for edge reconstruction based on meta-paths; an adjacency matrix based on masked meta-paths is obtained using a dynamic masking rate Will and node attribute X input encoder f E , get the potential node embedding Will have of Send to decoder f D , generate the decoded node embedding H φ , using H φ To reconstruct the adjacency matrix The loss of each meta-path is fused according to the weight to obtain the combined loss of edge reconstruction based on the meta-path A target attribute recovery module, used to obtain the loss of target attribute recovery; A subset of nodes of target node type t Sampling is performed, and a learnable mask mark [M] is used to mask each attribute of the subset to obtain a masked attribute matrix Will Input encoder f E , get the potential node embedding Through the flow model i Transformed into a new set of latent coordinates A and Send to decoder f D In the above example, the recovered node attributes Z are obtained and compared with the target attribute matrix X using a scaled cosine error. v and reconstruct the attribute matrix Z v , and get the loss of target attribute recovery The position feature prediction module is used to obtain the loss of position feature prediction; Mp2vec is used to extract the meta-path-aware position feature P, and then The decoder obtains the position feature P′ predicted by the model, and uses a scaled cosine error to compare the target position feature P with the predicted position feature P′ to obtain the position feature prediction loss; Scaling isotropic regularization module, constructing scaling isotropic regularization, combined with and The loss constitutes the total loss, and the weight parameters of the scalable hidden isometric manifold autoencoder are optimized through the back-propagation algorithm; The classification model training module uses the pre-trained scalable hidden isometric manifold autoencoder to obtain the hidden layer features, and then converts the hidden layer features into a new set of potential coordinates through the flow model i for training the classifier; And the testing module uses the classification model to classify the test set citation network nodes and obtain the classification results.
9. A computer system comprising a memory, a processor, and a computer program / instruction stored in the memory and executable on the processor, characterized in that: When the computer program / instructions are executed by a processor, the steps of the citation network node classification method of scalable implicit isometric manifold according to any one of claims 1-7 are implemented.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the citation network node classification method of scalable implicit isometric manifold according to any one of claims 1-7 are implemented.
