Adaptive enhanced staged graph representation learning method based on citation information

Abstract

The invention relates to the technical field of data processing, and provides an adaptive enhancement staged graph representation learning method based on quotation information, which adopts a data enhancement method and comprises a topological structure level adaptive enhancement method, which comprises the following steps: calculating the centrality of an edge based on a node centrality measurement function, generating a first sampling probability through a truncation probability, and generating a second sampling probability through a truncation probability; taking the first sampling probability as an edge deletion probability, and reserving the topological structure based on the edge deletion probability; the node attribute level adaptive enhancement method comprises the following steps: respectively calculating a corresponding first weight and a corresponding second weight based on discrete node features and continuous node features; and generating a second sampling probability through the truncation probability, taking the second sampling probability as a feature shielding probability, and retaining the feature dimension based on the feature shielding probability. According to the method, topological structure level adaptive enhancement and node attribute level adaptive enhancement are introduced on the basis of the GRLWPT to produce the comparison graph, so that the accuracy in a node classification task and the robustness of an algorithm are improved.

Description

Technical Field

[0001] This application relates to the technical field of data processing, and specifically relates to a phased graph representation learning method with adaptive enhancement based on citation information. Background Art

[0002] As an important representation form of entity relationships in the real world, the graph structure has typical significance in application scenarios such as academic citation networks. In the context of citation networks, the citation relationships between papers naturally form a graph topology structure. How to effectively mine the implicit academic association information in such graph data has become a key research direction of current graph representation learning technology. However, there are two major technical bottlenecks commonly existing in practical applications: (1) The problems of graph data sparsity and heterogeneity: Graph data in real scenarios often exhibit high-dimensional sparse features and non-uniform distribution characteristics, resulting in traditional data mining methods being difficult to effectively capture the potential association rules between nodes; (2) The defect of multi-dimensional similarity preservation: Graph nodes have dual attributes of neighborhood similarity (adjacent node features are similar) and structural similarity (topologically equivalent node features are similar). Existing graph representation learning methods mostly adopt a single similarity modeling strategy, which is difficult to achieve the collaborative preservation of the two types of features, significantly restricting the generalization performance of downstream tasks.

[0003] To address the above problems, the current mainstream solution mainly adopts a graph neural network (GNN) combined with a supervised learning framework. However, this method has significant limitations: First, the acquisition cost of high-quality labeled data is high, seriously restricting the scalability of the method; Second, supervised learning is prone to causing the model to overfit specific labeling tasks and is difficult to learn general knowledge representations with transfer and generalization capabilities.

[0004] Chinese invention patent CN117973489A discloses a phased graph representation learning method based on citation information. This method invented the GRLWPT algorithm, which improved the classification performance of the citation network by learning the neighborhood similarity and structural similarity of the graph in phases.

[0005] However, when performing visual analysis on different hyperparameter settings in GRLWPT, GRLWPT is sensitive to four hyperparameters for random data augmentation. The reason for this phenomenon is that a completely random graph data augmentation scheme will destroy the inherent semantics of the graph data. That is, the data augmentation processing method in step 3 of this technical solution is sensitive to hyperparameters, reducing the robustness of the algorithm. Summary of the Invention

[0006] To help solve the above technical problems, this application provides a phased graph representation learning method with adaptive enhancement based on citation information, adopting the following technical solutions: An adaptive enhanced phased graph representation learning method based on citation information, including a learning framework that learns node structure similarity and node neighborhood similarity in phases, extracts general node structure similarity information from data in different domains through subgraph contrast, and then learns the neighborhood similarity of nodes in a preset domain through node-level contrast. When using node-level contrast to learn the neighborhood similarity of nodes, an adaptive data augmentation method is adopted, including: A topological structure-level adaptive enhancement method, including calculating the centrality of edges based on a node centrality metric function, generating a first sampling probability through a truncation probability, using the first sampling probability as the edge deletion probability, and retaining the topological structure based on the edge deletion probability; A node attribute-level adaptive enhancement method, including calculating corresponding first weights and second weights based on discrete node features and continuous node features respectively; generating a second sampling probability through a truncation probability, using the second sampling probability as the feature masking probability, and retaining the feature dimension based on the feature masking probability.

[0007] Preferably, the learning framework that learns node structure similarity and node neighborhood similarity in phases, extracts general node structure similarity information from data in different domains through subgraph contrast, and then learns the neighborhood similarity of nodes in a preset domain through node-level contrast, includes: Step S1: Sample citation information subgraphs in the r-order neighborhood node set of the original graph; Step S2: The pre-trained model identifies the structural similarity of nodes through subgraph-level contrast learning, and obtains the citation network embedding vectors trained by the subgraph-level contrast learning model; Step S3: The pre-trained model identifies the structural similarity of nodes through node-level contrast learning, and obtains the citation network embedding vectors trained by the node-level contrast learning model; Step S4: Use a classifier to classify citations into similar paper topics according to the citation network embedding vectors trained in Step S2 and Step S3.

[0008] Preferably, the topological structure-level adaptive enhancement method is used to randomly remove edges from the graph according to the adaptively obtained probability, including: Step S311: Randomly obtain a subset from the edge set E according to the edge-taking probability , and the edge-taking probability P is calculated by the following formula: , where represents the first sampling probability of removing edge , is the connection set of the view obtained after the topological structure-level data augmentation method; Step S312: Set the metric function of node centrality , the centrality of an edge is defined as the average of the centralities of the two nodes it connects: , where, is the average of the centralities of the two nodes connected by the edge; Step S313: Calculate the first sampling probability in the following way , , , where, represents the maximum value of, represents the minimum value of, represents the hyperparameter that controls the overall probability of removing edges, represents the truncation probability.

[0009] Preferably, the node attribute-level adaptive enhancement method is used to add noise to node attributes by setting the values of some dimensions to zero in node features, including: Step S321: Sample using the random variable , and the value of each dimension of the random variable is obtained from the Bernoulli distribution: , is the second sampling probability, represents the dimension; Step S322: For any discrete node , the discrete node feature of is , calculate the first weight of dimension in the following way: , For any continuous node , the continuous node feature of the continuous node is , calculate the second weight of dimension in the following way: ; Step S323: Calculate the second sampling probability in the following way: , , where, represents the maximum value of, represents The average value, The hyperparameter representing the enhanced strength of the control feature, represents the truncation probability.

[0010] Preferably, the node centrality function is the degree centrality function or the eigenvector centrality function or the PageRank centrality function or the closeness centrality function.

[0011] In summary, the present application has the following advantages: (1) The present application proposes to introduce topological structure-level adaptive enhancement and node attribute-level adaptive enhancement on the basis of GRLWPT to generate comparison graphs, improving the accuracy in node classification tasks and the robustness of the algorithm.

[0012] (2) The present application realizes an unsupervised graph representation learning method, which is beneficial to be extended to various data sets. At the same time, it will also have a beneficial impact on common downstream tasks such as node classification and link prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is the overall flowchart of a phased graph representation learning method for citation network information of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] The following further illustrates the present application with reference to the drawings. The structure and principle of the present application are very clear to those skilled in the art. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0015] Figure 1 is the overall flowchart of a phased graph representation learning method for citation network information of the present invention. It can be seen from Figure 1 that the inventive point of a phased graph representation learning method based on citation information and adaptive enhancement of the present application lies in introducing a new data enhancement method in step S3.

[0016] The method of the present application includes a learning framework that learns node structure similarity and node neighborhood similarity in stages. By comparing subgraphs, it extracts general node structure similarity information from data in different domains. Then, through node-level comparison, it learns the neighborhood similarity of nodes in a preset domain. When using node-level comparison to learn the neighborhood similarity of nodes, an adaptive data augmentation method is adopted, including a topological structure-level adaptive augmentation method and a node attribute-level adaptive augmentation method. The topological structure-level adaptive augmentation method includes calculating the centrality of edges based on a node centrality metric function, generating a first sampling probability through a truncation probability, using the first sampling probability as the edge deletion probability, and retaining the topological structure based on the edge deletion probability. The node attribute-level adaptive augmentation method includes: calculating corresponding first and second weights based on discrete node features and continuous node features respectively; generating a second sampling probability through a truncation probability, using the second sampling probability as the feature masking probability, and retaining the feature dimension based on the feature masking probability. These two methods will be described in detail below.

[0017] The adaptive augmentation phased graph representation learning method based on citation information of the present application includes step S1: sampling a citation information subgraph from the r-order neighborhood node set of the original graph; step S2: a pre-trained model identifies the structural similarity of nodes through subgraph-level contrast learning, obtaining a trained citation network embedding vector of the subgraph-level contrast learning model; step S3: a pre-trained model identifies the structural similarity of nodes through node-level contrast learning, obtaining a trained citation network embedding vector of the node-level contrast learning model; step S4: using a classifier to classify citations into similar paper topics based on the trained citation network embedding vectors obtained in step S2 and step S3.

[0018] The data augmentation method includes: Step S31: A topological structure-level adaptive augmentation method for randomly removing edges from the graph according to an adaptively obtained probability, including: Step S311: Randomly obtaining a subset from the edge set E according to the edge-taking probability , and the edge-taking probability P is calculated by the following formula: ; where, represents the first sampling probability of removing the edge , is the connection set of the view obtained after the topological structure-level data augmentation method; Step S312: Set a metric function for node centrality , and define the centrality of an edge as the average of the centralities of the two nodes connected by the edge: ; where, is the average of the centralities of the two nodes connected by the edge; Step S313: Calculate the first sampling probability in the following manner ; ; ; wherein, represents the maximum value of represents the minimum value of is a hyperparameter that controls the overall probability of removing edges, represents the truncation probability.

[0019] Step S32: Node-attribute-level adaptive enhancement method for adding noise to node attributes by setting the values of some dimensions in the node features to zero, including: Step S321: Use a random variable for sampling, and the value of each dimension of the random variable is obtained from a Bernoulli distribution: , is the second sampling probability, represents the dimension; Step S322: For any discrete node , the discrete node feature of is; Calculate the first weight of dimension in the following manner: ; For any continuous node , the continuous node feature of the continuous node is , and calculate the second weight of dimension in the following manner: ; Step S323: Calculate the second sampling probability in the following manner: ; ; wherein, represents the maximum value of represents the average value of is a hyperparameter that controls the feature enhancement intensity, represents the truncation probability.

[0020] In the above steps, the node centrality function It is the degree centrality function, eigenvector centrality function, PageRank centrality function, or closeness centrality function.

[0021] Next, the data augmentation method will be further introduced in detail.

[0022] When performing graph data augmentation at the topological level, GPA adopts a very direct method to destroy the original graph, randomly deleting edges in the graph according to the adaptively obtained probability. That is, a subset is randomly obtained from the edge set E according to the probability , and the edge-taking probability is as follows: ; where represents the first sampling probability of removing the edge , is the connection set of the view obtained after the data augmentation method at the topological structure level. According to the discussion before this section, it can be concluded that the inherent meaning should be the importance of the edge . If an edge is more important, then its corresponding should be smaller, so that the augmentation function is more likely to destroy unimportant edges while retaining important topological structures in the original graph.

[0023] Next, this section will continue to introduce how the sampling probability used in graph data augmentation in GPA is determined. In the field of graph representation learning, node centrality is a widely used metric that can be used to quantify the influence of nodes in a graph. Similarly, edge centrality can also be defined through the centrality of nodes. Specifically, given a metric function of node centrality, the edge centrality can be defined as the average of the centralities of the two nodes it connects: .

[0024] After calculating the connection centrality, GPA calculates the probability of its occurrence according to the connection centrality. Since there may be an order-of-magnitude gap between the central values of nodes, for example, in a social network, some individuals may have thousands of followers, while ordinary people only have a few followers. Therefore, GPA uses the log function to reduce the influence of dense nodes, that is . Finally, GPA obtains through a normalization operation: ; where represents the maximum value of represents the minimum value of represents the hyperparameter that controls the overall probability of removing edges, Represents the truncation probability. To avoid excessive damage to the graph structure caused by too large a removal probability, GPA adopts a truncation probability less than 1, that is, the maximum probability of removing edges cannot exceed .

[0025] For the selection of the node centrality function, GPA uses the following four centrality measures, including degree centrality, eigenvector centrality, PageRank centrality, and closeness centrality.

[0026] The method of data augmentation for node attributes is similar to salt-and-pepper noise in digital image processing. GPA adds noise to node attributes by randomly masking the values of a part of the dimensions (setting to zero) in the node features. Specifically, GPA uses a random variable for sampling, and the values of each of its dimensions are independently obtained from the Bernoulli distribution, that is , and the feature matrix obtained through node attribute-level adaptive augmentation:[[]] ; where represents the concatenation operation, represents the multiplication operation.

[0027] Similar to the idea of topological-level augmentation The inherent meaning should be able to reflect the importance of the -th dimension of the node features. GPA assumes that the frequently occurring node feature dimensions are important and defines the weights of the feature dimensions as follows: for discrete one-hot node features, that is, for any node in the dimension (feature matrix) , its feature is . GPA calculates the importance of dimension as follows: ; where is the node centrality measure used to quantify the importance of nodes. Regarding the measurement method of node centrality, it has been introduced in the topological structure-level adaptive augmentation section of this section and will not be elaborated here. The first term represents the occurrence of dimension in node . One can imagine a citation network where each node represents an article, the edges represent citation relationships, and the features represent the keywords in the article. The keywords that appear in a highly cited article should be considered very important features.

[0028] In addition, for the continuous node features of node , it is impossible to directly calculate the occurrence times of each one-hot encoded value. At this time, GPA uses node ​ In the absolute value of the dimension to measure the importance of features: .

[0029] Similar to the adaptive enhancement at the topological structure level, GPA obtains the sampling probability by normalizing the calculated weights: ; ; where represents the maximum value of represents the average value of represents the hyperparameter that controls the feature enhancement intensity, represents the truncation probability.

[0030] Verify the effect of applying the present invention to the classification tasks in the three citation datasets of Cora, Pubmed, and DBLP. Among them, the Cora dataset is a citation network dataset composed of machine learning papers and is a dataset that has been very popular for graph deep learning in recent years. This dataset has a total of 2708 sample points and 5429 edges. Each sample point is a scientific paper, and all sample points are divided into 7 categories. Each paper is represented by a bag of words, and each word in the bag of words is a word that appears in that paper. Each sample point also has 1433 features, which are TF-IDF values in the bag-of-words model. The Pubmed dataset is a biomedical literature citation network dataset that contains 19,717 articles and 44,338 citation relationships. Each sample point is a scientific paper, and all sample points are divided into 3 categories. Each article is represented by a bag of words, and each word in the bag of words is a word that appears in that article. Each sample point also has 500 features, which are TF-IDF values in the bag-of-words model. DBLP is a large computer science literature index library that contains a large amount of information on papers and authors in the field of computer science. It provides a dataset containing all the bibliographic records that make up DBLP. The DBLP dataset contains multiple types of nodes, including authors, papers, terms, and conferences. DBLP contains 14,328 papers, 4057 authors, 20 conferences, and 7723 terms. The experimental process is to use the graph neural network learned by the framework of learning the structural similarity and neighborhood similarity of nodes in stages to embed the graph data to obtain the embedding vector. Then, the embedding vector is input into the classifier for classification. The verification process is as follows: Step 1: Dataset Partitioning. In this citation dataset, the graph is constructed from the citation links between scientific papers. Specifically, nodes correspond to articles, and undirected edges correspond to citation links between articles. Additionally, each node has an article type and a bag-of-words model, and the TF-IDF values in the bag-of-words model represent the features of the node. In the e-commerce dataset, nodes represent products, and edges represent that two products are often purchased together.

[0031] On these networks, this paper randomly selects 10% of the nodes as the training set, 10% of the nodes as the validation set, and the remaining nodes as the test set.

[0032] Step 2: Encoder Selection. For the transductive learning task, the encoder adopted in this section is a single-layer graph convolutional network model, and its propagation rule is , where is the adjacency matrix with self-loops (links between a node and itself) inserted, is its corresponding degree matrix, . For the non-linear activator , RELU is selected in this section. RELU is a parametrically learnable linear transformation, and it has learnable parameters. can be applied to each node.

[0033] Step 3: Classifier Selection. The research goal of this paper is to improve the quality of the embedding vectors obtained by unsupervised graph representation learning. In addition to exploring the embedding methods of the graph representation model, it is also necessary to select an appropriate node classification model to evaluate the quality of the obtained embedding vectors. Considering three points: the different scales of the datasets selected in the experiment, the connection between different attributes of the node embedding vectors, and the experiment focusing on improving the quality of the embedding vectors obtained by the graph representation learning model (rather than the effect of the classification model), Logistic regression method is finally selected as the model for performing the node classification task during the experiment. Example of the node classification task: An entire citation network consists of n articles, and these n articles belong to m different topics. Use the classifier to classify the embedding vectors obtained by the present invention to obtain the labels of the topics to which all articles belong.

[0034] Step 4: Use the accuracy evaluation metric to evaluate the model, that is: . The evaluation results are shown in the following table. It can be seen that the method of the present invention (GPA) has achieved a certain degree of improvement compared with other algorithms.

[0035]

Claims

1. An adaptive enhanced phased graph representation learning method based on citation information, including a learning framework that learns node structure similarity and node neighborhood similarity in phases, extracts general node structure similarity information from data in different domains through subgraph comparison, and then learns the neighborhood similarity of nodes in a preset domain through node-level comparison, characterized in that When using node - level contrastive learning to learn the neighborhood similarity of nodes, an adaptive data augmentation method is adopted, including: A topological - structure - level adaptive augmentation method, including calculating the centrality of edges based on a node centrality metric function, generating a first sampling probability through a truncation probability, using the first sampling probability as the edge deletion probability, and retaining the topological structure based on the edge deletion probability; A node - attribute - level adaptive augmentation method, including calculating the corresponding first weight and second weight based on discrete node features and continuous node features respectively; generating a second sampling probability through a truncation probability, using the second sampling probability as the feature masking probability, and retaining the feature dimension based on the feature masking probability.

2. The adaptive enhanced phased graph representation learning method based on citation information according to claim 1, wherein The learning framework for learning node - structure similarity and node - neighborhood similarity in stages extracts general node - structure similarity information from data in different domains through sub - graph contrast, and then learns the neighborhood similarity of nodes in a preset domain through node - level contrast, including: Step S1: Sampling a citation information sub - graph from the r - order neighborhood node set of the original graph; Step S2: The pre - trained model identifies the structural similarity of nodes through sub - graph contrastive learning, and obtains the trained citation network embedding vector of the sub - graph contrastive learning model; Step S3: The pre - trained model identifies the structural similarity of nodes through node - level contrastive learning, and obtains the trained citation network embedding vector of the node - level contrastive learning model; Step S4: Using a classifier to classify citations into similar paper topics according to the trained citation network embedding vectors in Step S2 and Step S3.

3. The adaptive enhancement phased graph representation learning method based on citation information according to claim 1, characterized in that The topological - structure - level adaptive augmentation method is used to randomly remove edges in the graph according to the adaptively obtained probability, including: Step S311: Randomly obtain a subset from the edge set E according to the edge selection probability , and the edge selection probability P is calculated by the following formula: ， Among them, represents the first sampling probability of removing an edge , and is the connection set of the view obtained after the topological structure-level data augmentation method; Step S312: Set the measurement function of node centrality , and define the centrality of an edge as the average of the centralities of the two nodes connected by the edge: ， Among them, is the average of the centralities of the two nodes connected by this edge; Step S313: Calculate the first sampling probability in the following manner , ， ， Among them, represents the maximum value of represents the minimum value of is a hyperparameter representing the overall probability of controlling the removal of edges, represents the truncation probability.

4. The adaptive enhancement phased graph representation learning method based on citation information as claimed in claim 1, wherein The node - attribute - level adaptive augmentation method is used to add noise to node attributes by setting the values of some dimensions to zero in node features, including: Step S321: Use a random variable for sampling. The value of each dimension of the random variable is obtained from the Bernoulli distribution: , is the second sampling probability, representing the dimension; Step S322: For any discrete node , the discrete node feature is , Calculate the dimension in the following manner for the first weight: , For any continuous node , the continuous node has continuous node characteristics of , and the second weight of dimension is calculated by the following method: ； Step S323: Calculate the second sampling probability in the following manner : ， ， Among them, represents the maximum value of represents the average value of is a hyperparameter representing the enhancement intensity of the control feature, represents the truncation probability.

5. The adaptive enhancement phased graph representation learning method based on citation information according to claim 3 or 4, characterized in that Node centrality function It is the degree centrality function, eigenvector centrality function, PageRank centrality function, or closeness centrality function.

Images