Instance learning-based graph domain adaptive node classification method

Abstract

The invention is applicable to the technical field of knowledge migration, and provides a graph domain adaptive node classification method based on instance learning, which comprises the following steps of: acquiring a source domain data set with a mark and a target domain data set without a mark; the representation of the source domain data set and the representation of the target domain data set are learned through a GNN model, modeling is conducted on the source domain data set and the target domain data set, and embedded vectors of the source domain data set and the target domain data set are obtained; the embedded vectors of the target domain data set are classified through KNN, and target domain category pseudo labels are obtained; performing fine-grained instance contrast learning on embedded vectors of the source domain graph data set and the target domain graph data set; and after classification is finished, displaying a final classification result of the target domain graph data set. According to the method, the defects of an existing graph domain adaptation technology in cross-domain tasks are overcome, and the method has wide applicability and can be applied to graph data analysis tasks in various cross-domain scenes, such as social network analysis, recommendation systems, knowledge graphs and biological networks.

Description

Technical Field

[0001] The present invention belongs to the technical field of knowledge transfer, and in particular relates to a graph domain adaptive node classification method based on instance learning. Background Art

[0002] Graph Domain Adaptation (GDA) is a cutting-edge research direction in the field of transfer learning and graph neural networks. It aims to address the limitations of traditional graph neural networks (GNNs) in cross-domain scenarios, especially the problem of how to effectively transfer knowledge when there are significant distribution differences between the source domain and the target domain. With the widespread application of graph structured data in social networks, knowledge graphs, biological networks and other fields, the importance of graph domain adaptation has become increasingly prominent.

[0003] With the rise of deep learning technology, graph neural networks (GNNs) have achieved remarkable results in tasks such as node classification and link prediction of graph structured data. However, traditional graph neural networks, such as graph convolutional networks (GCNs) and graph attention networks (GATs), have good performance on isomorphic graphs (i.e., the source domain and target domain have similar distributions), but in practical applications, data often comes from different distributions, i.e., there is a significant domain shift between the source domain and the target domain. The core idea of ​​graph domain adaptation technology is to reduce the differences between the source domain and the target domain in graph structure and node feature distribution, so that the model can effectively transfer the knowledge learned from the source domain to the target domain, thereby achieving good performance in tasks such as node classification in the target domain.

[0004] Early graph domain adaptation methods mainly borrowed adversarial learning methods from traditional domain adaptation, and introduced adversarial loss between the source domain and the target domain to make the graph encoder unable to distinguish the node representations of the source domain and the target domain, thus achieving distribution alignment. These methods demonstrated the potential of graph domain adaptation and promoted the rapid development of this field. However, graph domain adaptation technology still faces many challenges:

[0005] 1) Differences in graph structure between source and target domains: The graph topology structures of the source and target domains are often significantly different. For example, the source domain may be a social network with a high density of connections between nodes and an obvious community structure; while the target domain may be a knowledge graph with sparse connections between nodes and complex categories. This heterogeneity in graph structure makes it difficult for node representation learning methods to effectively migrate between the two domains. In graph domain adaptation, aligning the graph structures of the source and target domains is a core challenge. Simple adversarial training methods have difficulty capturing the complex local and global structural differences in graphs, so new graph structure alignment strategies are urgently needed to better achieve cross-domain migration.

[0006] 2) Inconsistent feature distribution: Even if the graph structures of the source domain and the target domain are similar, the difference in the distribution of node features will have a serious impact on the migration effect of the model. For example, in social networks, the characteristics of users may be composed of attributes such as interests and geographic location, while in biological networks, the characteristics of genes may be composed of their biological functions and molecular structures. The differences in these features lead to large errors in the target domain when directly using the feature representation of the source domain. Most of the current graph domain adaptation methods adopt a global alignment strategy, trying to minimize the overall distribution difference between the source domain and the target domain in the feature space. However, in actual applications, the local distribution differences are more significant, and more sophisticated alignment methods need to be introduced, especially for further exploration of feature alignment at the class level or instance level.

[0007] 3) Reliability of pseudo-label generation: When the target domain lacks labeled data, a common method is to generate pseudo-labels on the target domain through the model of the source domain and perform supervised learning on the target domain. However, the initial model has limited understanding of the target domain, and the generated pseudo-labels may contain a lot of noise. These inaccurate pseudo-labels will affect subsequent training and may even cause the model to fall into the wrong training path.

[0008] 4) Insufficient cross-domain alignment: Most existing graph-domain adaptation methods focus on global distribution alignment, that is, aligning the source domain and the target domain through adversarial loss or minimizing distribution differences. This ignores the differences between class level and instance level to a certain extent, resulting in insufficient alignment, which affects the generalization ability of the model in the target domain. To solve this problem, researchers have proposed class-level and instance-level alignment strategies in recent years.

[0009] 5) Computational complexity and efficiency: Graph structure data is often large-scale and highly complex. Especially in cross-domain scenarios, the complexity of the graph structure of the source and target domains increases the computational burden of model training. Existing graph domain adaptation methods usually rely on a large amount of computing resources, especially in the process of adversarial training and multi-round iterative pseudo-label generation, the computational overhead is huge. Therefore, how to reduce the computational complexity while ensuring performance is a key technical challenge.

[0010] In summary, graph domain adaptation technology faces problems such as insufficient generalization ability, reliance on a large amount of labeled data, insufficient cross-domain alignment, and low computational efficiency. To address these challenges, this paper proposes a graph domain adaptation node classification method based on instance learning. Summary of the invention

[0011] The purpose of the present invention is to provide a graph domain adaptive node classification method based on instance learning, aiming to solve the problems raised in the above background technology.

[0012] The purpose of the present invention is achieved through the following technical solutions:

[0013] A graph domain adaptive node classification method based on instance learning includes the following steps:

[0014] Step S1, obtaining a labeled source domain dataset and an unlabeled target domain dataset;

[0015] Step S2: learn the representation of the source domain dataset and the target domain dataset through the GNN model, model the source domain dataset and the target domain dataset respectively, and obtain the embedding vectors of the source domain dataset and the target domain dataset;

[0016] Step S3: classify the embedding vector of the target domain data set through KNN to obtain the target domain category pseudo label;

[0017] Step S4, performing fine-grained instance comparative learning on the embedding vectors of the source domain graph dataset and the target domain graph dataset;

[0018] Step S5: Classification is completed and the classification result of the final target domain graph dataset is displayed.

[0019] Furthermore, the specific process of step S2 is as follows:

[0020] Step S21, obtaining a source domain data set and a target domain data set, wherein the data set is first encoded by an encoder with local perturbation to obtain a local embedding vector of the data set;

[0021]

[0022] Where σ(·) is the activation function; express The diagonal matrix of ; represents the adjacency matrix enhanced by self-loop, A represents the adjacency matrix, I N represents the identity matrix; represents the local output of the i-1th layer, W (i) Represents the weight matrix of the i-th layer; local shift parameter Applied to each layer to fine-tune local embeddings;

[0023] Step S22: the data set is then encoded by an encoder with global perturbation to obtain a global embedding vector of the data set;

[0024] Formula 2:

[0025] Where σ(·) is the activation function; represents the diagonal matrix of P; P represents the forward point mutual information matrix; represents the global output of the i-1th layer, W(i) Represents the weight matrix of the i-th layer; global shift parameter Applied to each layer to fine-tune the global embedding;

[0026] Step S23: Adopt dynamic attention mechanism F att , dynamically assigning local embedding vectors H according to the relevance of weights l The weight α1 and the global embedding vector H g The weight of α2;

[0027] Formula 3: [α1, α2] = F att ([H l ,H g ])

[0028] The final embedding vector is constructed by weighting the local embedding vector and the global embedding vector according to the dynamically calculated weight coefficients α1 and α2;

[0029] Formula 4:

[0030] Where H represents the final embedding vector; α1 and α2 represent the contribution of the local embedding vector and the global embedding vector in the final embedding vector, respectively.

[0031] Furthermore, the specific process of step S3 is as follows:

[0032] Step S31: Use the KNN classification algorithm to retrieve the nearest neighbor node using the Euclidean distance between node embeddings. k,i To represent the target sample The first K nearest neighbors of , and then use the predicted probability average vector of the K nearest neighbors as the target sample The pseudo-prediction probability vector

[0033] Formula 5:

[0034] in, represents mathematical expectation; Represents the target sample The jth neighbor node of ; Represents the target sample The predicted probability vector of the jth neighbor node;

[0035] Step S32: Take the category with the highest probability in the pseudo prediction probability vector as the target sample Pseudo-labels

[0036] Formula 6:

[0037] Where k is the category index; Represents the target sample The probability value of belonging to category k.

[0038] Furthermore, the specific process of step S4 is as follows:

[0039] Step S41: target sample Features Defined as anchor points, features of samples of the same category in the source domain Considered as positive samples, features of samples of different categories in the source domain It is regarded as a negative sample; based on the InfoNCE loss, the inter-domain contrast loss is proposed

[0040] Formula 7:

[0041] in, Represents the source domain and the anchor point The set of positive samples with the same label, k represents An element in represents the true label of the source sample, Represents the target sample Pseudo labels of Represents the source domain and the anchor point A set of negative samples with different labels; represents transposition; τ is the temperature coefficient;

[0042] Step S42: source sample Features Defined as anchor points, features of samples of the same category in the target domain Considered as positive samples, the characteristics of samples of different categories in the target domain is considered as a negative sample; we get the inter-domain contrast loss

[0043]

[0044] in, Represents the target domain and the anchor point A set of positive samples with the same label, Represents the true label of the source sample; Represents the target domain and the anchor point A set of negative samples with different labels; τ is the temperature coefficient;

[0045] Step S43: and Combined, we get the final inter-domain instance comparison loss L Inter :

[0046] Formula 9:

[0047] Among them, N s and N t represent the number of samples in the source domain and the target domain respectively.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. Accuracy of cross-domain node classification: This paper significantly reduces the structural and feature differences between the source domain and the target domain by introducing multi-scale perturbation coding (local and global perturbations) and instance-level alignment strategies, thereby achieving more accurate cross-domain node classification. This approach improves the generalization ability of the model in the target domain, especially when processing heterogeneous graph data.

[0050] 2. Reliability of pseudo-label generation: The present invention adopts a pseudo-label generation strategy based on KNN, which effectively reduces the proportion of noise labels in the pseudo-label generation process. By dynamically screening highly reliable pseudo-labels, the present invention enables the model to gradually learn effective information in the target domain, significantly improving the reliability and stability of the classification results.

[0051] 3. Adaptability to graph structural heterogeneity: This invention uses a multi-scale perturbation mechanism to process graph structural information at different levels. This mechanism is highly robust to the topological differences between the source domain and the target domain, especially when facing complex heterogeneous graph structures.

[0052] 4. Model training and reasoning efficiency: This paper proposes an efficient graph structure alignment algorithm, which significantly reduces the computational complexity while ensuring classification accuracy. The algorithm is suitable for large-scale graph data processing, reduces computational overhead, and improves the model training and reasoning efficiency, so that it can be better applied to actual scenarios.

[0053] 5. Practicality and scalability of the model: The present invention solves the shortcomings of existing graph domain adaptation technology in cross-domain tasks by comprehensively adopting the above-mentioned innovative technologies. This innovative achievement has wide applicability and can be applied to graph data analysis tasks in various cross-domain scenarios, such as social network analysis, recommendation systems, knowledge graphs, and biological networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0055] In order to have a clearer understanding of the technical features, purposes and beneficial effects of the present invention, the technical solution of the present invention is now described in detail below, but it should not be construed as limiting the applicable scope of the present invention.

[0056] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0057] An embodiment of the present invention provides a graph domain adaptive node classification method based on instance learning, and its flow chart is as follows: Figure 1 As shown, the method comprises the following steps:

[0058] Step S1, obtaining a labeled source domain dataset and an unlabeled target domain dataset;

[0059] Step S2: learn the representation of the source domain dataset and the target domain dataset through the GNN model, model the source domain dataset and the target domain dataset respectively, and obtain the embedding vectors of the source domain dataset and the target domain dataset;

[0060] Step S3: classify the embedding vector of the target domain data set through KNN to obtain the target domain category pseudo label;

[0061] Step S4, performing fine-grained instance comparative learning on the embedding vectors of the source domain graph dataset and the target domain graph dataset;

[0062] Step S5: Classification is completed and the classification result of the final target domain graph dataset is displayed.

[0063] Specifically include:

[0064] 1) The specific process of step S2 (corresponding to the node encoding module) is as follows:

[0065] Step S21, obtaining a source domain data set and a target domain data set, wherein the data set is first encoded by an encoder with local perturbation to obtain a local embedding vector of the data set;

[0066] Formula 1:

[0067] Where σ(·) is the activation function; express The diagonal matrix of ; represents the adjacency matrix enhanced by self-loop, A represents the adjacency matrix, I N represents the identity matrix; represents the local output of the i-1th layer, W (i) Represents the weight matrix of the i-th layer; local shift parameter is strategically applied at each layer to fine-tune the local embedding.

[0068] Step S22: the data set is then encoded by an encoder with global perturbation to obtain a global embedding vector of the data set;

[0069] Formula 2:

[0070] Where σ(·) is the activation function; represents the diagonal matrix of P; P represents the positive pointwise mutual information (PPMI) matrix; represents the global output of the i-1th layer, W (i) Represents the weight matrix of the i-th layer; global shift parameter is strategically applied at each layer to fine-tune the global embedding.

[0071] Step S23: In order to seamlessly integrate the adjusted local embedding vector and the global embedding vector into a unified embedding, we adopt a dynamic attention mechanism F att , which dynamically assigns weights α1 and α2 according to the relevance of the weights;

[0072] Formula 3: [α1, α2] = F att ([H l ,H g ])

[0073] The final embedding vector is constructed by weighting the local embedding vector and the global embedding vector according to the dynamically calculated weight coefficients α1 and α2;

[0074] Formula 4:

[0075] Where H represents the final embedding vector; α1 and α2 represent the contribution of the local embedding vector and the global embedding vector in the final embedding vector, respectively.

[0076] 2) The specific process of step S3 (corresponding to the pseudo-label modeling method) is as follows:

[0077] Step S31: Use the K-Nearest Neighbor (KNN) classification algorithm to retrieve the nearest neighbor node using the Euclidean distance between node embeddings. k,i To represent the target sample The first K nearest neighbors of , and then directly use the predicted probability average vector of these K nearest neighbors as the target sample The pseudo-prediction probability vector

[0078] Formula 5:

[0079] in, represents mathematical expectation; Represents the target sample The jth neighbor of Represents the target sample The predicted probability vector of the jth neighbor of .

[0080] Step S32: Take the category with the highest probability in the pseudo prediction probability vector as the target sample Pseudo-labels

[0081] Formula 6:

[0082] Where k is the category index; Represents the target sample The probability value of belonging to category k.

[0083] 3) The specific process of step S4 (corresponding to the instance-level cross-domain alignment strategy) is as follows:

[0084] Step S41: target sample Features Defined as anchor points, features of samples of the same category in the source domain are regarded as positive samples, while the characteristics of samples of different categories in the source domain It is regarded as a negative sample. Based on the InfoNCE loss, the inter-domain contrast loss is proposed.

[0085] Formula 7:

[0086] in, Represents the source domain and the anchor point The set of positive samples with the same label, k represents An element in represents the true label of the source sample, Represents the target sample Pseudo labels of Represents the source domain and the anchor point A set of negative samples with different labels; represents transposition; the temperature coefficient τ is empirically set to 0.5.

[0087] Step S42: Similarly, the source sample Features Defined as anchor points, features of samples of the same category in the target domain are regarded as positive samples, and the characteristics of samples of different categories in the target domain is regarded as a negative sample. Therefore, we get the inter-domain contrast loss

[0088] Formula 8:

[0089] in, Represents the target domain and the anchor point A set of positive samples with the same label, Represents the true label of the source sample; Represents the target domain and the anchor point A set of negative samples with different labels; the temperature coefficient τ is 0.5.

[0090] Step S43: and Combined, we get the final inter-domain instance contrast loss L Inter :

[0091] Formula 9:

[0092] Among them, N s and N t represent the number of samples in the source domain and the target domain respectively.

[0093] Embodiment 1, reference network graph domain adaptation node classification task example;

[0094] 1) Experimental purpose: To evaluate the performance improvement of the proposed method in the graph domain adaptive node classification task.

[0095] 2) Datasets: The experiments used three real-world cross-network homogeneity datasets provided by ArnetMiner: ACMv9 (A), Citationv1 (C), and DBLPv7 (D). We selected one of the datasets as the source domain each time and the other two datasets as the target domains, and performed a total of six graph-domain adaptation tasks in these citation networks.

[0096] 3) Comparison models, including two categories:

[0097] a. Traditional single-domain node classification model: graph convolutional network (GCN), graph attention network (GAT).

[0098] b. Node classification models for graph domain adaptation: Unsupervised Domain Adaptation Graph Convolutional Network (UDAGCN), Adversarial Cross-Network Deep Network Embedding (ACDNE), Adversarial Domain Adaptation Graph Convolutional Network (AdaGCN), Semi-supervised Graph Domain Adaptation Network (SGDA).

[0099] 4) Evaluation indicators: Classification accuracy is used as the main evaluation indicator.

[0100] The experimental results are summarized in Table 1, showing the classification accuracy of each model on different GDA tasks:

[0101] Table 1 Comparison of classification effects

[0102]

[0103]

[0104] 5) Result analysis:

[0105] The experimental results in Table 1 show that the method of the present invention achieves more effective graph structure and feature alignment between the source domain and the target domain, and significantly improves the accuracy of cross-domain node classification. In the case of an unlabeled target domain, the present invention reduces label noise and improves the generalization ability of the model through a pseudo-label generation strategy. In particular, compared with existing graph domain adaptation methods, the classification accuracy of the present invention on various graph domain adaptation tasks has been significantly improved, and its average classification accuracy has reached 76.83%, which is a clear advantage over other methods.

[0106] These results not only verify the effectiveness and robustness of the proposed method in dealing with cross-domain node classification tasks, but also demonstrate its excellent adaptability and performance improvement when dealing with unlabeled target domains and complex cross-domain structures.

[0107] The above are only preferred embodiments of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These should also be regarded as the protection scope of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A graph domain adaptive node classification method based on instance learning, characterized in that: The following steps are involved: Step S1, obtaining a labeled source domain dataset and an unlabeled target domain dataset; Step S2: learn the representation of the source domain dataset and the target domain dataset through the GNN model, model the source domain dataset and the target domain dataset respectively, and obtain the embedding vectors of the source domain dataset and the target domain dataset; Step S3: classify the embedding vector of the target domain data set through KNN to obtain the target domain category pseudo label; Step S4, performing fine-grained instance comparative learning on the embedding vectors of the source domain graph dataset and the target domain graph dataset; Step S5: Classification is completed and the classification result of the final target domain graph dataset is displayed.

2. The graph domain adaptive node classification method based on instance learning according to claim 1 is characterized in that: The specific process of step S2 is as follows: Step S21, obtaining a source domain data set and a target domain data set, wherein the data set is first encoded by an encoder with local perturbation to obtain a local embedding vector of the data set; Formula 1: Where σ(·) is the activation function; express The diagonal matrix of ; represents the adjacency matrix enhanced by self-loop, A represents the adjacency matrix, I N represents the identity matrix; represents the local output of the i-1th layer, W (i) Represents the weight matrix of the i-th layer; local shift parameter Applied to each layer to fine-tune local embeddings; Step S22: the data set is then encoded by an encoder with global perturbation to obtain a global embedding vector of the data set; Formula 2: Where σ(·) is the activation function; represents the diagonal matrix of P; P represents the forward point mutual information matrix; represents the global output of the i-1th layer, W (i) Represents the weight matrix of the i-th layer; global shift parameter Applied to each layer to fine-tune the global embedding; Step S23: Adopt dynamic attention mechanism F att , dynamically assigning local embedding vectors H according to the relevance of weights l The weight α1 and the global embedding vector H g The weight of α2; Formula 3: [α1, α2] = F att ([H l ,H g ]) The final embedding vector is constructed by weighting the local embedding vector and the global embedding vector according to the dynamically calculated weight coefficients α1 and α2; Formula 4: Where H represents the final embedding vector; α1 and α2 represent the contribution of the local embedding vector and the global embedding vector in the final embedding vector, respectively.

3. The graph domain adaptive node classification method based on instance learning according to claim 2 is characterized in that: The specific process of step S3 is as follows: Step S31: Use the KNN classification algorithm to retrieve the nearest neighbor node using the Euclidean distance between node embeddings. k,i To represent the target sample The first K nearest neighbors of , and then use the predicted probability average vector of the K nearest neighbors as the target sample The pseudo-prediction probability vector Formula 5: in, represents mathematical expectation; Represents the target sample The jth neighbor of Represents the target sample The predicted probability vector of the jth neighbor node; Step S32: Take the category with the highest probability in the pseudo prediction probability vector as the target sample Pseudo-labels Formula 6: Where k is the category index; Represents the target sample The probability value of belonging to category k.

4. The graph domain adaptive node classification method based on instance learning according to claim 3 is characterized in that: The specific process of step S4 is as follows: Step S41: target sample Features Defined as anchor points, features of samples of the same category in the source domain Considered as positive samples, features of samples of different categories in the source domain It is regarded as a negative sample; based on the InfoNCE loss, the inter-domain contrast loss is proposed Formula 7: in, Represents the source domain and the anchor point The set of positive samples with the same label, k represents An element in represents the true label of the source sample, Represents the target sample Pseudo labels of Represents the source domain and the anchor point A set of negative samples with different labels; represents transposition; τ is the temperature coefficient; Step S42: source sample The characteristic f i s Defined as anchor points, features of samples of the same category in the target domain Considered as positive samples, the characteristics of samples of different categories in the target domain is considered as a negative sample; we get the inter-domain contrast loss Formula 8: in, Represents the target domain and the anchor point A set of positive samples with the same label, represents the true label of the source sample; Represents the target domain and the anchor point A set of negative samples with different labels; τ is the temperature coefficient; Step S43: and Combined, we get the final inter-domain instance comparison loss L Inter : Formula 9: Among them, N s and N t represent the number of samples in the source domain and the target domain respectively.

Images