Environment-adaptive geographic network invariant representation learning model and method
By using an environment-adaptive invariant representation learning model, the problem of generalization ability of geographic network models under environmental changes is solved. Through environment identification and representation separation techniques, combined with the MoE predictor, stable prediction of the model under different environments is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-23
- Publication Date
- 2026-03-27
AI Technical Summary
Existing graph neural network models lack adaptability and generalization ability when facing changes in the geographic network environment, resulting in unstable model performance, especially with decreased generalization ability when dealing with OOD data.
An environment-adaptive invariant representation learning model is proposed. It identifies the structure and features of node neighbors through an environment recognizer, separates the invariant representation from the background representation, and uses a MoE predictor to dynamically adjust the prediction strategy, combined with the environmental variable E for adaptive prediction.
It improves the model's generalization performance in diverse environments, enabling it to better adapt to environmental changes and maintain prediction consistency and stability under different environments.
Smart Images
Figure CN121743956A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the field of electronic information technology, in particular to an environment adaptive geographic network invariant representation learning model and method. BACKGROUND
[0002] Graph Neural Networks (GNN) have become the mainstream method for geographic network learning due to their superior ability in processing graph data. By capturing the topological relationships and attribute information between nodes (entities), geographic network representation is obtained, which is widely used in traffic flow prediction, urban planning, environmental monitoring, and other fields. Geographic network data has complex elements such as spatial entities, entity relationships, and entity features. This complexity leads to the basic assumption of graph neural network methods that training and testing data come from the same distribution (IID), which is invalid in reality, leading to more serious out-of-distribution (OOD) generalization problems. Therefore, there are many challenges in modeling and analysis.
[0003] Due to the influence of environmental changes, social and economic development, or sudden events, the structure and node features of geographic networks will change significantly, leading to unstable or ineffective performance of the model when applied. Existing methods usually cannot effectively deal with this problem during training, so the model lacks adaptability and generalization when facing OOD data. SUMMARY
[0004] The present application proposes an environment adaptive invariant representation learning model to address the problems in the background art, aiming to reduce the negative impact of environmental changes on the generalization ability of geographic network representation learning models.
[0005] Technical solution:
[0006] The present application first proposes an environment adaptive invariant representation learning model, which consists of three modules.
[0007] Firstly, considering the significant impact of environmental variables on representation learning in graph data such as geographic networks, the present application designs an environment recognizer. This module learns a latent environment vector E representing the discrete distribution of the environment each node is in by analyzing the neighbor structure and feature information of the target node. Then, a representation distinguishing module is proposed, which uses the generated environment variable E to decompose the node representation into invariant representation C and background representation S, and enhances the attention to C, reducing the interference of S. Finally, an adaptive hybrid predictor is adopted, based on the Mixture of Experts (MoE) architecture, combining C, S, and E to achieve environment adaptive prediction. This module models different environmental patterns through multiple GNN branches / experts, and dynamically weights the expert output using E, so that the model can flexibly adapt to specific environments and improve the generalization performance in diverse environments.
[0008] (a) Explanation of symbols
[0009] The graph dataset is G = (V, E, X), where V = {v1, v2...v...} N} represents a node in the graph, with the number of nodes being N = |V|, and E represents an edge in the graph, with node v i The neighbors are represented as N(v i Each node v i Each has a corresponding label y i Its sample space is ΩY. The representation space of node V is Z∈R. N*d d represents the dimension of the representation. The representation consists of two types: Z = (C, S), where C represents the invariant representation and S represents the background representation.
[0010] (ii) Environment Variable Identification Module (Environment Detector, ED)
[0011] Changes in environmental variables (such as alterations in graph structure or feature distribution) can reduce the generalization ability of a model, making them an unavoidable factor in graph representation learning. However, environmental variables are often not known information, making the effective identification of these latent variables a crucial issue. Referring to current research (such as EERM and LiSA), modeling different environments by examining changes in neighbor structure (such as degree distribution and connectivity patterns) has proven to be an effective means of capturing the environment. In graph neural networks, node representations are typically generated through neighbor aggregation; therefore, neighbor structure and features are naturally important carriers of environmental information. For example, in a geographic network, a graph is constructed using POI nodes that people originate from and arrive at. The flow of edges and the types of neighboring nodes reflect environmental differences: during the week, the neighbors of shopping mall POI nodes are mostly workplace POI nodes, with lower node degrees (lower foot traffic); on weekends, the neighbors are mostly university town POI nodes, with higher node degrees (higher foot traffic). By observing changes in neighboring nodes (such as degree and feature distribution), the environment of the target node (such as weekday or weekend) can be inferred.
[0012] In summary, this invention makes the following assumptions about environmental variables:
[0013] Hypothesis 1: Neighbor structure and neighbor characteristic information are explicit elements representing the environment.
[0014] Therefore, this invention designs an environmental variable identifier based on this assumption. The goal is to capture the environment vector corresponding to each node. ,in The node corresponding to the environment variable. The first-order neighborhood, It is the first encoding aggregation representation layer, The number of settings of the environment vector, that is, the environment vector of each node is a dimensional vector. Specifically, is modeled as a discrete vector sampled from a "categorical distribution" . Where is the probability distribution of the environment category, which is expressed by the formula:
[0015]
[0016] where, is the hidden representation of the neighbor node after convolution of the layer, is a parameter obtained by training, is the hidden dimension of
[0017] However, directly sampling from will cause the gradient to be non-differentiable, so the Gumbel-Softmax technique is used to approximate the differentiability, and the formula is:
[0018]
[0019] where, is a temperature (temperature) hyperparameter that controls the approximation degree of the generated result to the discrete distribution. When is small, the result tends to be a discrete one-hot vector, when it is large, the result is smoother.
[0020] (Three) representation separation module (Representation Separator, RS)
[0021] Traditional graph neural networks usually learn a single representation, which mixes causal features and environment-related pseudo-related information, resulting in decreased generalization ability when the environment changes. To this end, the invention proposes to distinguish between invariant representation C and background representation S. By enhancing attention to C and reducing the interference of S, the model can focus more on the causal relationship stable with the label Y, thereby improving the cross-environment prediction ability.
[0022] Therefore, two networks and are constructed to learn invariant representation C and background representation S based on the encoded representation :
[0023]
[0024]
[0025] wherein, , .
[0026] Further, observing the invariant representation and background representation, we find that:
[0027]
[0028]
[0029] Therefore, the present application summarizes that:
[0030] a) Regardless of the similarity and difference of the environment, the similarity of the invariant representation C of the same category is high, and the similarity of the invariant representation C of different categories is low;
[0031] b) Regardless of the similarity and difference of the category, the similarity of the background representation S of the same environment is high, and the similarity of the background representation S of different environments is low;
[0032] Based on this finding, the present application constructs invariant representation constraints and background representation constraints to force and The network can learn causal invariant information and environment-related information.
[0033] (Four) MoE predictor module (MoE Predictor)
[0034] After obtaining the invariant representation C and the background representation S through the representation distinguishing module, they are weighted and spliced to input the predictor , so that the model pays more attention to the invariant representation C. However, in order to adapt to different environments, it is insufficient to rely only on a fixed predictor to cope with environmental changes. At the same time, the traditional predictor (such as a single fully connected layer) is difficult to adapt to environmental changes under fixed parameters, especially in the unseen environment (OOD), the generalization ability is limited. Therefore, a predictor based on expert mixing (MoE) is proposed, combined with the environment vector E captured by the environment variable identification module (ED) to dynamically adjust the prediction strategy. MoE uses K parallel GNN branches as "experts", and each branch models potential environmental patterns, and the pseudo-environment vector of each node acts as a "router" to dynamically select or weight the expert output. For this purpose, two MoE architectures are adopted to adapt to different graph characteristics.
[0035] GCN-based MoE architecture: suitable for regular structure graphs by uniformly weighting and aggregating neighbor features. For the l-th layer node u, its representation vector is given by:
[0036]
[0037] here, It is the degree of node u. and These are the trainable weights of the k-th branch. Yes, it is an activation function.
[0038] MoE architecture based on GAT: Introduces an attention mechanism, suitable for capturing graphs with high heterogeneity among neighbors. Layer l representation. Updated to
[0039]
[0040]
[0041] here, The attention weights for node u and its neighbor v are obtained by performing a linear transformation on the node features and then applying LeakyReLU and Softmax. Finally, after L layers of feature propagation, each node u obtains a representation. Then it is mapped to the final prediction through a fully connected layer. .
[0042] (v) Regularization Optimization Module
[0043] To constrain the learning method of variable distribution and prevent overfitting or learning unreasonable variable distribution, three regularizations were designed based on supervised loss to constrain the learned invariant representation, background representation, and environmental variables, respectively.
[0044] Regularization of the environment distribution: To ensure the robustness and diversity of the pseudo-environment distribution e generated by the environment variable recognizer, and to encourage e to approximate the predicted distribution of the environment encoder while maintaining the smoothness of the distribution, thereby enhancing the model's adaptability to unseen environments, a regularization constraint term is introduced:
[0045]
[0046] in, It is the number of nodes. This represents the number of pseudo-environment categories. It is the cross-entropy loss, which optimizes the distribution of e to approximate the output of the environmental encoder by minimizing the cross-entropy loss. This ensures that e reflects environmental information in the characteristics of its neighbors, thus preventing e from degenerating into an overly sharp distribution; Entropy regularization maximizes the entropy of e, encourages smooth distribution, and enhances robustness and diversity. This is a hyperparameter that controls the strength of entropy regularization.
[0047] Invariant Representation Constraint: To ensure that the invariant representation C maintains consistent predictive power for label Y under different environments E, this invention employs an invariant risk minimization (IRM) constraint. Input feature X, invariant representation through... Extract, of which It is a feature extractor. It is a linear mapping. The objective of IRM constraints is to optimize the uniformity of environment-specific risks, as shown in the formula:
[0048]
[0049] in, It is a set of pseudo-environments. It is the number of pseudo-environments. It is a pseudo-environment The risk function is as follows.
[0050] To ensure To ensure consistent risk across different environments and prevent model overfitting to specific environments, a gradient penalty term is introduced.
[0051]
[0052] Background Representation Constraints: To bring S closer to environmental information, this invention predicts E by supervising S:
[0053]
[0054] here It is a node The predicted probability of belonging to pseudo-environment category k.
[0055] Then, cross-entropy loss is used to encourage... Get as close to E as possible:
[0056]
[0057] in, It is the training set sample index. It is a pseudo-environmental distribution obtained by assuming that "neighbor node information and structure reflect potential environmental factors".
[0058] This invention also proposes an environment-adaptive geographic network invariant representation learning method, which is implemented using the environment-adaptive geographic network invariant representation learning model proposed in this invention.
[0059] a. Data Acquisition: In order to reflect the environmental adaptability of the SEARL model, it is necessary to collect data that reflects the structure and feature distribution of the geographic network. First, capture geographic entities, relationships and features from the geographic network, G = (V, E, X). Second, set the environmental offset construction time or subgraph offset. Finally, perform data preprocessing to convert the collected data into the graph structure format required by the model.
[0060] b. Model application: The implementation requires three main steps. Specifically, the model generates latent environmental variables E through an environment estimator, separates the invariant representation C and the background representation S, and uses the MoE predictor to achieve environmental adaptation.
[0061] c. Results evaluation and output: The results are evaluated and the indicators are output using accuracy (acc) and macro-F1 score.
[0062] Beneficial effects of the present invention
[0063] This invention explores the reasons for the reduced generalization ability of existing models when dealing with geographic network distribution shifts from the perspective of environmental variables. Analysis reveals that this is mainly due to spurious correlations between background representations and labels introduced by environmental variables, leading to a decline in model prediction performance after environmental changes. To address this issue, an adaptive environment-aware representation learning model for geographic networks (SEARL) is proposed. This model consists of three modules, with the main objective of learning invariant representations that are stable with respect to labels and suppressing background representations that are sensitive to label relationships when the environment changes. Extensive experiments were conducted based on the environment-adaptive SEARL model, including geographic network and social network scenarios, to reveal and analyze the model's performance in different scenarios. Through comparative studies with other methods, this invention finds that SEARL achieves satisfactory results in terms of generalization. Attached Figure Description
[0064] Figure 1 This is a SEARL framework diagram of the present invention. Detailed Implementation
[0065] The present invention will be further described below with reference to embodiments, but the scope of protection of the present invention is not limited thereto:
[0066] Combination Figure 1 An environment-adaptive geographic network invariant representation learning model, comprising:
[0067] Environmental variable identification module: includes an environmental estimator that captures the potential environmental variables E of different sampling points through the neighbor structure and neighbor feature information of the target node;
[0068] The differentiation module utilizes environmental variable E to assist two branch networks, extracting invariant representation C and background representation S respectively. Invariant representation C is constrained to have a stable causal relationship with label Y, maintaining prediction consistency under different environments. Background representation S is designed to be sensitive to environmental changes and has a spurious correlation with label.
[0069] MoE Predictor Module: Combines latent environmental variable E, invariant representation C, and background representation S to dynamically select a suitable expert combination.
[0070] The specific architecture of the model is as described in the invention description. The experimental verification of the model is as follows:
[0071] (I) Introduction to the dataset
[0072] To verify the experimental results, this invention selected social network datasets of different sizes, including Cora, PubMed, Citeseer, Twitch, and Arxiv, as well as the geographic network dataset Education. Furthermore, to evaluate SEARL's capabilities in OOD scenarios, this invention processed the geographic network dataset and referenced the design of social network datasets used in current research on potential environmental impacts.
[0073] ① Geographic network dataset:
[0074] Education: This dataset uses a 2021 node-based classification dataset of education levels at the county level in the United States. Using US counties as nodes, the dataset obtains full-year statistical data for 2021 from the US Census American Community Survey (ACS) as attribute information, including statistics on education, economy, etc. The final dataset contains 809 nodes, 593,996 edges, 218-dimensional features, and 4 categories of education levels.
[0075] ② Social network datasets:
[0076] Cora: The Cora dataset is a commonly used dataset for citation network classification tasks in the field of graph data. It contains an academic paper citation network covering papers in the field of machine learning, with a total of 2708 papers, 5429 edges, 1433 feature dimensions, and covering 7 categories.
[0077] Citeseer: The Citeseer dataset is a widely used dataset for academic citation network classification tasks. This dataset contains citation networks for papers in the field of computer science, with 3312 papers, 4723 edges, and 3703 feature dimensions, comprising 6 categories.
[0078] PubMed: The PubMed dataset is a commonly used dataset for document classification tasks, particularly in the biomedical field. This dataset contains 19,717 abstracts from the PubMed database, with 44,338 edges, 500 feature dimensions, and covers three categories.
[0079] Twitch is a dataset of game players. Nodes represent players with game-related characteristics, and edges represent friendship relationships between players.
[0080] Arxiv: Adapted from OGB's arXiv citation network for computer science papers. In this directed graph dataset, nodes represent computer science (CS) papers on arXiv, and directed edges represent citation relationships between papers. Its task is to classify the subject areas of CS papers, a classification problem involving 40 categories.
[0081] Table 1 shows the dataset used in the experiment.
[0082] Table 1. Introduction to the datasets used in the experiment
[0083]
[0084] Specifically, the above dataset was processed as follows:
[0085] For the Education dataset, the data segmentation is based on the geographical differences of the counties: counties located in the eastern region are designated as ID data, while counties located in the central and western regions are designated as OOD1 and OOD2 data, respectively. Here, the eastern, central, and western regions are defined according to the geographic regions defined by the U.S. Census Bureau. After segmentation, there are 324 ID nodes, 345 OOD1 nodes, and 140 OOD2 nodes. The training and test sets for the ID nodes are divided in a 7:3 ratio, while the remaining regional categories use all data as the test set.
[0086] For social network datasets, refer to the settings of existing studies: (1) Cora, Citeseer and Pubmed: retain the original node labels, and artificially synthesize artifact node features to introduce distribution offset, thereby distinguishing ID (in-distribution) and OOD (out-of-distribution) data; (2) Arxiv: use the publication year of the paper to divide the data. Papers published from 2005 to 2014 are used as ID data, and papers published after 2014 are used as OOD data: papers published from 2014 to 2016 are used as OOD1, 2016 to 2018 are used as OOD2, and 2018 to 2020 are used as OOD3; (3) Twitch: use subgraph-level data division. The nodes of subgraphs DE, PT and RU are used as ID data, while the nodes of ES, FR and EN are used as OOD data: ES, FR and EN are used as OOD1, OOD2 and OOD3 respectively.
[0087] (II) Comparison Method
[0088] IRM, DANN, CORAL, ERM, EERM, and the latest CANET model were selected as baselines for comparison. IRM is a stable learning method for handling domain-specific data; DANN and Deep CORAL employ adversarial training and statistical alignment methods, respectively; ERM is a classic empirical risk minimization method, serving as the most basic training paradigm, directly optimizing model performance on the observed data distribution, while EERM improves generalization ability by exploring environment invariance; CANET is a state-of-the-art graph neural network model that combines causal inference and environment-invariant learning to address the distribution shift problem. These models represent different mechanisms for graph neural networks to process graph data, thus providing a solid performance benchmark for the research of this invention. By using these models as baselines, the performance of the method of this invention can be comprehensively evaluated and demonstrated. The specific models are described below:
[0089] ① IRM:
[0090] This is an invariant learning method designed to achieve robust generalization across environments. The method learns environment-independent characteristics by finding data representations or features that ensure the best classifier trained on these representations can adapt to all environments.
[0091] ② DANN:
[0092] By introducing an additional domain classifier, the model can simultaneously reduce the distributional differences between the source and target domains when learning feature representations. This process can be viewed as a form of adversarial training.
[0093] ③ CORAL:
[0094] The impact of domain shift can be mitigated by minimizing the distance between the second-order statistics of the source domain features and the target domain features.
[0095] ④ ERM:
[0096] Empirical Risk Minimization (ERM) learns model parameters by minimizing the average loss on the training data, without considering the distributional bias of the data. Its core idea is to optimize model performance under the observed distribution, but this may lead to insufficient generalization ability on unseen distributions.
[0097] ⑤ EERM:
[0098] Explore extrapolation risk minimization (EERM), which maximizes the risk variance of multiple virtual environments by introducing an adversarial training context explorer (graph structure editor), enabling graph neural networks to achieve OOD generalization in a single observation environment.
[0099] ⑥ CANET:
[0100] CANET reveals the core reason for the failure of GNNs in OOD generalization from a causal analysis perspective—potential environmental confounding bias—and proposes a robust training method that does not require environmental labels. This method counteracts confounding bias through collaborative learning between the environment estimator and the expert hybrid GNN predictor, thereby improving the model's generalization ability under node-level distribution shifts and significantly outperforming existing methods in multiple benchmark tests.
[0101] (III) Evaluation Indicators
[0102] This invention primarily employs two common evaluation metrics to assess the performance of geographic network representation learning models: accuracy (acc) and macro-F1 score. These two metrics comprehensively reflect the model's performance across different tasks, especially when dealing with geographic network data exhibiting class imbalance, providing a deeper understanding of the model's performance. The following provides a detailed introduction to these two evaluation metrics.
[0103] (1) Accuracy
[0104] Accuracy is one of the most fundamental and commonly used evaluation metrics in classification tasks, reflecting the overall correctness of the model's predictions. Specifically, accuracy calculates the proportion of correctly predicted samples out of the total number of samples. For multi-class classification tasks, the formula for accuracy is as follows:
[0105]
[0106] In geographic network representation learning tasks, accuracy is a direct metric for measuring the overall classification performance of a model. While accuracy is easy to understand and calculate, relying solely on it can be misleading in cases of class imbalance. This is because the model may be biased towards predicting the class with the larger sample size, resulting in higher accuracy, but without any actual improvement in predictive ability. Therefore, in practical applications, it is necessary to combine other metrics for comprehensive evaluation.
[0107] (2) Macro F1 value
[0108] In situations where class imbalance exists, the F1 score, as a comprehensive indicator that considers both precision and recall, can better measure model performance. The macro F1 score (Macro-F1) calculates the F1 score for each class and then averages it, thus avoiding the interference of class imbalance on the evaluation results.
[0109] The F1 score is the harmonic mean of precision and recall, and its formula is:
[0110]
[0111] Precision calculates the proportion of samples that the model predicts as positive, but which are actually positive; Recall calculates the proportion of all samples that are actually positive, but which are correctly predicted as positive by the model. The steps for calculating the macro F1 score are as follows: a. Calculate the F1 score for each class; b. Calculate the average of the F1 scores for all classes.
[0112] The formula for the macro F1 value is:
[0113]
[0114] Where C is the total number of categories, It is the F1 value of the i-th class.
[0115] The macro F1 score effectively overcomes class imbalance because it assigns equal weight to the performance of each class, regardless of the number of samples in each class. Therefore, the macro F1 score considers not only the accuracy of the prediction but also the model's performance on a minority of classes, making it particularly important for geographic network data with uneven class distribution.
[0116] By employing both metrics simultaneously, this invention can comprehensively evaluate the predictive ability of geographic network representation learning models in different environments. In particular, when facing practical problems such as class imbalance and distribution shift, it can more objectively reflect the generalization ability of the model.
[0117] (iv) Model parameter settings
[0118] During model training, the number of epochs in the social network dataset was set to 500, and the epoch with the best performance in the validation set was used for testing. Adam optimization was used to update the weights of the neural network. For model hyperparameters, a grid search was performed on each dataset on the validation set. The search space for all hyperparameters is as follows:
[0119] Number of GNN layers L: [2,3,4,5];
[0120] Hidden dimension H: [32, 64, 128];
[0121] Dropout ratio: [0.0, 0.1, 0.2, 0.5];
[0122] Learning rate: [0.001, 0.005, 0.01, 0.02];
[0123] Weight decay: [0, 5e-5, 5e-4, 5e-3];
[0124] The number of pseudo-environments, K: [3,5,10];
[0125] Gumbel-Softmax temperature parameter φ: [1, 2, 3, 5, 10].
[0126] For all datasets, OOD settings were implemented to ensure that the validation and training sets did not follow the same distribution. Finally, the mean and standard deviation of the three repeated experiments were calculated.
[0127] (V) Experimental Environment Setup
[0128] The experiments were run on Python 3.7, PyTorch 1.13.0, and PyTorch Geometric 2.1.0. Two deep learning stations were used to run all experiments: a DGX A100 with 4×80G and an A6000 with 3×48G.
[0129] (1) Comparative Experiment and Analysis of Geographic Network Datasets
[0130] We evaluated the GCN and GAT backbone networks on the Education dataset, combining CANET and SEARL methods in ID (Eastern Region), OOD1 (Central Region), and OOD2 (Western Region) environments. Experimental results show that CANET significantly outperforms SEARL in all environments, especially on the GAT backbone. Specifically, GAT-CANET achieves 53.29% Accuracy and 48.12% Macro-F1 in the ID environment, 54.59% and 53.19% in the OOD1 environment, and 44.88% and 42.87% in the OOD2 environment. In contrast, GAT-SEARL's Accuracy and Macro-F1 are only 41.36% and 26.03% in the ID environment, and further decline in the OOD environment. This result demonstrates that CANET's design better captures invariant features, thus exhibiting stronger generalization ability in out-of-distribution environments.
[0131] Referring to Table 2, from the perspective of the backbone network, GAT-CANET outperforms GCN-CANET in all environments. For example, its accuracy is improved by 1.74% in the OOD1 environment and by 2.2% in the Macro-F1 environment. This indicates that GAT's attention mechanism can more effectively capture the relationships between nodes, especially in geographic network data where node characteristics may be significantly affected by regional differences. However, GAT-SEARL performs worse than GCN-SEARL. For instance, its accuracy in the OOD1 environment drops from 51.73% to 34.25%, and its Macro-F1 accuracy drops from 48.05% to 21.97%.
[0132] Table 2. Prediction results of different models on the Education dataset before and after setting the structural distribution shift (accuracy ± standard deviation % and Macro-f1 ± standard deviation %).
[0133]
[0134] (2) Comparative Experiment and Analysis of Small-Scale Social Network Datasets
[0135] To evaluate the performance of the SEARL model proposed in this chapter, the performance of different methods was compared on the small-scale social network datasets Cora, CiteSeer, and PubMed. The performance comparison results of different methods are shown in Table 3. Seven methods based on GCN and GAT backbone networks were implemented using PyTorch open-source code, including IRM, Coral, DANN, ERM, EERM, CANET, and SEARL. All models were run according to the parameter settings mentioned in the paper, and the average value was taken from multiple experiments to ensure the stability of the results.
[0136] By comparing the accuracy of SEARL with other baseline models, we found that SEARL exhibits significant performance improvements on all small-scale datasets. In experiments based on the GCN backbone network, compared with baseline methods IRM, Coral, DANN, ERM, and EERM, SEARL shows significant improvements in both OOD and ID accuracy, with improvements ranging from 2.23% to 20.52%. Compared to CANET, SEARL outperforms CANET in OOD accuracy, achieving improvements of 0.29% on the Cora dataset, 0.69% on the CiteSeer dataset, and 2.02% on the PubMed dataset. However, in terms of ID accuracy, SEARL lags slightly behind CANET on the Cora and CiteSeer datasets, but the difference is small on the PubMed dataset, only 0.17%.
[0137] In experiments based on the GAT backbone network, SEARL also performed well. Referring to Table 3, the baseline methods under the GAT framework—IRM, Coral, DANN, ERM, and EERM—showed some performance improvements compared to the GCN framework. For example, IRM's OOD accuracy on the Cora dataset improved from 74.19% ± 2.60% to 91.63% ± 1.27%, but still did not surpass SEARL's performance. Compared to CANET, SEARL improved both OOD and ID accuracy on the Cora and CiteSeer datasets. On the PubMed dataset, SEARL's OOD accuracy improved by 0.83%, but its ID accuracy was slightly lower than CANET's.
[0138] Table 3 shows the prediction results (accuracy ± standard deviation %) of different models after setting different distributions for the Cora, Pubmed, and Citeseer datasets.
[0139]
[0140] (3) Comparative experiments and analysis of large-scale social network datasets
[0141] The results of seven methods based on GCN and GAT on large-scale social network datasets are compared, as shown in Tables 4 and 5.
[0142] On the Twitch dataset, SEARL's performance is relatively close to that of state-of-the-art methods (such as CANET and other baselines), but there is still room for improvement. In experiments based on the GCN backbone network, SEARL is 0.55% lower in ID accuracy than CANET. Compared to baselines such as IRM and Coral, SEARL's OOD performance is more stable, indicating that it has a certain generalization ability in subgraph-level distributed offset scenarios. In experiments based on the GAT backbone network, SEARL is 0.80% lower in ID accuracy than CANET, but its OOD accuracy is significantly improved, especially in OOD2 and OOD3, where the difference is only 0.23% and 0.73%, respectively, which is very small. Other baseline methods under the GAT framework have improved in both ID and OOD accuracy, but SEARL's overall performance is still superior, showing its adaptability to subgraph-level environmental changes.
[0143] On the Arxiv dataset, SEARL achieved better overall accuracy than all comparison models. In experiments based on the GCN backbone network, SEARL achieved 0.24% higher ID accuracy than the CANET model, and also improved accuracy in the OOD part. Compared with baselines such as IRM, Coral, and DANN, SEARL's improvement is more significant, especially in the OOD scenario, with accuracy improvements ranging from 2.71% to 8.22%. In experiments based on the GAT backbone network, SEARL achieved 0.42% higher ID accuracy than CANET, and generally improved OOD accuracy. It also showed improvements in both ID and OOD accuracy compared to other baseline methods. Therefore, SEARL demonstrates the best overall performance on the Arxiv dataset.
[0144] Table 4. Prediction results (accuracy ± standard deviation %) of different models with different offsets on the Twitch dataset.
[0145]
[0146] Table 5 shows the prediction results (accuracy ± standard deviation %) of different models after setting different offsets on the Arxiv dataset.
[0147]
[0148] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.
Claims
1. An environment-adaptive geographic network invariant representation learning model, characterized in that... It includes: Environmental variable identification module: includes an environmental estimator that captures the potential environmental variables E of different sampling points through the neighbor structure and neighbor feature information of the target node; The differentiation module utilizes a latent environmental variable E to assist two branch networks, extracting an invariant representation C and a background representation S respectively. The invariant representation C is constrained to have a stable causal relationship with the label Y, maintaining prediction consistency under different environments. The background representation S is designed to be sensitive to environmental changes and has a spurious correlation with the label. MoE Predictor Module: Combines latent environmental variable E, invariant representation C, and background representation S to dynamically select a suitable expert combination.
2. The model according to claim 1, characterized in that... Design an environment estimator to capture the environment vector corresponding to each node. Where N(v) is the first-order neighborhood of node v corresponding to the environment variable, l is the l-th layer of the encoding aggregation representation, and k represents the number of environment vectors set.
3. The model according to claim 2, characterized in that... Modeled as a "categorical distribution" The discrete vector obtained by sampling; where The probability distribution representing environmental categories is expressed by the formula: in, It is the hidden representation of the neighboring nodes N(v) obtained after l layers of convolution. The parameters H are obtained through training. The hidden dimension; Wherein, τ is a temperature hyperparameter that controls the degree to which the generated result approximates the discrete distribution.
4. The model according to claim 1, characterized in that... Indicates the distinguishing module construction g C and g S Two networks, based on the encoded representation Z∈R N×D Learn the invariant representation C and the background representation S separately: C=g C (Z) S=g S (Z) Where S∈R H , S∈R H By increasing attention to C and reducing interference from S, the model can focus more on the stable causal relationship with label Y, thereby improving its cross-environment prediction capabilities.
5. The model according to claim 4, characterized in that: a) Regardless of environmental differences, invariance within the same category indicates high similarity of C, while invariance between different categories indicates low similarity of C; b) Regardless of whether the categories are the same or different, the background representation S in the same environment has high similarity, while the background representation S in different environments has low similarity.
6. The model according to claim 1, characterized in that... The invariant representation C and the background representation S are concatenated with weights and then input into the MoE predictor module. Combined with the environmental vector E captured by the environmental variable identification module, the prediction strategy is dynamically adjusted. The MoE predictor module utilizes K parallel GNN branches as "experts," with each branch modeling potential environmental patterns and using the pseudo-environment vector from each node. It acts as a "router," dynamically selecting or weighting expert outputs.
7. The model according to claim 6, characterized in that... The MoE predictor module is based on the GCN architecture, which aggregates neighbor features through uniform weighting, making it suitable for graphs with regular structures; for a node u in the (l+1)th layer, its representation vector... It is given by the following formula: Where, d u It is the degree of node u. and Let be the trainable weight matrix of the k-th branch in layer l, and σ be the activation function. Summing is performed on nodes v adjacent to node u, where a uv =1 indicates that there is an edge between nodes u and v. This represents the feature representation of node v at layer l.
8. The model according to claim 6, characterized in that... The MoE predictor module is based on the GAT architecture and introduces an attention mechanism, making it suitable for capturing graphs with high heterogeneity among neighbors; the (l+1)th layer represents... Updated to here, Let be the attention weights between node u and its neighbor node v. and b l,k ∈R 2H These are trainable parameters. This means summing the N nodes in the graph, starting from node w=1, by performing a linear transformation on the node features and then applying LeakyReLU and Softmax; finally, after l layers of feature propagation, each node u obtains its representation. Then it is mapped to the final prediction through a fully connected layer.
9. The model according to claim 1, characterized in that... The model also includes a regularization optimization module, which designs three regularizations to constrain the learned invariant representation, the background representation, and the environment variables, respectively: Regularization constraints for environment variables Where N is the number of nodes and K is the number of pseudo-environment categories; It is the cross-entropy loss, which optimizes e to approximate the distribution π of the environmental encoder output by minimizing the cross-entropy loss. i,k This ensures that e reflects environmental information in the characteristics of its neighbors, thereby preventing e from degenerating into an overly sharp distribution; For entropy regularization, we maximize the entropy of e, encourage smooth distribution, and enhance robustness and diversity; γ is a hyperparameter that controls the strength of entropy regularization. Invariant representation constraint L irm : Where ε is the set of pseudo-environments, |ε| is the number of pseudo-environments, and L e Here, h represents the risk function under the pseudo-environment e, C is the invariant representation, and w represents a trainable parameter. To ensure that the risk of w remains consistent across different environments and to prevent the model from overfitting to a specific environment, a gradient penalty term L is introduced. penalty : Background represents constraint L env To bring S closer to environmental information, E is predicted by supervising S: in h is the predicted probability that node i belongs to pseudo-environment category k. S S represents the environment predictor. i h is the background representation of node i. S (S i ) k The environment predictor h S Obtain the background representation S of node i i Then, based on the background representation, the predicted probability that its environment belongs to pseudo-environment category k is obtained; Then, cross-entropy loss is used to encourage... Get as close to E as possible: Where i is the index of a sample in the training set, E i,k The pseudo-environmental distribution, N, is obtained by assuming that "neighbor node information and structure reflect potential environmental factors". train This is the number of training samples.
10. An environment-adaptive geographic network invariant representation learning method, characterized in that... The model described in any one of claims 1-9 is used for implementation.