LLM guidance-based graph expert routing semi-supervised field generalization method and system

Abstract

The invention relates to the field of neural network model supervised learning, in particular to a graph expert routing semi-supervised field generalization method and system based on LLM guidance. A graph expert routing semi-supervised field generalization method based on LLM guidance comprises the following steps: acquiring graph structure data containing manual annotation node labels, and taking the graph structure data as original annotation data; and for a target node in the graph structure data, extracting center sub-graph data corresponding to the target node according to a preset hop count range. According to the method, the problems of weak cross-domain generalization, poor routing self-adaption and label scarcity of a traditional model are solved through multi-hop center sub-graph processing, expert suitability evaluation by a large language model, knowledge distillation migration routing logic, weighted fusion of features and pseudo-label expansion labeling data, the feature expression and expert selection precision is improved, and the method is suitable for large-scale popularization and application. The prediction error caused by distribution offset is reduced, and the robustness and classification performance of the model in an unknown domain are enhanced.

Description

Technical Field

[0001] This invention relates to the field of supervised learning of neural network models, and in particular to a semi-supervised domain generalization method and system based on LLM-guided graph expert routing. Background Technology

[0002] Graphs are a fundamental data structure for capturing complex relationships and structural dependencies. Graph Neural Networks (GNNs), with their proven effectiveness and computational efficiency in graph-based machine learning, have become a general method for various tasks. Their core task, node classification, has been widely applied in social network analysis, biochemistry, recommender systems, and other fields. However, GNNs rely on the assumption that training and testing data are independent and identically distributed, making them susceptible to out-of-distribution (OOD) problems. Their performance degrades significantly when data distribution shifts, making it difficult to adapt to the diverse data distribution requirements in reality.

[0003] To prevent GNN generalization failure, invariant learning is a promising strategy for generalizable graph learning. Its core is to identify stable causal features or invariant structures and remove spurious associations between non-essential features and labels. For example, the semi-supervised domain adaptation method based on active adversarial learning in Chinese patent application CN112541580A first initializes the feature extraction network, domain discriminator, and classifier. Through adversarial game theory, the feature extractor confuses the discriminator's judgment of sample source, narrowing the distribution distance between the source and target domains and extracting domain-invariant features. Then, through active learning, high-value data is labeled from unlabeled samples in the target domain to supplement supervisory information. Finally, a multi-class discriminator alleviates inter-domain distribution differences and strengthens cross-domain invariant feature learning, providing a feasible approach for GNN generalization.

[0004] However, to effectively separate invariant components from spurious associations, graph neural network models need exposure to diverse training environments. In real-world scenarios, unseen domains often contain numerous potential influencing factors, the quantity and attributes of which are difficult to define and reproduce. Label scarcity further exacerbates this problem, making it difficult to reliably construct the diverse training environments required for invariant learning. Furthermore, graph data is characterized by heterogeneous neighborhood structures and uneven node degree distribution, which is drastically different from standard Euclidean structure data. Since the message-passing framework of graph neural networks primarily prioritizes highly modular subgraphs and nodes with high degree, graph neural network models are highly susceptible to simplicity bias. Ultimately, graph neural network models tend to rely on salient but potentially misleading association features, sacrificing the extraction of the underlying invariant substructures necessary for generalization and robust prediction.

[0005] The superior generalization ability of Large Language Models (LLMs) offers new avenues for addressing related challenges. Their rich pre-trained knowledge and semantic reasoning capabilities provide valuable guidance for graph neural network learning. Some studies use LLMs as enhancers to enrich node or graph representations. For example, Eli Chien et al.'s method of node feature extraction by self-supervised multi-scale neighborhood prediction (In Proceedings of the International Conference on Learning Representations) provides a starting point in this direction. Subsequent research incorporates LLM's semantic understanding capabilities, further enriching node representations by parsing the context of the graph structure. Other methods directly use LLMs as predictors, such as Haiteng Zhao et al.'s (2023) Gimlet framework (A unified graph-text model for instruction-based molecule zero-shot learning) which constructs a unified graph-text fusion model for instruction-based molecule zero-shot learning, directly leveraging LLMs to generate prediction results. However, LLM is essentially a discrete text output design based on a decoder architecture, which limits its accurate representation of complex graph semantics. Furthermore, directly using it to generate graph task prediction results can easily lead to unstable output or irrelevant results. Summary of the Invention

[0006] This invention provides a semi-supervised domain generalization method and system for graph expert routing based on LLM guidance to solve the above problems.

[0007] To solve the above-mentioned technical problems, this application provides the following technical solution: A semi-supervised domain generalization method for graph expert routing based on LLM guidance includes the following steps: Obtain graph structure data containing manually labeled nodes and use the graph structure data as the original labeled data; for the target node in the graph structure data, extract the central subgraph data corresponding to the target node according to the preset hop count range, input the central subgraph data into the preset expert module corresponding to the hop count, and perform feature extraction and prediction calculation on the central subgraph data through the expert module to obtain the node features and prediction results output by the expert module; The structural information, node connection relationships, and prediction results contained in the central subgraph data are converted into prompt text. The prompt text is then input into a pre-trained large language model. The adaptability of the expert module is evaluated by the large language model based on zero-shot reasoning ability, and the expert decision data output by the large language model is obtained. Using expert decision data as a monitoring signal, the routing logic corresponding to the expert decision data is migrated to the preset routing module through knowledge distillation. The routing module then generates routing allocation data based on node characteristics. Based on the routing allocation data, the node features and prediction results are weighted and fused to obtain enhanced node features; Pseudo-label data is generated based on enhanced node features. Pseudo-label data with confidence scores higher than a preset threshold are selected to augment the original labeled data, resulting in augmented labeled data. Using the original labeled data and the augmented labeled data, a node classifier built based on enhanced node features is trained under supervision to obtain a graph semi-supervised domain generalization model.

[0008] The basic principle and beneficial effects of this invention are as follows: This invention acquires labeled graph structure data as initial supervision information, extracts central subgraphs for target nodes according to different hop counts, and uses corresponding expert modules for feature and prediction processing. Then, the structural information of the central subgraphs and related prediction results are transformed into prompt text input to a large language model to obtain adaptation decisions. Through knowledge distillation, the routing logic is transferred to a lightweight routing module to achieve reasonable feature and result fusion, resulting in enhanced node features with stronger discriminative power. Based on these enhanced features, high-confidence pseudo-labels are generated to expand the labeled data. Finally, the expanded supervision data is used to train a classifier to obtain a graph semi-supervised domain generalization model. This model can improve the cross-domain adaptability of graph data in scenarios with limited labels, effectively alleviate prediction errors caused by domain distribution shifts, and simultaneously improve the rationality of expert selection (expert module selection) and the overall robustness of the graph semi-supervised domain generalization model.

[0009] This invention assigns central subgraph data with different hop counts to corresponding preset expert modules for targeted processing. This enables graph semi-supervised generalization models to learn the local and global structural information of nodes more precisely, avoiding the problem that a single model structure cannot adapt to diverse data distributions, and further improving the modeling ability for complex graph structures. For example, in mixed graph data containing densely connected nodes and sparsely connected nodes, preset expert modules with different hop counts can focus on capturing the local association features of densely connected regions and the global dependency features of sparsely connected regions, respectively, so that the structural information of various types of nodes can be fully represented. At the same time, the structural information, node connection relationships, and prediction results of the central subgraph data are converted into prompt text and evaluated for adaptability using a pre-trained large language model. This fully utilizes the context-awareness capability of the pre-trained large language model to achieve expert selection that is more in line with the actual data distribution. For example, in cross-domain graph tasks with large differences in node structure, the pre-trained large language model can accurately identify the structural characteristics of the current node through the prompt text and match the preset expert module most suitable for processing this type of structure. This improves the poor adaptability of traditional fixed routing methods and allows graph semi-supervised generalization models to maintain stable judgment capabilities in complex cross-domain scenarios.

[0010] This invention migrates the routing logic corresponding to expert decision data to a pre-defined lightweight routing module through knowledge distillation. This significantly reduces the operating cost and resource consumption of the pre-defined routing module while preserving the routing accuracy provided by the strong inference capabilities of the pre-trained large language model. It avoids the instability and inefficiency caused by directly using the pre-trained large language model for inference, thus improving the overall practicality and deployment feasibility of the technical solution. For example, in resource-constrained deployment scenarios such as edge computing devices, the pre-defined lightweight routing module can efficiently complete expert routing decisions without relying on high-performance hardware to support the operation of the pre-trained large language model. Simultaneously, by weighted fusion processing of node features and prediction results based on routing allocation data, the effective feature information output by the highly adaptable pre-defined expert module is enhanced, while noise interference from the poorly adaptable pre-defined expert module is suppressed. This further improves the discriminative and expressive power of node features, providing more reliable data support for subsequent pseudo-label data generation and node classifier training, enabling the node classifier to learn more discriminative cross-domain features.

[0011] This invention generates pseudo-label data based on enhanced node features and selects high-confidence pseudo-label data to expand the original labeled data. This can supplement effective supervision signals in semi-supervised scenarios with insufficient labeled samples, alleviating the problem of insufficient generalization ability of graph semi-supervised generalization models caused by label scarcity. For example, in a graph dataset where only 10% of nodes have manually labeled data, the high-confidence pseudo-label data generated by enhancing node features can increase the scale of effective supervision samples several times. Furthermore, the quality of the pseudo-label data is guaranteed by the strong representational ability of enhanced node features, avoiding interference from low-quality pseudo-labels during training, and allowing graph semi-supervised generalization models to learn more comprehensive cross-domain feature patterns. Meanwhile, training a node classifier based on enhanced node features using both original and augmented labeled data can strengthen the learning effect of the node classifier on domain-invariant features, significantly improving the classification accuracy of the graph semi-supervised domain generalization model in unknown domains. For example, in cross-domain tasks where the training domain is a social network graph and the test domain is an academic citation graph, the graph semi-supervised domain generalization model can accurately complete the classification prediction of unknown domain nodes by fusing the domain-invariant features learned from the two types of data. This makes the final graph semi-supervised domain generalization model more adaptable and stable in various cross-domain graph tasks.

[0012] In summary, this invention addresses the problems of weak cross-domain generalization, poor routing adaptation, and scarce labels in traditional models by using multi-hop central subgraph processing, large language model evaluation of expert adaptability, knowledge distillation transfer routing logic, weighted fusion features, and pseudo-labels to expand labeled data. It improves the accuracy of feature representation and expert selection, reduces prediction errors caused by distribution offset, and enhances the robustness and classification performance of the model in unknown domains.

[0013] Furthermore, preset expert modules corresponding to different hop counts are used to extract structural context information at different levels. The expert modules generate corresponding node features and prediction results through mutually independent information aggregation methods, and realize information fusion based on the structural context corresponding to the node.

[0014] Expert modules with different hop counts extract structural context information at different levels. They generate node features and prediction results through independent information aggregation methods and achieve information fusion by combining node structural context. This allows graph semi-supervised generalization models to simultaneously capture the local homogeneity features and global heterogeneity dependencies of graph data, avoiding the problem that a single model structure cannot adapt to diverse graph data distributions. It also effectively avoids the oversmoothing and overfitting problems caused by the indiscriminate aggregation of multi-hop neighborhoods in traditional GNNs. At the same time, the independent information aggregation method can prevent feature information with different hop counts from interfering with each other, improving the accuracy of feature extraction. The fusion method combined with node structural context makes the features more in line with the actual classification needs of nodes, laying a high-quality feature foundation for subsequent expert selection and route allocation, and improving the ability of graph semi-supervised generalization models to model complex graph structures.

[0015] Furthermore, each expert module constructs a correlation metric between nodes based on the number of times they co-occur. The obtained correlation metric is used as the adjacency relationship of the corresponding central subgraph data. A graph neural network with the same network structure and independent parameters is used to extract node representations from the central subgraph data. The extracted node representations are used as enhanced node features. The routing module assigns weights to the node representations of different expert modules, and then the node classifier is used to fuse them to obtain the final prediction result. The node classifier is a learnable classifier.

[0016] Each expert module constructs an association metric based on the number of node co-occurrences as the adjacency relationship of the central subgraph, which can more accurately uncover the implicit structural associations between nodes and better reflect the actual distribution characteristics of graph data compared to the original adjacency matrix. A graph neural network with the same network structure but independent parameters is used to extract node representations, ensuring the consistency and comparability of feature extraction by the expert modules. This also allows each expert module to perform specialized learning based on the neighborhood distribution of the corresponding hop count, adapting to the structural context features of different hop counts. The final prediction result is obtained by weighting through the routing module and fusing with a learnable node classifier, achieving adaptive weighted integration of expert features. This allows the graph semi-supervised generalization model to select more valuable expert representations based on node characteristics, improving the accuracy of node classification prediction. Simultaneously, the learnable classifier can better fit the mapping relationship between features and labels, further strengthening the classification capability of the graph semi-supervised generalization model.

[0017] Furthermore, for the target node, a dedicated context information is constructed for the central subgraph data corresponding to each expert module. The dedicated context information is transformed based on the structural context information of the corresponding hop count central subgraph data extracted by each expert module. It includes the guiding description of expert module selection, the attribute description of the hop count neighborhood of each expert module, and the node connection relationship description of the central subgraph data. Expert modules with different hop counts are matched with the dedicated attribute description of the corresponding hop count neighborhood. The node connection relationship of the central subgraph data is expressed in an associative manner.

[0018] Constructing dedicated contextual information for the central subgraph corresponding to each expert module transforms the unstructured information of the graph structure into a structured description that conforms to the understanding logic of the large language model, providing clear and effective information support for the expert adaptability evaluation of the large language model. The dedicated contextual information includes guidance descriptions, neighborhood attribute descriptions, and node connection relationship descriptions, enabling the large language model to accurately identify the processing characteristics of each expert module and the structural features of the corresponding central subgraph. The dedicated neighborhood attribute descriptions and node connection relationships of expert modules with different hop counts are matched using an associative expression, further improving the accuracy of the large language model in judging the adaptability relationship between experts and nodes, avoiding the bias in expert module selection caused by ambiguous information descriptions, and providing a reliable textual foundation for subsequent LLM-guided expert routing.

[0019] Furthermore, task-specific prompts are generated based on the constructed exclusive context information. These prompts include a task description for the node classification task and candidate options selected by the expert modules. The candidate options are a set of experts composed of all expert modules. The task-specific prompts are then fused with the prediction results output by each expert module to form complete prompt information input to the large language model. The large language model selects an expert module that is suitable for the node classification task based on the complete prompt information, and the selected expert module is taken as the optimal expert module.

[0020] Generating task-specific prompts containing task descriptions and expert candidate options based on contextual information clarifies the task objectives and expert selection range for large language models, making their reasoning and judgments more targeted. Integrating these task-specific prompts with the prediction results of each expert module to form complete prompt information allows the large language model to simultaneously evaluate expert suitability by considering graph structure features and expert prediction performance, improving the rationality and accuracy of expert selection. By selecting the appropriate expert module based on the complete prompt information and designating it as the optimal expert module, the large language model fully leverages its zero-shot reasoning and semantic understanding capabilities, achieving context-aware expert selection. This improves upon the poor adaptability of traditional fixed routing methods, enabling the model to match the most suitable expert module based on the specific structural characteristics of nodes, providing a core basis for subsequent weight updates and routing logic migration.

[0021] Furthermore, the prediction results output by each expert module are integrated into a prediction result set. This prediction result set, along with the complete prompt information, is input into a large language model, which outputs the optimal expert module identifier for the node classification task. A corresponding weight vector is configured for each expert module, and the weight vector corresponding to each expert module is initialized as a one-hot vector. In this one-hot vector, the dimension corresponding to the expert module is 1, and the other dimensions are 0. Based on the optimal expert module identifier, a label smoothing strategy is used to update the weight vector. The optimal expert module is assigned a smoothing coefficient weight value within a preset range, and the remaining weight values ​​are equally distributed among the non-optimal expert modules.

[0022] Integrating the prediction results of each expert module into a set and inputting it along with the complete prompt information into the large language model allows the large language model to comprehensively acquire the prediction performance of all experts, improving the accuracy of the output of the optimal expert module identifier. Assigning weight vectors to each expert module and initializing them as one-hot vectors achieves accurate initial allocation of expert weights, allowing the optimal expert module to play a core role in the initial stage. Based on the optimal expert module identifier, a label smoothing strategy is used to update the weight vectors, assigning a smoothing coefficient weight value within a preset range to the optimal expert module and equally distributing the remaining weights to non-optimal expert modules. This strengthens the core role of the optimal expert module in feature fusion, avoids the model overfitting problem caused by one-hot vector weight allocation, and retains a certain weight ratio for non-optimal expert modules, fully utilizing the feature information of each expert module, improving the richness of feature fusion, and providing a high-quality weight reference for subsequent learning and optimization of the routing function.

[0023] Furthermore, from the graph structure data node set containing manually labeled nodes and unlabeled nodes, a greedy node selection strategy is adopted to select the anchor node set with the objective of maximizing the number of activatable nodes in the anchor node set while satisfying a preset quantity constraint. The influence of the anchor node on the target node is obtained by calculating and summing the entropy values ​​of the prediction results output by each expert module. The transition probability between nodes is obtained by sampling paths through random walks and counting the number of co-occurrences between nodes. The transition probability is combined with the influence to obtain the quality of the anchor node's influence on the target node. The quality of influence is compared with a preset activation threshold, and the activation node set of the anchor node set is determined from the target nodes based on the comparison result. A routing function is obtained by distilling knowledge from the large language model based on the anchor node set. The distillation loss is calculated based on the weight vector of the anchor node and the output value of the routing function for splicing the multi-hop node representation of the anchor node. The routing function is iteratively optimized based on the distillation loss, and routing weights are assigned to the node representations of each expert module through the optimized routing function. The final prediction result of the target node is obtained by fusing the weighted node representations.

[0024] By maximizing the number of activatable nodes in the anchor node set and combining it with a greedy node selection strategy, the anchor node set can be selected with the strongest influence under a preset quantity constraint, achieving efficient distillation of routing knowledge in large language models and reducing the computational cost of subsequent inference. The influence of the anchor node on the target node is obtained by summing the entropy values ​​of expert prediction results, and the influence quality is obtained by combining the transition probability obtained from random walks. This can accurately quantify the actual degree of influence of the anchor node on the target node. Based on the influence quality and the preset activation threshold, the set of activated nodes is determined, which can screen out the nodes that are effectively influenced by the anchor node, improving the targeting and effectiveness of knowledge distillation. By distilling knowledge from a large language model using anchor node sets to obtain routing functions, and combining anchor node weight vectors with the output values ​​of the routing functions to calculate distillation loss and iteratively optimize the routing functions, an effective migration of context-aware routing logic from large language models to lightweight routing modules is achieved. This retains the routing accuracy guided by LLM while avoiding the inefficiencies and unstable output caused by directly using large language models for inference. By assigning routing weights to each expert module through the optimized routing function and fusing representations to obtain the final prediction results, adaptive expert feature weighted fusion is achieved. This can accurately allocate expert weights based on the multi-hop representations of nodes, improving the node classification accuracy and generalization ability of graph semi-supervised generalization models under distribution offsets.

[0025] Furthermore, for unlabeled nodes in the graph structure data, sharpened pseudo-labels are generated based on enhanced node features; the entropy value of the prediction results output by each expert module for the unlabeled nodes is calculated, and a preset quantile of the entropy value is taken as a dynamic threshold. The entropy value output by the best expert module for the unlabeled nodes is compared with the dynamic threshold, and unlabeled nodes with entropy values ​​less than or equal to the dynamic threshold are selected to form a high-quality pseudo-labeled node set; based on the high-quality pseudo-labeled node set, the unsupervised classification loss is calculated according to the prediction results after the sharpened pseudo-labels and the node classifier are fused with the node representations of each expert module, and the unsupervised classification loss is used to fine-tune the graph semi-supervised domain generalization model.

[0026] For unlabeled nodes, sharpened pseudo-labels are generated based on enhanced node features. This fully utilizes the strong discriminative power of enhanced node features, improving the quality of pseudo-labels and providing effective supervision signals for semi-supervised training. The prediction entropy values ​​of each expert module for unlabeled nodes are calculated, and a preset quantile is used as a dynamic threshold. This adaptively determines the pseudo-label selection criteria based on the prediction performance of all experts, better reflecting the actual prediction situation of the model compared to a fixed threshold. Comparing the entropy value of the optimal expert module with the dynamic threshold and selecting a high-quality pseudo-label node set effectively filters out low-confidence noisy pseudo-labels, avoiding interference from low-quality pseudo-labels in model training. Unsupervised classification loss is calculated based on the high-quality pseudo-label node set, and the model is fine-tuned. This fully utilizes the effective information of unlabeled nodes, supplementing supervision signals in scenarios with scarce labels, alleviating the problem of decreased generalization ability of graph semi-supervised generalization models due to insufficient labeled data. Simultaneously, fine-tuning training allows graph semi-supervised generalization models to better adapt to the feature distribution of unlabeled nodes, further improving the robustness and classification performance of graph semi-supervised generalization models in unknown domains.

[0027] Furthermore, the distance correlation coefficients of the node representations of all nodes output by each expert module are calculated, and the expert diversity loss is obtained by summing the distance correlation coefficients. For the labeled nodes in the graph structure data, the supervised classification loss is calculated based on the prediction results after the node classifier fuses the node representations of each expert module, as well as the manually labeled labels of the labeled nodes. The supervised classification loss of labeled nodes, the unsupervised classification loss of unlabeled nodes, the distillation loss, and the expert diversity loss are weighted and fused to obtain the overall loss function. Among them, the distillation loss and the expert diversity loss are respectively configured with corresponding balance factors. The graph semi-supervised domain generalization model is trained and optimized based on the overall loss function.

[0028] Calculating the distance correlation coefficients of the node representations of each expert module and summing them yields the expert diversity loss, which effectively quantifies the differences in the representations of each expert module. This loss constraint encourages each expert module to extract discriminative features, avoids functional overlap, and allows each expert module to more fully explore the graph structure features with different hop counts, improving the diversity and comprehensiveness of feature extraction. For labeled nodes, supervised classification loss is calculated by combining model prediction results and manually labeled tags. This allows generalization models in the semi-supervised graph domain to learn the basic rules of node classification on labeled data, constraining model prediction results to fit the true labels and providing core supervision signals for model training. Supervised classification loss, unsupervised classification loss, distillation loss, and expert diversity loss are then compared with the differences in the representations of each expert module. The overall loss function is obtained by weighted fusion of expert diversity loss and a balancing factor is configured for distillation loss and expert diversity loss. This enables the synergistic optimization of multiple loss terms, balancing multiple training objectives of graph semi-supervised domain generalization models, such as supervised learning, unsupervised learning, route knowledge distillation, and expert feature diversification, and avoiding a single loss term dominating the training of graph semi-supervised domain generalization models. The training optimization of the model based on the overall loss function enables the graph semi-supervised domain generalization model to have good node classification ability, cross-domain generalization ability, route adaptation ability, and expert feature extraction diversity. This effectively reduces the prediction error caused by distribution offset and significantly improves the semi-supervised domain generalization performance of graph semi-supervised domain generalization models in label-scarce scenarios. Attached Figure Description

[0029] Figure 1 This is a flowchart of the LLM-guided semi-supervised domain generalization method for graph expert routing in the embodiment. Figure 2 The graph shows the Micro F1 scores for different expert module types in the embodiment and the experimental results when the number of expert modules is increased proportionally. Figure 3 For example, A under different labeling rates in the embodiment C and A Performance comparison results of Task D; Figure 4 In the example, LEGO in A Visualization results of t-SNE analysis on task C; Figure 5 In the example, LEGO w / o Distill in A Visualization results of t-SNE analysis on task C; Figure 6 In the examples, the baseline method SGDA is in A Visualization results of t-SNE analysis on task C; Figure 7 In the example, A C and A Radar charts showing the performance of different LLMs on mission D; Figure 8 To illustrate the loss weight pair A in the example, C and A A 3D histogram showing the effect of the Micro F1 score on the D task; Figure 9 Smoothing coefficient With activation threshold In A C, A Effect analysis diagram on task D; Figure 10 Smoothing coefficient With activation threshold In C A, C Effect analysis diagram on task D. Detailed Implementation

[0030] The following will describe the concept and technical effects of the present invention clearly and completely with reference to embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the scope of protection of the present invention. Example Graph generalization (GDG) is an important research direction for addressing the significant performance degradation of graph neural networks on out-of-distribution data. Early work mainly focused on self-supervised pre-training, feature decoupling, and adversarial training. In recent years, causal inference-based methods have gradually become mainstream, while data augmentation and label semantic preservation strategies have also been extensively studied. Multi-component frameworks built with techniques such as meta-learning, causal inference, topological adversarial learning, and prototype hybridization have further promoted the development of graph generalization towards more complete solutions. However, most current methods typically assume that all source domain data is labeled, making them difficult to apply to real-world scenarios where labels are scarce or costly. Therefore, this invention focuses on graph generalization tasks in semi-supervised scenarios to improve the practicality and applicability of the methods.

[0031] Graph-based semi-supervised learning (GSSL) is a mainstream learning paradigm in scenarios with limited labeled samples but abundant graph structures and unlabeled data. Traditional methods primarily rely on label propagation and graph regularization to improve performance by forcing prediction smoothness. Recent research focuses more on model robustness and scalability, often combining regularization, pseudo-labels, data augmentation, and graph contrastive learning strategies. Based on various graph neural networks, several advanced semi-supervised learning paradigms, such as self-training and contrastive learning, have emerged and achieved good results in unsupervised representation learning. However, how to effectively implement graph-based semi-supervised learning under the condition of distributional bias remains insufficiently explored, which is one of the key issues addressed in this invention.

[0032] This invention leverages the zero-shot inference capability of large language models, which require no additional training, to extract structural information from the different hop count ranges of each node in graph-structured data. This information is then converted into natural language text information with a clear structured format that aligns with the semantic understanding logic of large language models. Based on this text information, expert modules constructed using graph neural networks adapted to node processing are automatically selected to build a graph semi-supervised domain generalization model. This significantly improves the generalization performance and classification performance of the graph semi-supervised domain generalization model on unseen new graph data.

[0033] In the LLM-guided semi-supervised domain generalization method for graph expert routing in this invention, the MoE framework, or Mixture of Experts framework, is a model architecture that uses multiple expert modules as the core and combines the routing mechanism of the routing module to achieve adaptive weighted fusion of features. The core is to use multiple functionally specialized expert modules to process graph data with different distribution characteristics, and then the routing module dynamically allocates weights to complete feature fusion.

[0034] In this embodiment, let Represents a graph structure data, where It includes A set of nodes with nodes. It is an edge set. The adjacency matrix is ​​represented as... If there are edges ,but ,otherwise .in Unlabeled nodes For labeled nodes; It is a graph adjacency matrix Corresponding node and nodes Element.

[0035] Furthermore, the node feature matrix can be represented as ,in It is the feature dimension. For the real number field. From a data generation perspective, the input graph (i.e., graph-structured data) can be decomposed into a set of overlapping central subgraphs (ego-graphs), nodes of A jump center subgraph can be defined as , where the feature matrix and adjacency matrix Each by node and Skip neighborhood export, that is, Let be the feature matrix of the k-hop center subgraph of the target node v; Let be the adjacency matrix of the k-hop center subgraph of the target node v.

[0036] When performing semi-supervised domain generalization on a graph, given a graph ,remember , This is the set of labeled nodes (also called the set of supervised nodes), where each node... A label was assigned , It is the number of categories. This is the category label for the node. If the node... Belongs to the Class, then ,otherwise The rest It is a set of unlabeled nodes.

[0037] In this embodiment, the distribution offset on the graph refers to the difference in the distribution of the central subgraph and its labels, i.e. , The distribution of the training data, i.e., the joint distribution of "central subgraph + label" in the training set; The distribution of the test data, i.e., the joint distribution of "center subgraph + labels" in the test set, and the distribution offset are... ; For nodes The central subgraph, For nodes The true category labels. The goal of semi-supervised domain generalization in this embodiment is to generalize based on source graphs with finite labels. Learn an optimal predictor This allows it to maintain good performance even on unseen target maps. As shown below: , in This is the loss function. Then, the learned graph semi-supervised domain generalization model is used to infer the node labels of the target graph. Here, the source and target graphs share the same label space. The expectation is based on the training distribution and is used to measure the average level of the loss function on the training data.

[0038] like Figure 1 As shown, the LLM-guided graph expert routing semi-supervised domain generalization method includes the following steps: Obtain graph structure data containing manually labeled nodes and use the graph structure data as the original labeled data; for the target node in the graph structure data, extract the central subgraph data corresponding to the target node according to the preset hop count range, input the central subgraph data into the preset expert module corresponding to the hop count, and perform feature extraction and prediction calculation on the central subgraph data through the expert module to obtain the node features and prediction results output by the expert module.

[0039] Among them, the expert modules are a group of dedicated graph neural network (GNN) modules designed for specific hop count center subgraphs. They have the same network structure but independent parameters. Each expert module has a clear "specialization range", that is, it only processes the center subgraph data of the corresponding hop count (such as 0 hops, 1 hop, 2 hops) and will not process information across hop counts. For example, the 0-hop expert module only processes the feature data of the target node itself (without neighboring nodes), focusing on the extraction of the node's own attribute features.

[0040] Among them, the preset expert modules corresponding to different hop counts are used to extract structural context information at different levels. The expert modules generate corresponding node features and prediction results through mutually independent information aggregation methods, and realize information fusion based on the structural context corresponding to the node.

[0041] Assume it exists Each expert module (i.e.) The total number of expert modules is 100,000. Each expert module learns a specific domain (i.e., the domain). Each domain may have potential distribution biases (for example, the test domain may contain a large number of potential influencing factors that did not appear during training, such as the addition of node types, changes in connection rules, expansion of attribute dimensions, etc. These factors cannot be fully reproduced or predicted before training, and are therefore potential distribution biases). Therefore, domain-specific modeling (which is the core operation of customizing model parameters and exclusive learning logic for each domain based on the unique distribution characteristics of different data domains, capturing key features within the domain in a targeted manner, avoiding false association interference caused by distribution biases, and ultimately ensuring the generalization performance of the model in cross-domain scenarios) is crucial.

[0042] To simplify the notation, this embodiment focuses on In a binary classification scenario, it is assumed that there are two distinct domains: the front and back domains. The domain-specific distribution of the experts matches the first domain, and the remaining... One expert is matched with the second domain.

[0043] Assuming the pre-defined routing module (a lightweight, learnable module pre-built to implement feature adaptive weighted fusion and transfer of routing decision logic from large language models in the expert module) has given routing weights. The following conditions must be met for the optimal expert module. , For other expert modules , This indicates that the routing module assigns equal weights to the best expert module while ignoring other expert modules.

[0044] node The final prediction result is defined as: , in For nodes By the The representation obtained from each expert module For the first The routing weight of each expert module, A learnable classifier. Further hypotheses regarding the classifier... Output the linear combination of the sigmoid transform for the expert module: , in It is the Sigmoid activation function. The weight matrix of the classifier. weight matrix The transpose of . In this form, This can be interpreted as the first An expert on the node The log-odds output is used. Due to the monotonicity of the sigmoid function, the larger the log-odds value, the higher the probability that the node is predicted to be in class 1. Class 1 is the first class (numbered 1) in the predefined set of classes (C classes in total) in the node classification task. The following lemma establishes a threshold-based decision rule for the aggregated expert logit.

[0045] The lemma states that there exists a threshold. Make: , In multiple settings In this context, a similar formula can be applied by replacing the sigmoid function with the softmax function and defining a category-specific threshold for each output dimension.

[0046] With this lemma, the performance of the proposed LEGO framework can be analyzed. Specifically, the goal of this embodiment is to demonstrate that a hybrid expert framework built on a hybrid expert (MoE) framework can outperform a single expert module in terms of classification accuracy.

[0047] Without loss of generality, this embodiment assumes the dynamic threshold in the lemma. Set it to 0. Let the first... Each expert module is paired with a node. The logit output is And let its expected value be For a given node Assuming experts The correct expert module (i.e., with) (The distribution is most aligned in a specific domain). Therefore, the expected logit It should exceed the positive margin. And any incorrect expert module It should have average logit output. This reflects that only the correct expert module can effectively model nodes. The distribution characteristics. In this embodiment, it is further assumed that the logit of different expert modules is independent and follows a sub-Gaussian distribution, that is: , in It is a constant, and is a sub-Gaussian parameter. Let be any real number, and be the independent variable of the exponential function in the definition of a sub-Gaussian random variable. Based on these assumptions, this embodiment can analyze the prediction error of a graph semi-supervised domain generalization model.

[0048] Assuming nodes The true label is 1. If only one expert module exists (i.e., ... ), and the expert module and node If the domain-specific distribution does not match, then the prediction error satisfies the lower bound: , Conversely, if there exists If there are multiple expert modules, the prediction error satisfies the upper bound: , in For boundary-dependent and second Gaussian parameters The constant.

[0049] The proof of the above is as follows: This embodiment first analyzes the case where only a single expert module is available, that is... Assume a node The true label is 1. Therefore, the prediction error of the semi-supervised domain generalization model for this graph is given by the following formula: , Let be a probability symbol. The goal of this embodiment is to prove that this probability is strictly greater than zero, i.e. .

[0050] Because this expert module and node The distribution of specific domains is inconsistent; in this embodiment, it is assumed that... Define the following events , , :

[0051] because It is not almost necessarily a constant; in this embodiment, it must have .if Then inevitably This is consistent with the assumption A contradiction. Therefore, there must be... Furthermore, the prediction error is strictly positive in the case of a single expert module.

[0052] Now considering The case of an expert module. Assume an expert module... With nodes The domains are correctly aligned, therefore the expected logit output is achieved. And all other expert modules It has zero-mean output. Therefore, the expected value of the aggregated logit is: , Assuming the expert module outputs It is an independent agent with variance. The distribution follows a sub-Gaussian pattern. In this embodiment, Hoeffding's inequality is used to constrain the tail probability: , Therefore, the prediction error of the graph semi-supervised domain generalization model satisfies: , For a certain constant .

[0053] It is worth noting that neighborhoods with different hop counts can capture different levels of contextual semantics: low hop count (e.g., 1 hop) neighborhoods typically reflect local homogeneous patterns, while high hop count neighborhoods encode global and heterogeneous dependencies. Therefore, in this embodiment, the information aggregation process is decoupled according to the hop count, and a set of prediction results is generated by designing a graph hybrid expert framework. That is, in the hybrid expert framework, expert modules with specific hop counts generate prediction results, and then adaptive fusion is performed based on node context to improve the robustness of graph semi-supervised domain generalization models under distributed offsets.

[0054] Each expert module constructs a correlation metric between nodes based on the number of times they co-occur. The obtained correlation metric is used as the adjacency relationship of the corresponding central subgraph data. A graph neural network with the same network structure and independent parameters is used to extract node representations from the central subgraph data. The extracted node representations are used as enhanced node features. The routing module assigns weights to the node representations of different expert modules, and then the node classifier is used to fuse them to obtain the final prediction result. The node classifier is a learnable classifier.

[0055] Specifically, for Jump center diagram In this embodiment, positive point mutual information (PPMI) between nodes is calculated to fully explore the structural information of the graph. Specifically, random walks are performed on the adjacency matrix. Upsample a set of paths, frequency matrix between nodes The co-occurrence frequency along the path is calculated; subsequently, the point-to-point mutual information between nodes is calculated as follows:

[0056] in Represents a node Appearing in node The probability in the context window, This represents positive mutual information between the two. Therefore, The larger the value, the higher the co-occurrence frequency between nodes is compared to the expected frequency when they are independent. The symbol is for probability; F is the node co-occurrence frequency matrix; , The index identifiers for the nodes in the graph are used to locate the frequency matrix F and the probability matrix. Elements in the PPMI matrix (like , ).

[0057] In this embodiment, As Jump center diagram The new adjacency matrix is ​​obtained, and nodes are extracted in the following way. The representation of: , in For having A graph neural network with multiple layers. For the first Enhanced node features output by an expert. For nodes of The feature matrix of the jump center subgraph. For nodes of Jump center subgraph PPMI matrix. It should be noted that in this embodiment... Each expert module uses the same network architecture, but its parameters are independent. Weights are assigned to each expert module using the routing mechanism of the hybrid expert routing module, and then a classifier generates the final prediction result. : , in For a learnable classifier, For the first The routing weights of each expert module.

[0058] For each node First, extract the data corresponding to different hop counts. Each central subgraph is used to effectively capture the context of multi-hop structures; subsequently, each central graph is... As input to the expert module, and to provide the expert module description to the large language model ( ) and mission objectives ( ),in It is a discrete token sequence for a large language model.

[0059] The structural information, node connection relationships, and prediction results contained in the central subgraph data are converted into prompt text. The prompt text is then input into a pre-trained large language model. The large language model's zero-shot reasoning ability is used to evaluate the fit of the expert module, resulting in expert decision data output by the large language model.

[0060] Specifically, for each expert module, a dedicated context information is constructed for the central subgraph data corresponding to the target node. The dedicated context information is transformed based on the structural context information of the corresponding hop count central subgraph data extracted by each expert module. It includes the guiding description of expert module selection, the attribute description of the hop count neighborhood of each expert module, and the node connection relationship description of the central subgraph data. Expert modules with different hop counts are matched with the dedicated attribute description of the corresponding hop count neighborhood. The node connection relationship of the central subgraph data is expressed in an associative manner.

[0061] More specifically, given a node For each expert module, a context is constructed for the central subgraph, in the form of "You are a graph neural network expert selector, given the following graph neural network experts: <expert>... given node..." And its neighborhood {graph}, where {graph} represents the connection relationship between nodes... Here, "<expert>" is a placeholder for the expert module description, and "{graph}" is the structural information of the central subgraph. It should be noted that the first... The input for each expert module is about The description of hop neighborhoods, for example, the description of a 1-hop expert module is "1 hop: directly connected neighboring nodes provide classification signals". The structural information of the central subgraph is described using an associative approach, such as "nodes..." Represents..., and its relationship with nodes , Connected…

[0062] Based on the constructed exclusive context information, task-specific prompts are generated. The task-specific prompts include the task description of the node-classified task and the candidate options selected by the expert modules. The candidate options are an expert set composed of all expert modules.

[0063] The task-specific prompts are fused with the prediction results output by each expert module to form complete prompt information that is input into the large language model. The large language model selects an expert module that is suitable for the node classification task based on the complete prompt information, and the selected expert module is taken as the optimal expert module.

[0064] More specifically, to guide the expert selection process, this embodiment designs task-specific prompts, requiring the large language model to select the most suitable expert for the node classification task based on the expert context. Specifically, the task-specific prompts not only include a task description but also provide a set of expert module selection options, in the form of, for example, "node..." Which expert should be selected for the classification task? Please provide the answer from the following experts: {answer}", where "{answer}" is a set of expert modules, i.e. Based on this, the contextual information of a node can be constructed into task-specific prompts. ,in The question text for the task. These are the independent prediction results from each expert module. Functions for building expert contexts A function to generate task-specific prompts.

[0065] Using the decision data from the expert module (the judgment results made by the expert module on nodes, i.e., the prediction results output by the expert module) as the supervision signal, the routing logic corresponding to the decision data of the expert module is transferred to the preset routing module through knowledge distillation. The routing module then generates routing allocation data based on node features. The routing module is a lightweight module obtained by transferring the routing decision logic from the pre-trained large language model through knowledge distillation.

[0066] Specifically, the prediction results output by each expert module are integrated into a prediction result set. The prediction result set, together with the complete prompt information (including complete task-specific prompt words), is input into the large language model, which then outputs the optimal expert module identifier that is suitable for the node classification task.

[0067] Configure a corresponding weight vector for each expert module, and initialize the weight vector corresponding to the expert module as a one-hot vector. The dimension of the one-hot vector corresponding to the expert module has a value of 1, and the other dimensions have a value of 0.

[0068] Based on the optimal expert module identifier, the weight vector is updated using a label smoothing strategy. The optimal expert module is assigned a smoothing coefficient weight value that takes values ​​within a preset range, and the remaining weight values ​​are equally assigned to the non-optimal expert modules.

[0069] More specifically, different expert modules first generate prediction results, which are then input into the large language model's discriminator to achieve context-aware routing, formally represented as: , For nodes The collection of all expert predictions. For the first An expert on the node The prediction results. Among them Initialized as a one-hot vector, and the first... The dimension is 1. Subsequently, the generated prediction results are input into the large language model, where the graph-supervised domain generalization model selects the most appropriate expert module. ( (Identifier for the optimal expert module) , In the hybrid expert framework, a label smoothing strategy is employed to assign lower weights to unselected expert modules. Accordingly, the weight vector... The update is as follows: , in This is the smoothing coefficient.

[0070] Then, based on the routing allocation data, the node features and prediction results are weighted and fused to obtain enhanced node features.

[0071] Specifically, from the graph structure data node set containing manually labeled nodes and unlabeled nodes without manually labeled nodes, a greedy node selection strategy is adopted to select the anchor node set with the goal of maximizing the number of activatable nodes in the anchor node set and satisfying the preset quantity constraints.

[0072] Performing large language model inference for each node incurs high computational overhead. Therefore, this embodiment distills a lightweight routing function from the large language model to achieve efficient expert selection. Inspired by the influence maximization (IM) problem in social networks, the aim is to select a set of anchor nodes. To maximize the utility of a large language model at low cost, it can be formally represented as: , in For anchor nodes The set of active nodes is defined as: , For anchor node to node The greatest influence, This is the activation threshold. For anchor nodes For nodes The impact on quality.

[0073] The influence of the anchor node on the target node is obtained by calculating and summing the entropy values ​​of the prediction results output by each expert module. The transition probability between nodes is obtained by sampling the path through random walk and counting the number of co-occurrences between nodes. The transition probability is combined with the influence to obtain the quality of the influence of the anchor node on the target node. The quality of the influence is compared with a preset activation threshold. Based on the comparison result, the set of activated nodes of the anchor node set is determined from the target nodes.

[0074] In practical implementation, this embodiment introduces the transition probability between nodes. (node arrive (transition probability), anchor node For nodes Impact on quality The calculation is as follows: , Entropy To influence the score, this embodiment employs a greedy node selection strategy to obtain the anchor node set. .

[0075] The routing function is obtained by distilling knowledge from the large language model based on the anchor node set. The distillation loss is calculated based on the weight vector of the anchor node and the output value of the routing function for splicing the multi-hop node representation of the anchor node. The routing function is iteratively optimized based on the distillation loss. The optimized routing function is used to assign routing weights to the node representations of each expert module. The final prediction result of the target node is obtained by fusing the weighted node representations.

[0076] Specifically, based on anchor node set Knowledge is distilled from a large language model to obtain a lightweight routing function. The distillation loss is defined as: , in For lightweight routing functions, The pre-defined cross-entropy loss function is used to measure the difference between the predicted distribution and the true / false labels. For anchor nodes The multi-hop representation concatenation result. It should be noted that, to effectively address domain offset, this embodiment performs knowledge redistillation from the large language model for each target domain before inference. The hybrid expert framework in [the framework name] has been updated to: .

[0077] Pseudo-label data is generated based on enhanced node features. Pseudo-label data with confidence scores higher than a preset threshold are selected to augment the original labeled data, resulting in augmented labeled data.

[0078] Specifically, for unlabeled nodes in graph structure data, sharpened pseudo-labels are generated based on enhanced node features; the entropy value of the prediction results output by each expert module for the unlabeled nodes is calculated, and a preset quantile of the entropy value is taken as a dynamic threshold. The entropy value output by the best expert module for the unlabeled nodes is compared with the dynamic threshold, and unlabeled nodes with entropy values ​​less than or equal to the dynamic threshold are selected to form a high-quality pseudo-labeled node set; based on the high-quality pseudo-labeled node set, an unsupervised classification loss is calculated according to the prediction results after the sharpened pseudo-labels and the node classifier are fused with the node representations of each expert module, and the unsupervised classification loss is used to fine-tune the graph semi-supervised domain generalization model.

[0079] More specifically, for each unlabeled node First, sharpened pseudo-labels are generated using a hybrid expert framework. This data is then used for node classification tasks. Specifically, this embodiment utilizes the entropy values ​​output by each expert. of (Preset activation threshold) quantiles, establish dynamic thresholds Defined as: , Based on this dynamic threshold, noisy pseudo-labels are filtered out, and only high-quality samples from the unlabeled data are retained, which can be formally represented as: , This is a high-quality set of pseudo-labeled nodes. These high-quality samples are used to fine-tune the graph semi-supervised domain generalization model, thereby improving its performance and enhancing its generalization ability under distribution shifts. The unsupervised classification loss is defined as... : .

[0080] Using the original labeled data and the augmented labeled data, a node classifier built based on enhanced node features is trained under supervision to obtain a graph semi-supervised domain generalization model.

[0081] Specifically, the distance correlation coefficients of the node representations of all nodes output by each expert module are calculated, and the expert diversity loss is obtained by summing the distance correlation coefficients. More specifically, to ensure that each expert can capture the different features of the central subgraph, this embodiment proposes a diversity objective function based on the distance correlation coefficient to promote the diversity of expert representations, as shown below: , in For all nodes Skip expert representation, This is the distance correlation coefficient function.

[0082] For labeled nodes in graph-structured data, a supervised classification loss is calculated based on the prediction results of the node classifier after fusing node representations from various expert modules, and the manually labeled tags of the labeled nodes. The supervised classification loss of labeled nodes, the unsupervised classification loss of unlabeled nodes, the distillation loss, and the expert diversity loss are weighted and fused to obtain the overall loss function; where the distillation loss and expert diversity loss are each configured with corresponding balance factors. More specifically... The calculation formula is as follows: , and These are the distance variance and distance covariance, respectively. They are used as the denominator in calculating the distance correlation coefficient, measuring the dispersion of the feature itself; and as the numerator in calculating the distance correlation coefficient, measuring the covariance between features.

[0083] Ultimately, the supervised classification loss with labeled nodes will be used. Knowledge distillation loss And expert diversity regularization loss Combining these, we obtain the overall objective function for LEGO: , in and This is a balancing factor used to control the weight of each loss term.

[0084] The graph semi-supervised domain generalization model is trained and optimized based on the overall loss function.

[0085] To verify the functionality of the graph semi-supervised domain generalization model constructed in this embodiment, the following experiments were conducted: The experiments were conducted using pre-stored ArnetMiner academic datasets, specifically selecting three subsets: ACMv9, Citationv1, and DBLPv7. These datasets are commonly used benchmarks for evaluating graph learning models and must be categorized into one of five predefined research areas: artificial intelligence, computer vision, databases, information security, and networking. ACMv9 represents the computer science field, covering ACM conferences / journals with a relatively balanced distribution across disciplines; Citationv1 comprises comprehensive academic citations, spanning multiple disciplines, and its distribution deviates significantly from ACMv9; DBLPv7, also in the computer science field, is the smallest and its distribution differs markedly from the other two. These datasets provide diverse graph structures and data distributions, making them suitable for evaluating the methods in this embodiment.

[0086] To evaluate the LEGO proposed in this embodiment, it is compared with several baseline methods, including GCN, GraphSAGE, GAT, GIN, CaNet, MARIO, LDAT, and SGDA. These baseline methods cover a variety of types, including: basic neural network models that process node features independently of graph structure; standard graph neural network architectures trained on the source domain by minimizing empirical risk, representing common graph representation learning techniques without explicit domain generalization mechanisms; and cutting-edge methods specifically designed for graph distribution out-generalization or domain adaptation, employing various strategies to address distribution shifts.

[0087] The experiments followed a pre-defined workflow, and all GNN-based baselines used data from the source domain. The model is trained on labeled data for a specified cross-domain generalization task. To evaluate performance compared to established graph representation learning techniques, several widely adopted architectures were selected. Among them, an MLP was used as the baseline model to quantify achievable performance without considering the inherent graph structure.

[0088] All experiments were conducted on a single NVIDIA RTX 3090 GPU (24GB of VRAM in this example). A hybrid expert framework was implemented using PyTorch and PyG (referred to as "Ours" in the table). A random partitioning strategy was employed to ensure diversity. Supervised training was performed using 5% of the node labels in the source domain. With a 5% source domain label ratio, ACMv9 was abbreviated as "A", Citationv1 as "C", and DBLPv7 as "D". C represents a graph with A as the source domain and C as the target domain; the representation is similar for other tasks. Table 1 below shows a comparison of model performance on six domain generalization tasks.

[0089] The MoE network consists of multiple expert modules and a router network (i.e., a pre-defined routing module). All experiments were conducted using five fixed random seeds to ensure reproducibility. The hybrid expert framework was trained using the Adam optimizer with a learning rate of 1e-3 and L2 weight decay of 2e-3. The hybrid expert framework used Qwen2.5-7B as the default LLM, and models such as Gemma-12B were also experimented with. The performance of the target domain nodes on the Micro F1 score (corresponding to "Micro" in Table 1) and the Macro F1 score (corresponding to "Macro" in Table 1) is reported.

[0090]

[0091] Table 1 - Performance Comparison of Models on Six Cross-Domain Generalization Tasks with a Source Domain Labeling Rate of 5% As shown in Table 1, LEGO can be used with only State-of-the-art performance was achieved in all six domain generalization tasks of the source tag. In the task, LEGO achieved a Micro F1 score of 75.7%, significantly outperforming SGDA (71.8%) and traditional graph neural networks.

[0092] LEGO's performance advantages stem from two aspects. Firstly, the guiding role of the large language model enhances the model's generalization ability by distilling expert module knowledge. Secondly, the hybrid expert framework enables adaptive topological information aggregation, dynamically selecting expert modules for nodes with high uncertainty, thus compensating for the lack of adaptability in baseline methods. Furthermore, LEGO's expert-enhanced pseudo-label strategy (i.e., generating high-quality pseudo-labels using more stable and discriminative predictions from multiple expert modules and the routing module in the MoE framework for unsupervised training) leverages expert module consistency and the confidence of the large language model to achieve effective knowledge transfer, demonstrating significant advantages in label-scarce scenarios. Extensive evaluation results on datasets with varying distributions fully demonstrate LEGO's robustness and superiority in cross-graph domain generalization tasks.

[0093] To evaluate the effectiveness of each LEGO component, the following ablation experiments were conducted in this embodiment. Four variant models were designed for comparison: w / o MoE (removing the hybrid expert framework and using a single expert module); w / o LLM (removing the large language model guidance and not performing knowledge distillation); w / o DIV (removing the diversity objective function of the expert module and having no representational difference constraints); and w / o HQ (removing the dynamic pseudo-labeling strategy of the perceptual expert module and using naive pseudo-labels). The results of the ablation experiments are shown in Table 2 below.

[0094] Table 2 - Ablation Experiment Results The ablation experiments show that removing the Hybrid Expert Framework (w / o MoE) causes the most significant drop, proving it to be a core component of LEGO. Disabling Large Language Model Guidance (w / o LLM) and Dynamic Pseudo-Label Strategy (w / o HQ) reduces Micro F1 / Macro F1 scores by 7%–13%, highlighting their importance in expert routing and unlabeled data utilization. Finally, removing Diversity Loss (w / o DIV) leads to a 2.9%–4.0% decrease in the performance of graph semi-supervised domain generalization models, validating its value in optimizing expert representations. These results fully demonstrate that each component of LEGO plays a crucial role in the overall performance of graph semi-supervised domain generalization models.

[0095] This embodiment also analyzes the parameter sensitivity of the graph semi-supervised domain generalization model.

[0096] Figure 2 The Micro F1 scores for different expert module types and the experimental results when the number of expert modules is increased proportionally, such as... Figure 2 As shown, Figure 2The left side shows the Micro F1 scores for different expert module types, demonstrating the impact of the number of expert module types on the performance of graph semi-supervised domain generalization models; Figure 2 The right side shows the experimental results as the number of expert modules increases proportionally, demonstrating the performance changes when the number of instances of each type of expert module increases proportionally. Figure 2 It can be seen that increasing the diversity of expert module types or the number of instances of each type of expert module can improve the performance of generalization models in graph semi-supervised domains, but the performance improvement will gradually decrease after exceeding the optimal value.

[0097] Figure 3 To evaluate A under different labeling rates C (as) Figure 3 (as shown in the left part) and A D (as in) Figure 3 (The right part shows the performance comparison results of the tasks.) Figure 3 The results show that higher labeling rates generally lead to better performance, but LEGO still exhibits excellent performance at low labeling rates, significantly outperforming the comparison methods, demonstrating its effectiveness in semi-supervised domain generalization tasks.

[0098] Figure 7 For A C (as) Figure 7 (as shown in the left part) and A D (as in) Figure 7 The right side shows a radar chart illustrating the performance of different types of LLMs on the task, demonstrating the impact of different large language models on LEGO performance. Figure 7 The results show that larger or more powerful language models generally provide better guidance, but the performance gap between medium-sized models (such as 7B, where 7B represents 7 billion parameters) and large-scale models (such as 12B / 14B) is not significant, while the performance of small-scale models (such as 3B / 4B) shows a significant decline.

[0099] In this embodiment, the impact of the number of expert modules on model performance was further investigated. For example... Figure 2 The two experimental scenarios are shown. Figure 2 The results on the left show the performance trend of graph-based semi-supervised generalization models as the diversity and number of expert modules increase. The results indicate that increasing the variety of specialized expert modules can positively improve Micro F1 within a certain range by providing stronger representation capabilities and processing stability, but the gain becomes insignificant beyond this range. Figure 2 The results on the right show the impact of increasing the number of instances for each type of expert module. This indicates that increasing the number of instances of the same type of expert module also helps improve performance, but beyond the optimal value, the gain gradually diminishes and performance may even decrease.

[0100] like Figure 3 As shown, by setting different label rates in the source domain, the performance and robustness of the graph semi-supervised domain generalization model in scenarios with scarce labeled data were tested. The experiment was conducted on A... C and A The results were performed on task D. The results show that a higher labeling rate generally leads to better performance, and the method in this embodiment still maintains strong performance under low labeling rate conditions, outperforming other comparative methods, demonstrating its effectiveness in generalization scenarios in the semi-supervised domain.

[0101] This embodiment analyzes the loss weight hyperparameter ( , , The effect of ) on Micro F1 score, such as Figure 8 As shown, Figure 8 (Left and right) A C ( Figure 8 (left part) and A D ( Figure 8 The heatmap for the task (right side) is shown. These hyperparameters control the balance between unsupervised classification loss, knowledge distillation loss, and expert diversity regularization loss, respectively. The results show that the optimal weights may vary depending on the specific domain generalization task. Generally, the performance of graph semi-supervised domain generalization models is sensitive to these weights and requires careful tuning to obtain optimal results. Figure 8 In the model, the performance is affected by the loss weight hyperparameters. and These two parameters are quite sensitive. They are used to balance the weights of unsupervised classification loss, knowledge distillation loss, and diversity regularization loss of expert modules. Their optimal values ​​will vary depending on the task and need to be tuned for specific tasks.

[0102] Figure 9 and Figure 10 It also shows the smoothing coefficient. With activation threshold Generalizing tasks in four domains ( Figure 9 A is on the left. C, Figure 9 A is on the right. D, Figure 10 C is on the left. A, Figure 10 C is on the right. The influence of parameter D). The results show that the performance of graph semi-supervised domain generalization models is affected by the combined effect of these two parameters, with optimal performance mostly occurring in... =0.8、 The value is around 0.05, indicating that proper smoothing of LLM guidance information and careful selection of pseudo-labels are both crucial to the model's performance.

[0103] This embodiment also explores the impact of different large language model types on performance, and the results are as follows: Figure 7 (Left, Right) A C and A As shown in Task D, larger or more powerful LLMs generally provide better guidance. Although the performance difference between 7B and 12B–14B models is not significant, smaller 3B–4B models show a significant performance drop of 3%–4%.

[0104] In this embodiment, in A In Task C, a pre-defined t-SNE (t-distributed Stochastic Neighbor Embedding, a high-dimensional data visualization method) is used to visualize the model representation of the graph semi-supervised domain generalization model, qualitatively evaluating the distillation effect guided by the large language model. Figure 4-6 As shown. The visualization includes three models: the full LEGO model (including large language model distillation), the LEGO variant (without large language model distillation), and the baseline method SGDA, which correspond to... Figure 4 , Figure 5 , Figure 6 . Figure 4-6 The color of a node corresponds to its true category. The results show that the guidance of the large language model can effectively optimize the gating mechanism of the hybrid expert framework and achieve better expert module allocation; while after removing the distillation component, the clustering effect of the node representation degrades significantly, further proving the core role of the large language model component in improving feature discriminability.

[0105] To carefully verify the individual contributions of key components in the method proposed in this embodiment, the performance of four variant models was evaluated on the same six-domain generalization task. The results and the degree of performance degradation are shown in Table 3 below:

[0106] Table 3 - Ablation Study Results on the Citationv1 Dataset Benchmark The results show that removing the hybrid expert framework leads to the most significant performance degradation across all tasks and metrics, with an average decrease of approximately 29.2% in Micro F1 scores and an average decrease of 34.5% in Macro F1 scores. For example, in A... On the C task, the MicroF1 score plummeted by 32.4 percentage points. In particular, after removing the MoE architecture, all other expert-related mechanisms failed, and the graph semi-supervised domain generalization model degenerated into a simple MLP-based approach. This significant performance drop demonstrates that MoE's ability to leverage specialized experts and adaptively fuse their decisions is key to the success of the graph semi-supervised domain generalization model, far outperforming single-encoder structures in these challenging domain generalization scenarios.

[0107] Removing the LLM-based expert module selection distillation mechanism results in a significant performance degradation, with an average decrease of approximately 7.3% in Micro F1 scores and 12.9% in Macro F1 scores. This degradation is evident in D... This is particularly evident in tasks of the A level (Micro F1 score drops by 13.5 percentage points). This indicates that the semantic guidance of LLM plays a crucial role in optimizing expert routing in MoE, especially for uncertain nodes. Without such external high-level semantic guidance, the autonomous gating mechanism of MoE is significantly less effective in assigning adaptive expert modules to nodes to achieve optimal generalization.

[0108] Removing the diversity loss within the MoE results in a noticeable but relatively mild performance degradation, with an average decrease of 2.9% in the Micro F1 score and 4.0% in the Macro F1 score. Although small, this consistent decline across tasks indicates that the regularization term plays a crucial role in optimizing the representations learned by each expert module. By encouraging consistency within expert modules and clearer specialization, diversified neighborhood exploration can also prevent the routing network from getting trapped in local optima.

[0109] Removing the expert-aware dynamic pseudo-label strategy in this embodiment also leads to a significant performance drop, with an average decrease of 7.0% in Micro F1 scores and 11.6% in Macro F1 scores. This highlights the importance of this refined semi-supervised learning component, which can selectively utilize unlabeled source domain data. By combining expert module consistency with LLM information to filter reliable pseudo-labels, this invention can effectively learn using only 5% labeled data and alleviate the error propagation problem common in traditional naive pseudo-labeling methods.

[0110] In this embodiment, Replacing it with the original graph adjacency matrix results in an overall performance decrease of approximately 1-2%. This is because, without this model design that is highly sensitive to topology, the differences in feature extraction among heterogeneous experts are weakened, making it more difficult for the routing network to assign nodes to appropriate expert modules.

[0111] This embodiment also includes an LLM-guided graph expert routing semi-supervised domain generalization system that uses an LLM-guided graph expert routing semi-supervised domain generalization method.

[0112] The above are merely embodiments of the present invention. The invention is not limited to the fields covered by these embodiments. Commonly known structures and characteristics in the solutions are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are able to access all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A semi-supervised domain generalization method for graph expert routing based on LLM guidance, characterized in that, Includes the following steps: Obtain graph structure data containing manually labeled nodes and use the graph structure data as the original labeled data; for the target node in the graph structure data, extract the central subgraph data corresponding to the target node according to the preset hop count range, input the central subgraph data into the preset expert module corresponding to the hop count, and perform feature extraction and prediction calculation on the central subgraph data through the expert module to obtain the node features and prediction results output by the expert module; The structural information, node connection relationships, and prediction results contained in the central subgraph data are converted into prompt text. The prompt text is then input into a pre-trained large language model. The adaptability of the expert module is evaluated by the large language model based on zero-shot reasoning ability, and the expert decision data output by the large language model is obtained. Using expert decision data as a monitoring signal, the routing logic corresponding to the expert decision data is migrated to the preset routing module through knowledge distillation. The routing module then generates routing allocation data based on node characteristics. The node features are weighted and fused with the prediction results based on the routing allocation data to obtain enhanced node features. Pseudo-label data is generated based on enhanced node features. Pseudo-label data with confidence scores higher than a preset threshold are selected to augment the original labeled data, resulting in augmented labeled data. Using the original labeled data and the augmented labeled data, a node classifier built based on enhanced node features is trained under supervision to obtain a graph semi-supervised domain generalization model.

2. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 1, characterized in that: Preset expert modules corresponding to different hop counts are used to extract structural context information at different levels. The expert modules generate corresponding node features and prediction results through mutually independent information aggregation methods, and realize information fusion based on the structural context corresponding to the node.

3. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 2, characterized in that: Each expert module constructs a correlation metric between nodes based on the number of times they co-occur. The obtained correlation metric is used as the adjacency relationship of the corresponding central subgraph data. A graph neural network with the same network structure and independent parameters is used to extract node representations from the central subgraph data. The extracted node representations are used as enhanced node features. The routing module assigns weights to the node representations of different expert modules, and then the node classifier fuses them to obtain the final prediction result. The node classifier is a learnable classifier.

4. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 2, characterized in that: For each expert module, a dedicated context information is constructed for the central subgraph data of the target node. The dedicated context information is transformed based on the structural context information of the central subgraph data of the corresponding hop count extracted by each expert module. It includes the guidance description of expert module selection, the attribute description of the hop count neighborhood of each expert module, and the node connection relationship description of the central subgraph data. Expert modules with different hop counts are matched with the dedicated attribute description of the corresponding hop count neighborhood. The node connection relationship of the central subgraph data is expressed in an associative manner.

5. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 4, characterized in that: Based on the constructed exclusive context information, task-specific prompts are generated. The task-specific prompts include the task description of the node-classified task and the candidate options selected by the expert modules. The candidate options are an expert set composed of all expert modules. The task-specific prompts are fused with the prediction results output by each expert module to form complete prompt information that is input into the large language model. The large language model selects an expert module that is suitable for the node classification task based on the complete prompt information, and the selected expert module is taken as the optimal expert module.

6. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 5, characterized in that: The prediction results output by each expert module are integrated into a prediction result set. The prediction result set and the complete prompt information are then input into the large language model, which outputs the optimal expert module identifier that is suitable for the node classification task. Configure a corresponding weight vector for each expert module, and initialize the weight vector corresponding to the expert module as a one-hot vector. The dimension of the one-hot vector corresponding to the expert module is 1, and the other dimensions are 0. Based on the optimal expert module identifier, the weight vector is updated using a label smoothing strategy. The optimal expert module is assigned a smoothing coefficient weight value that takes values ​​within a preset range, and the remaining weight values ​​are equally assigned to the non-optimal expert modules.

7. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 6, characterized in that: From the graph structure data node set containing manually labeled nodes and unlabeled nodes without manually labeled nodes, a greedy node selection strategy is adopted to select the anchor node set with the goal of maximizing the number of activatable nodes in the anchor node set and satisfying the preset quantity constraints. The influence of the anchor node on the target node is obtained by calculating and summing the entropy values ​​of the prediction results output by each expert module. The transition probability between nodes is obtained by sampling the path through random walk and counting the number of co-occurrences between nodes. The transition probability is combined with the influence to obtain the quality of the influence of the anchor node on the target node. The quality of the influence is compared with a preset activation threshold. Based on the comparison result, the set of activated nodes of the anchor node set is determined from the target nodes. The routing function is obtained by distilling knowledge from the large language model based on the anchor node set. The distillation loss is calculated based on the weight vector of the anchor node and the output value of the routing function for splicing the multi-hop node representation of the anchor node. The routing function is iteratively optimized based on the distillation loss. The optimized routing function is used to assign routing weights to the node representations of each expert module. The final prediction result of the target node is obtained by fusing the weighted node representations.

8. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 7, characterized in that: For unlabeled nodes in graph structure data, sharpened pseudo-labels are generated based on enhanced node features; Calculate the entropy value of the prediction results output by each expert module for the unlabeled nodes, take the preset quantile of the entropy value as a dynamic threshold, compare the entropy value output by the best expert module for the unlabeled nodes with the dynamic threshold, and select unlabeled nodes with entropy values ​​less than or equal to the dynamic threshold to form a high-quality pseudo-labeled node set; based on the high-quality pseudo-labeled node set, calculate the unsupervised classification loss according to the prediction results after the sharpened pseudo-labels and the node classifier are fused with the node representations of each expert module, and use the unsupervised classification loss to fine-tune the graph semi-supervised domain generalization model.

9. The LLM-guided semi-supervised domain generalization method for graph expert routing according to claim 7, characterized in that: Calculate the distance correlation coefficient of the node representation of all nodes output by each expert module, and sum the distance correlation coefficients to obtain the expert diversity loss; For labeled nodes in graph structure data, the supervised classification loss is calculated based on the prediction results of the node classifier after fusing the node representations of each expert module, and the manually labeled labels of the labeled nodes. The supervised classification loss of labeled nodes, the unsupervised classification loss of unlabeled nodes, the distillation loss and the expert diversity loss are weighted and fused to obtain the overall loss function. The distillation loss and the expert diversity loss are respectively configured with corresponding balance factors. The graph semi-supervised domain generalization model is trained and optimized based on the overall loss function.

10. A semi-supervised domain generalization system for graph expert routing based on LLM guidance, characterized in that, The LLM-guided semi-supervised domain generalization method for graph expert routing, as described in any one of claims 1-9, was used.

Images