A personalized federated graph learning method and system suitable for cross-domain graph classification

Abstract

The application discloses a personalized federated graph learning method and system suitable for cross-domain graph classification, and general graph spectrum knowledge is shared in view of the influence of structural heterogeneity in global cooperation, the general spectrum knowledge is shared in a global spectrum encoder including a global feature value encoder and a global filtering encoder, and customers benefit from the cooperation; wherein, other components representing non-general knowledge are reserved locally, each customer customizes personalized graph convolution for its own graph characteristics, and negative influence of spectrum bias is avoided; in view of the influence of structural heterogeneity in local application, a learnable preference is configured for each customer, a personalized preference module is used to cooperate with the work of the global spectrum encoder, personalized graph preference adjustment is performed, and the personalized graph preference adjustment is adapted to the unique graph structure of each customer; wherein, a regularization term is used to limit the personalized preference module to focus on local preference, and an over-reliance problem caused by the regularization term is solved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of graph learning and data analysis, and relates to a personalized federated graph learning method and system, in particular to a personalized federated graph learning method and system suitable for cross-domain graph classification. BACKGROUND

[0002] Graph neural networks (GNNs) have shown their superiority in modeling graph data. However, a large amount of graph data is actually generated by edge devices, which brings privacy problems and challenges of data silos. To solve these difficulties, federated learning has recently been applied to graph learning [Document 2]. It allows models of individual clients to collaborate without sharing local data, and makes federated graph learning a promising direction. However, the problem of non-independent and identically distributed is still a major challenge in federated graph learning, as the graph data of different clients may vary greatly. In this case, a single global model is difficult to adapt well to each client's local data and achieve good performance. To solve these challenges, personalized federated graph learning emerges, providing each client with a customized GNN to achieve the best local performance.

[0003] However, personalized federated graph learning still faces major challenges from data heterogeneity, especially in cross-domain tasks [Document 1][Document 3]. For example, collaboration between clients belonging to social networks and molecular structures, respectively. Existing methods inevitably face two major difficulties brought by structural heterogeneity:

[0004] 1. Due to structural heterogeneity in global collaboration, federated learning collaboration is difficult to share general knowledge to benefit each client, resulting in traditional methods even worse than local training.

[0005] 2. Due to structural heterogeneity in local application, models under global collaboration are difficult to adapt to each client's unique graph data and graph structure, thus tending to capture inappropriate features.

[0006] The basic training and testing steps of conventional personalized federated graph learning methods can be similarly described as follows:

[0007] 1. Data preparation: Collect graph data from one or more domains and preprocess it.

[0008] 2. Model design: Build a neural network model suitable for graph data.

[0009] 3. Distributed training: Train the model on different clients, each using its local data.

[0010] 4. Model aggregation: Aggregate model updates from different clients into a global model.

[0011] 5. Iterative optimization: Repeat the distributed training and model aggregation steps until the model performance reaches the expectation.

[0012] 6. Test evaluation: Evaluate the local model accuracy using each client's own test graph dataset, and take the mean of the test accuracy of each client.

[0013] Cross-domain federated graph collaboration brings great structural heterogeneity, which hinders effective collaboration between clients. To solve this problem, the current work adopts structural similarity clustering or suggests sharing structural encoding knowledge to mitigate the negative impact of heterogeneity. However, these strategies have two significant drawbacks. First, from the global impact of structural heterogeneity, existing methods ignore cross-domain structural heterogeneity. Therefore, it cannot be guaranteed that structural information is equivalent to general knowledge. Therefore, sharing structural encoding information inevitably causes clients to learn non-general structural knowledge from other clients, resulting in knowledge conflicts and unsatisfactory collaboration.

[0014] Second, from the local impact of structural heterogeneity, due to the message passing nature of GNNs, different graph structures of different clients require different message passing schemes. Therefore, when applying the scheme provided by the aggregated GNN to different graph structures of different clients, the scheme shows bias, leading to feature bias. Since this problem fundamentally stems from the uniqueness of the client dataset in this case, these biases are attributed to the diverse preferences of different clients. None of the current methods can explicitly meet the unique preferences of each client's graph structure.

[0015] In summary, in order to simultaneously address the impact of structural heterogeneity on global collaboration and local application, how to achieve the sharing of general knowledge while designing personalized plans to meet the preferences of different datasets from each client in cross-domain collaboration is a current problem that needs to be solved.

[0016] [Document 1] Tan Y, Liu Y, Long G, et al. Federated learning on non-iid graphs via structural knowledge sharing [C] / / Proceedings of the AAAI conference on artificial intelligence. 2023, 37(8): 9953-9961.

[0017] [Document 2] Huang W, Wan G, Ye M, et al. Federated graph semantic and structural learning [C] / / Proc. Int. Joint Conf. Artif. Intell. 2023:3830-3838.

[0018] [Document 3] Xie H, Ma J, Xiong L, Yang C. Federated graph classification over non-iid graphs. Advances in Neural Information Processing Systems. 2021 Dec 6;34:18839-52. SUMMARY

[0019] In order to solve the influence of cross-domain structural heterogeneity on both local and global federated graph learning in the background art, the present application provides a personalized federated graph learning method and system suitable for cross-domain graph classification.

[0020] The technical solution of the method of the present application is: a personalized federated graph learning method suitable for cross-domain graph classification, in order to address the influence of structural heterogeneity in global cooperation, general graph spectral knowledge is shared in the global spectral encoder including a global feature value encoder and a global filter encoder, so that customers benefit from cooperation; wherein the other components representing non-universal knowledge are retained locally, and each customer customizes personalized graph convolution for its own graph characteristics to avoid the negative effects of spectral bias;

[0021] In order to address the influence of structural heterogeneity in local application, a learnable preference is configured for each customer, and a personalized preference module is used to adjust personalized graph preferences by cooperating with the work of the global spectral encoder, so that it adapts to the unique graph structure of each customer; wherein a regularization term is used to limit the personalized preference module to focus on local preferences and solve the problem of excessive dependence caused thereby.

[0022] As a preferred embodiment, the general graph spectral domain knowledge sharing is implemented by the following steps:

[0023] Step 1: Map the eigenvalues of all graphs in all clients from scalars to meaningful vectors, and concatenate them as initial input;

[0024] Step 2: Client i uploads its local spectral encoder weights to aggregate into a global spectral encoder to share general spectral coding knowledge, and uses the direct average of the local spectral encoder weights of all clients to achieve fair cooperation and cross-dataset knowledge sharing.

[0025] As preferred, in step 1, the feature values are mapped from the scalar to the meaning vector by φ e (θ e ; λ, j): R 1 → R k The feature values are mapped from the scalar to the meaning vector;

[0026]

[0027] where θ e represents the parameters of the feature value encoder φ e , λ represents a vector composed of all feature values in the graph, β is used to control the importance of λ; k represents the dimension of the mapped vector; j represents the use of the sine and cosine trigonometric functions when even and odd, respectively; c is used to keep the feature values within a predetermined appropriate value range to distinguish different feature values of the trigonometric function; R represents the real number set;

[0028] Subsequently, the n feature values and their trigonometric function mapped encodings are spliced to obtain a k+1 dimensional vector, and then transformed into a k dimensional vector using the feature value encoder φ e to complete the feature value encoding process, denoted as X = [φ e (θ e ; λ1, j),..., φ e (θ e ; λ n , j)] T ∈ R n×(k+1) , where n represents the number of nodes in the client graph.

[0029] As preferred, in step 2, a multi-head attention module is first used to stack several attention mechanism modules to process the feature value vector, capture the spectral knowledge and relationship between the feature value vectors, and use the output of the attention mechanism module for the local spectral decoder ψ d to learn new feature values λ m from the expressive representation of the spectrum:

[0030]

[0031] where Q, K, V, correspond to the feature matrices of the query, key, and value components in the attention mechanism and parameters Q, K, V, which together constitute the attention mechanism module, λ m ∈ R n×1 is the feature value filtered by the mth attention head; R represents the real number set;

[0032] The client i uploads its local eigenvalue encoder weights to share the common spectral coding knowledge, the implementation process is: at the beginning of the t-th iteration, t≥0, the central server distributes its global eigenvalue encoder weights to update the local eigenvalue encoder weights, then the client i updates its local eigenvalue encoder weights using its dataset Di and sends to the central server, where represents the local eigenvalue encoder parameters of the client i in the t-th iteration, the central server aggregates the local encoder parameter updates received from each client and modifies the global encoder parameters to obtain the initial global parameters in the t+1-th iteration

[0033]

[0034] Where, the client number i∈[1..N]; N represents the total number of clients.

[0035] As preferred, each client customizes a personalized graph convolution for its own graph characteristics, the implementation process is:

[0036] After obtaining M eigenvalues filtered by attention heads, a local filter encoder φ f (θ f ;B)(R M+1 →R k ) is used to construct the basis of personalized graph convolution; where θ f is the parameter of the local filter encoder, B is the coordinate basis of graph spectral transformation, and k is the hidden layer dimension, that is, φ f is input first and then

[0037]

[0038] Λ m =diag(λ m );

[0039] Where, the original basis B m ∈R n×n , n represents the number of nodes in the client graph, which is equivalent to the number of eigenvalues; the original basis B m is constructed by the eigenvector U and eigenvalue λ m of the original graph, where the calculation process of the eigenvector U is LU=λU, λ represents a vector composed of all eigenvalues in the graph, and L is the Laplacian matrix of the graph, which is obtained by L=D-A, where D is the degree matrix of the graph, and A is the adjacency matrix;

[0040] The client i uploads the local filter encoder parameters θf The global filter encoder is aggregated and the local filter encoder is updated with the global filter encoder weights to share the general spectral coding knowledge; the sharing of the local filter encoder and the global filter encoder constitutes a general spectral coding knowledge sharing strategy;

[0041] In local training, after passing through the local filter encoder, the personalized graph convolution Where x v is the representation of node v from the previous layer, x′ v represents the output of the current layer, θ Conv refers to the parameters of the entire graph convolution, and σ refers to the activation function. represents the filter base finally used for graph convolution.

[0042] The representation of all nodes in the graph is aggregated through an average pooling layer to form the overall feature representation of the graph:

[0043]

[0044] Where h j is the average value of all node features in the graph , that is, the graph feature; V refers to the total number of nodes in the graph . N refers to the number of graphs included in the graph dataset owned by the client i.

[0045] As a preferred, the personalized graph preference adjustment is performed by setting a personalized preference module P(δ) to adjust the features extracted by each client i to explicitly meet its special needs and unique graph structure preferences.

[0046] Based on the entire graph neural network model of each client, there are three parts, that is, the local feature extractor composed of all network layers except the last layer classification head The classification head CC(θ c , h i ) of the last layer and the personalized preference module P(δ) obtain h′ i = h i + δ as the final feature suitable for meeting the preferences of the client for the graph by integrating h = F(θ f ; x i ) with the preference adjustment δ; where θ F , x i are the parameters of the local feature extractor F and the input; θ f is the parameter of the local filter encoder; h i is the output of the feature extractor, which is the average value of all node features in the graph, that is, the graph feature; θ c is the parameter of the local filter encoder; h i is the output of the feature extractor, which is the average value of all node features in the graph, that is, the graph feature; θ ch is the classification head C parameter and input; δ is the parameter of the preference module, whose dimension is equal to the graph feature representation h output by the feature extractor i .

[0047] As a preference, the regularization term is used to limit the personalized preference module to focus on local preferences; the specific implementation process is:

[0048] The mean value of the local graph feature of each round is dynamically calculated in each batch training process:

[0049]

[0050] wherein, and represent the local average graph feature of the last batch and the current batch, that is, the average value of each dimension of the graph feature of each batch; m is a set hyperparameter, used to balance the importance of the current batch and the previous batch;

[0051] After local training, the client uploads the mean value of its local graph feature to the server for global consensus calculation, wherein direct average is used to aggregate the global consensus:

[0052]

[0053] wherein, N represents the number of training samples of client i, refers to the global average consensus calculated from all samples of all clients;

[0054] The client uses the adjusted feature h ′ i to perform classification z i ′ =C(θ c ,h i ′ ), using mean square error MSE to measure the distance between the local graph feature mean value and the global graph mean value obtained from the last round, and after the mean square error MSE regularization term, the local loss of client i is:

[0055]

[0056] wherein, y i , represent the label and graph data of the graph data respectively, and τ is a hyperparameter for controlling the proportion of the regularization term in the loss.

[0057] The technical scheme of the system of the present application is: a personalized federated graph learning system suitable for cross-domain graph classification, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the personalized federated graph learning method suitable for cross-domain graph classification when executing the program.

[0058] Compared with the prior art, the present application has the following advantages:

[0059] (1) The present application first reveals the structural heterogeneity between cross-domain graph datasets from the spectral perspective. Specifically, the present application visualizes the algebraic connectivity and eigenvalue distribution and finds that there are significant spectral differences between datasets in different domains. On the contrary, the spectral characteristics within the same domain are more similar. This shows that when training using existing algorithms, the cooperation of federated graph learning is actually sharing non-universal spectral knowledge, which reduces the private model capability of each client. Inspired by previous research on spectral GNNs, the present application designs universal spectral knowledge sharing to share universal spectral knowledge in the spectral encoder, so that clients can benefit from cooperation without knowledge conflict. In addition, the present application retains other components representing non-universal knowledge locally to solve the knowledge conflict problem. Therefore, each client can customize personalized graph convolution for its own graph characteristics, avoiding the negative effects of spectral bias.

[0060] (2) The present application designs appropriate personalized plans for each client graph structure locally. Specifically, based on the current universal spectral knowledge sharing method, the present application considers the message passing nature of GNNs. From the spectral perspective, the spectral encoder strongly influences the way messages are passed. Therefore, when applying the aggregated spectral encoder locally to different graph structures, they may deviate from the message passing scheme most suitable for the client. As a result, the client extracts inappropriate frequency information, resulting in inappropriate features. To meet the unique structural needs of each client, the present application first configures a learnable preference for each client and proposes personalized graph preference adjustment. These personalized preference modules adjust the extracted features to adapt to the unique graph structure of each client by participating in the work of the global spectral encoder. In addition, the present application introduces a regularization term to limit the preference module from focusing on local preferences to solve the problem of over-reliance caused thereby.

[0061] (3) The present application is easy to integrate into existing graph learning frameworks, providing an effective method for solving the problem of cross-domain graph classification personalized federated graph learning in the field of structural heterogeneity and declining cooperation effect. BRIEF DESCRIPTION OF DRAWINGS

[0062] The technical solutions of the present application are further illustrated below by using examples and specific embodiments. In addition, some drawings are also used in the process of explaining the technical solutions. For those skilled in the art, other drawings and the intention of the present application can also be obtained without creative labor based on these drawings.

[0063] Figure 1 The method principle of the embodiment of the present application is shown in the following. EMBODIMENT

[0064] In order to facilitate those skilled in the art to understand and implement the present application, the present application is further described in detail below in combination with drawings and examples. It should be understood that the examples described herein are only used to illustrate and explain the present application, and are not used to limit the present application.

[0065] The embodiment shares the spectrum encoder in the federal scenario, which includes a global feature value encoder and a global filter encoder, collectively referred to as a global spectrum encoder; the spectrum decoder is not shared, and only a local spectrum decoder exists. Each client has its own local spectrum encoder, and when the server performs global parameter aggregation in each round, the aggregation produces a global spectrum encoder called, which is then distributed to the client to update the local spectrum encoder.

[0066] See Figure 1 The personalized federated graph learning method for cross-domain graph classification provided by the embodiment adopts general graph spectrum domain knowledge sharing. In view of the influence of structural heterogeneity in global cooperation, the general spectrum knowledge is shared in the global spectrum encoder including the global feature value encoder and the global filter encoder, so that the clients benefit from the cooperation; wherein the other components representing non-general knowledge are reserved locally, and each client customizes personalized graph convolution for its own graph characteristics to avoid the negative influence of spectrum bias.

[0067] In one embodiment, the general graph spectrum domain knowledge sharing includes the following steps:

[0068] Step A1: Map the feature values of all graphs in all clients from scalars to meaning vectors, and splice them as initial inputs;

[0069] In one embodiment, the graph-level task is performed, and each graph is a separate sample, that is, each client internally has many independent graphs, and the feature vector of each graph data is independently calculated by spectral decomposition. The embodiment directly uses λ to explain the graph data processing process.

[0070] In each client, φ e (θ e ; λ, j): R 1 → Rk map the eigenvalues from scalar to meaning vector;

[0071]

[0072] where θ e is the parameter of eigenvalue encoder φ e , λ represents a vector composed of all eigenvalues in the graph, β is used to control the importance of λ; k represents the dimension of the mapped vector; j represents the use of sin and cos trigonometric functions mapping when even and odd respectively; c is used to keep the eigenvalues within a predetermined appropriate value range to distinguish different eigenvalues of trigonometric functions; R represents the real number set;

[0073] Subsequently, the n eigenvalues and their trigonometric function mapped encodings are spliced to obtain a k+1 dimensional vector, and then the eigenvalue encoder φ e is used to retransform it into a k dimensional vector to complete the eigenvalue encoding process, denoted as X = [φ e (θ e ; λ1, j),..., φ e (θ e ; λ n , j)] T ∈ R n×(k+1) , where n represents the number of nodes in the client graph.

[0074] Step A2: The client i uploads its local spectral encoder weights to aggregate into a global spectral encoder to share general spectral encoding knowledge, and uses direct averaging of local spectral encoder weights of all clients to achieve fair cooperation and cross-dataset knowledge sharing.

[0075] In an embodiment, a multi-head attention module is first used to process the eigenvalue vectors by stacking several attention mechanism modules to capture the spectral knowledge and relationship between the eigenvalue vectors, and the output of the attention mechanism module is used for the local spectral decoder ψ d to learn new eigenvalues λ m from the expressive representation of the spectrum:

[0076]

[0077] where Q, K, V, correspond to the feature matrices of the query, key, and value three components in the attention mechanism and parameters Q, K, V, which together constitute the attention mechanism module, λ m ∈ R n×1 is the eigenvalue filtered by the mth attention head; R represents the real number set;

[0078] The client i uploads its local eigenvalue encoder weights to share the common spectral coding knowledge, and the specific implementation process is: at the beginning of the t-th iteration, t≥0, the central server distributes its global eigenvalue encoder weights to update the local eigenvalue encoder weights, and then the client i uses its dataset D i to update its local eigenvalue encoder weights and sends to the central server, where represents the local eigenvalue encoder parameters of the client i in the t-th iteration, and the central server aggregates the local encoder parameter updates received from each client and modifies the global encoder parameters to obtain the initial global parameters in the t+1-th iteration

[0079]

[0080] Where, the client number i∈[1..N]; N represents the total number of clients.

[0081] Meanwhile, in view of the influence of structural heterogeneity in local application, this embodiment configures a learnable preference for each client, and adopts a personalized preference module to adjust the personalized graph preference by cooperating with the work of the global spectral encoder, so that it adapts to the unique graph structure of each client; wherein, a regularization term is adopted to limit the personalized preference module to focus on local preference, and solve the problem of excessive dependence caused thereby.

[0082] In an embodiment, each client customizes a personalized graph convolution for its own graph characteristics, and the specific implementation process is:

[0083] After obtaining the eigenvalues filtered by the M attention heads, a local filter encoder φ f (θ f ;B)(R M+1 →R k ) is used to construct the basis of the personalized graph convolution; wherein, θ f is the parameter of the local filter encoder, B is the coordinate basis of the graph spectral transformation, and k is the hidden layer dimension, that is, φ f is input first, and then

[0084]

[0085] Λ m =diag(λ m );

[0086] Where, the original basis B m ∈R n×n, n represents the number of nodes in the client graph, which is equivalent to the number of eigenvalues; the original base B m is constructed from the eigenvectors U and eigenvalues λ of the original graph m , where the calculation process of the eigenvectors U is LU = λU, λ represents a vector composed of all eigenvalues in the graph, L is the Laplacian matrix of the graph, which is obtained by L = D - A, where D is the degree matrix of the graph, and A is the adjacency matrix;

[0087] The client i uploads the local filter encoder parameters θ f The global spectral encoder is aggregated, and the local filter encoder is updated with the global filter encoder weight to share the general spectral coding knowledge; the uploading, aggregation, and delivery strategies of the two encoders (the local filter encoder and the global filter encoder) are the same as the sharing of the eigenvalue encoder, and the sharing of the two encoders (the local filter encoder and the global filter encoder) together constitutes the general spectral coding knowledge sharing strategy.

[0088] In local training, after passing through the local filter encoder, the personalized graph convolution is constructed, where x v is the representation of the node v from the previous layer, x′ v represents the output of the current layer, θ Conv represents the parameters of the entire graph convolution, and σ represents the activation function. represents the filter base used for the final graph convolution. The representations of all nodes in the graph are aggregated through an average pooling layer to form the overall feature representation of the graph:

[0089]

[0090] where h j is the average value of the features of all nodes in the graph , that is, the graph feature; V represents the total number of nodes in the graph . N represents the number of graphs included in the graph dataset owned by the client i.

[0091] In an embodiment, the personalized graph preference adjustment is performed by setting a personalized preference module P(δ) to adjust the features extracted by each client i to meet its special needs and unique graph structure preference.

[0092] Based on the entire graph neural network model of each client, there are three parts, that is, the local feature extractor composed of all network layers except the last layer classification head , the classification head CC(θ c , h i ) of the last layer, and the personalized preference module P(δ) obtain h′ by integrating h = F(θ f ; x i ) and the preference adjustment δi = h i + δ, as the final fit to satisfy the client's preference for the graph; where θ F , x i is the local feature extractor F parameter and input; θ f is the local filter encoder parameter; h i is the feature extractor output, which is the average of all node features in the graph, i.e., the graph feature; θ c , h is the classification head C parameter and input; δ is the preference module parameter, which has the same dimension as the graph feature representation h i .

[0093] In an embodiment, the regularization term is used to limit the personalized preference module to focus on local preferences; the implementation process is as follows:

[0094] In each batch of training process, the mean value of the local graph feature of each round is dynamically calculated:

[0095]

[0096] wherein, and represent the local average graph feature of the previous batch and the current batch, that is, the average value of each dimension of the graph feature of each batch; m is a set hyperparameter, used to balance the importance of the current batch and the previous batch;

[0097] After local training, the client uploads the mean value of its local graph feature to the server for global consensus calculation, wherein direct average is used to aggregate the global consensus:

[0098]

[0099] wherein, N represents the number of training samples of the client i, refers to the global average consensus calculated from all samples of all clients.

[0100] Specifically, for the client i, the adjusted feature h′ i is used to perform classification z′ i = C(θ c , h′ i ), and the local loss can now be represented as:

[0101]

[0102] wherein, y i , represent the label of the graph data and the graph data, respectively.

[0103] Subsequently, this embodiment uses the mean square error (MSE) to measure the distance between the local graph feature mean and the global graph mean obtained from the last round. This measurement serves as a regularization term to encourage the local graph feature mean to be closer to the global graph mean, thereby forcing the preference module to focus on the preference and solving the over-reliance problem. Through the regularization term of this embodiment, the client i local loss is:

[0104]

[0105] Where τ is a hyperparameter that controls the proportion of the regularization term in loss.

[0106] By implementing this regularization term, the conflict between global consensus extraction and client-specific preference adjustment is avoided, and the over-reliance problem considered previously is successfully avoided, forcing the preference module to focus on the client-specific preference.

[0107] This embodiment also provides a personalized federated graph learning system suitable for cross-domain graph classification, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the personalized federated graph learning method suitable for cross-domain graph classification when executing the program.

[0108] The following further illustrates the present application through specific experiments.

[0109] In order to verify the effectiveness of the present application, this embodiment uses 15 public graph classification datasets from four different domains, including small molecules (MUTAG, BZR, COX2, DHFR, PTC_MR, AIDS, NCI1), bioinformatics (PROTEIN, OHSU, Peking_1), social networks (IMDB-BINARY, IMDB-MULTI) and computer vision (Letter-low, Letter-high, Letter-med). Some of the datasets provide node features, and the graph labels are binary or multi-class. This embodiment creates six different non-independent and identically distributed (non-IID) settings, namely: (1) a cross-dataset setting based on seven small molecule datasets (SM); (2)-(6) cross-dataset and cross-domain settings based on datasets from two different domains (BIO-SM, SM-CV) and three different domains (BIO-SM-SN, BIO-SN-CV, CHEM-SN-CV).

[0110] This embodiment compares the method of the present invention with several state-of-the-art federated learning methods for optimizing the personalized model for each client: (1) Local as the first baseline; (2) FedAvg; (3) FedProx, which addresses the heterogeneity problem in FL; (4) FedCP, a state-of-the-art pFL method; (5) FedSage; (6) GCFL; and (7) FedStar, three state-of-the-art FGL methods. Experimental results are shown in Tables 1 and 2 below:

[0111] Table 1

[0112] Method SM SM-BIO SM-CV SM-BIO-SN BIO-SN-CV SM-SN-CV Local 77.33±1.15 72.52±1.86 82.24±1.73 ]]> ​ 71.13±1.32 72.59±2.70 78.23±0.54 FedAvg 74.12±2.10 67.82±1.63 81.21±1.00 67.31±2.56 70.93±2.91 75.33±1.06 FedProx 69.35±3.36 67.27±4.17 70.02±2.27 63.89±4.33 69.32±1.75 67.15±2.25 FedCP 77.58±2.00 71.15±1.77 81.59±0.40 71.07±1.23 73.74±2.53 ]]> ​ 78.17±1.78 ]]> ​ FedSage 75.61±1.16 72.60±3.18 76.23±0.49 70.84±0.88 69.69±1.11 73.36±0.86 GCFL 77.71±1.53 72.05±2.20 72.64±0.71 70.43±1.39 67.91±2.15 71.79±0.21 Fedstar 78.63±2.11 ]]> ​ 72.71±1.22 ]]> ​ 78.84±1.07 72.60±2.45 69.51±3.24 75.94±0.40 FedSSP 79.62±2.23 73.66±2.34 84.29±0.68 72.37±2.18 ]]> ​ 75.07±2.70 79.12±1.23

[0113] This experiment demonstrates the federated graph classification results of various methods under six different non-independent identically distributed (non-IID) settings, including one cross-dataset setting (SM), two cross-bidomain settings (BIO-SM, SM-CV), and one cross-multidomain setting (BIO-SM-SN, BIO-SN-CV, SM-SN-CV). The final average test accuracy is summarized in the two tables above. The results show that the proposed method, FedSSP, outperforms all other baseline methods on five of the six settings. Notably, traditional federated learning algorithms such as FedAvg and FedProx do not outperform self-trained methods due to the strong cross-dataset and cross-domain non-IID challenges in this scenario. Correspondingly, algorithms designed specifically for pFL or pFGL scenarios, such as FedStar and FedCP, perform better here.

[0114] This invention innovates in two aspects: seeking effective global collaboration and appropriate local application. Regarding global collaboration, inspired by previous work in the graph domain, this invention employs a shared eigenvalue encoder and filter encoder to promote the sharing of universal spectral knowledge, overcoming cross-domain spectral heterogeneity while also addressing cross-domain structural heterogeneity. In terms of local application, this invention designs a learnable graph feature preference adjustment module for each client. It successfully achieves feature adjustment to meet the graph preferences of each client by utilizing a combination of personalized preference modules and regularization terms. Furthermore, this invention performs well in various cross-domain or cross-data collaboration scenarios, effectively handling diverse scenarios and enhancing the model's robustness in practical applications. This invention is easily integrated into existing graph learning frameworks, providing an effective method to address the problems of domain structural heterogeneity and decreased collaboration performance in personalized federated graph learning for cross-domain graph classification.

[0115] It should be understood that the above-described embodiments are part of the embodiments of the present application, but not all the embodiments. In addition, the technical features of each embodiment or individual embodiments provided by the present application can be combined with each other to form a feasible technical solution, and such combination is not restricted by the order of steps and / or structure mode, but must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unfeasible, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection required by the present application.

[0116] It should be understood that the above description of the preferred embodiments is more detailed, and therefore should not be considered as limiting the scope of patent protection of the present application. Those skilled in the art can make substitutions or modifications under the inspiration of the present application without departing from the scope of protection claimed by the present application, and all fall within the scope of protection of the present application. The scope of protection claimed by the present application shall be subject to the appended claims.

Claims

1. A personalized federated graph learning method suitable for cross-domain graph classification, characterized in that: For the influence of structural heterogeneity in global cooperation, general graph spectral domain knowledge sharing is adopted, and general spectral knowledge is shared in the global spectral encoder including global eigenvalue encoder and global filter encoder, so that customers benefit from cooperation; wherein, other components representing non-general knowledge are reserved locally, and each customer customizes personalized graph convolution for its own graph characteristics, avoiding the negative impact of spectral bias; For the influence of structural heterogeneity in local application, a learnable preference is configured for each customer, and personalized graph preference adjustment is performed by cooperating with the work of the global spectral encoder through the personalized preference module, so that it adapts to the unique graph structure of each customer; wherein, a regularization term is used to limit the personalized preference module to focus on local preference, and solve the problem of over-reliance caused thereby; The general graph spectral domain knowledge sharing comprises the following steps: Step 1: Map the eigenvalues of all graphs in all client terminals from scalar to meaningful vector, and splice them as initial input; wherein within each client the feature values are mapped from scalars to meaning vectors by mapping the feature values from scalars to meaning vectors; wherein, representative eigenvalue encoder parameters, denotes a vector consisting of all eigenvalues in the graph, for controlling importance; denotes the dimension of the mapped vector; denotes the use of the trigonometric functions sin and cos for even and odd, respectively; for keeping the eigenvalues within a predetermined range of appropriate values to distinguish different eigenvalues of the trigonometric functions; denotes the set of real numbers; Then, n The encodings obtained by mapping each eigenvalue and its trigonometric function are concatenated to obtain... 3D vector, then using an eigenvalue encoder Re-transformed into A dimensional vector completes the feature value encoding process, denoted as . ,in This indicates the number of nodes in the client graph; Step 2: Client Upload their local spectral encoder weights to aggregate into a global spectral encoder to share general spectral coding knowledge, using a direct average of local spectral encoder weights of all clients to achieve fair cooperation and cross-dataset knowledge sharing; Among them, first use multi-head attention module, stack several attention mechanism modules to process the feature value vector, capture the frequency spectrum knowledge and relationship between the feature value vectors, and use the output of the attention mechanism module for the local frequency spectrum decoder of the feature value decoding Learning new feature values from expressive representations of the frequency spectrum : wherein, the feature matrix corresponding to the query, key, and value components in the attention mechanism and parameters Q, K, and V, collectively constitute an attention mechanism module, is a feature value filtered by the first attention head; denotes the set of real numbers; The client Upload its local eigenvalue encoder weights to share general spectral coding knowledge. The specific implementation process is as follows: In the first... At the beginning of the next iteration, The central server distributes its global feature value encoder weights. To update the local feature encoder weights, then the client Use its dataset Update its local feature value encoder Weight and will Send to the central server, where Representing the Second iteration client The local feature value encoder parameters are obtained by the central server, which aggregates and receives updates to the local encoder parameters from each client and modifies the global encoder parameters to obtain the first value. Initial global parameters for the next iteration ; Wherein, the client number ; N represents the total number of clients; Each customer customizes personalized graph convolution for its own graph characteristics, and the specific implementation process is as follows: In obtaining After filtering the feature values ​​using an attention head, a local filter encoder is used. Construct a basis for personalized graph convolutions; where, These are the parameters for the local filter encoder. The coordinate basis for the graph spectrum transformation, For the hidden layer dimension, that is, the input is first... Later obtained = ; ； ； wherein the original basis , represents the number of nodes in the client graph, which is equivalent to the number of eigenvalues; the original basis is constructed by the eigenvectors and eigenvalues of the original graph, wherein the calculation process of the eigenvectors is , represents a vector composed of all eigenvalues in the graph, is the Laplacian matrix of the graph, obtained by , wherein is the degree matrix of the graph, is the adjacency matrix; Clients uploading local filter encoder parameters after each round of training aggregate a global filter encoder and update the local filter encoder with the global filter encoder weights to share general spectral coding knowledge; the sharing of the local filter encoder and the global filter encoder jointly constitute a general spectral coding knowledge sharing strategy; In local training, after the local filter encoder, a personalized graph convolution is constructed where is the representation of the node from the previous layer, denotes the output of the current layer, denotes the parameters of the entire graph convolution, denotes the activation function; denotes the filter base finally used for the graph convolution; The representation of all nodes in the graph is aggregated through an average pooling layer to form the overall feature representation of the graph: wherein is the graph characteristic of the graph is the average value of all node characteristics in the graph is the total number of nodes in the graph is the number of graph datasets owned by the client is the number of graphs 2. The personalized federated graph learning method suitable for cross-domain graph classification according to claim 1, wherein: The personalized graph preference adjustment is performed by setting a personalized preference module , which adjusts the extracted features of each client i to explicitly meet its special needs and unique graph structure preferences; Based on the entire graph neural network model for each client, it consists of three parts: a local feature extractor composed of all network layers except the last classification head layer. The last layer of classification headers And personalized preference module By With preference adjustment Integration , as the final feature suitable for satisfying the client's preference for graphs; among which, , For local feature extractor Parameters and inputs; These are the parameters for the local filter encoder; The output of the feature extractor is the average value of the features of all nodes in the graph, i.e., the graph features; For classification head Parameters and inputs; The parameters of the preference module are equal to the graph feature representation output by the feature extractor. .

3. The personalized federated graph learning method suitable for cross-domain graph classification of claim 2, wherein: The regularization term is used to limit the personalized preference module to focus on local preference; the specific implementation process is as follows: The mean value of the local graph features of each round is dynamically calculated in the training process of each batch: where, and denote the local average graph features of the previous batch and the current batch, i.e., the average value of each dimension of each batch graph feature; is a hyper-parameter set to balance the importance of the current batch and the previous batch. After local training, the clients upload the mean of their local graph features to the server for global consensus computation, where direct averaging is used to aggregate the global consensus: where N denotes the number of clients training samples of the client, refers to the global average consensus computed from all samples of all clients; Clients utilize adjusted features Classification is performed The distance between the local map feature mean and the global map mean obtained from the previous round is measured using the mean squared error, MSE. After the mean squared error, MSE, regularizer, the client The local loss is: ； wherein, respectively represent a label of graph data and graph data, is a hyperparameter to control the proportion of regularization term in loss.

4. A personalized federated graph learning system suitable for cross-domain graph classification, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the personalized federated graph learning method for cross-domain graph classification according to any one of claims 1 to 3.

Images