Heterogeneous federal map learning method and system

Abstract

The invention belongs to the technical field of machine learning, and relates to a heterogeneous federal map learning method and system. The method comprises the following steps: in each client, constructing a globally shared symbiotic space, and generating a unified target semantic prototype through a label propagation mechanism; generating a prototype distribution matrix, and uploading the prototype distribution matrix to a server side; generating a global prototype by aggregating the prototype distribution matrix of each client, and distributing the global prototype to each client; embedding and mapping the nodes into a hash bucket by using a hash function, generating aligned local hash codes and local anchor point embedding, and uploading the local hash codes and the local anchor point embedding to a server side; aggregating the local hash codes and the local anchor point embedding to obtain global hash codes and global anchor point embedding; global hash coding and global anchor point embedding are optimized through a graph auto-encoder and consistency constraint, and alignment of the concentrated graph is optimized. According to the invention, the flexibility and efficiency of federal map learning are improved, the data privacy and security are ensured, and the communication overhead is reduced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine learning, and particularly relates to a heterogeneous federated graph learning method and system. BACKGROUND

[0002] Graph representation learning as a powerful and effective method of graph data analysis has been widely studied and made significant progress in various application scenarios such as recommendation systems, citation network analysis, scene graph generation, etc. However, in the above scenarios, due to reasons such as user privacy protection and commercial competition, different institutions often have their own graph data and do not share with other institutions, limiting the scalability of the traditional centralized training method of graph representation learning. For example, in different e-commerce systems, the personal information and shopping information of each user are only known by the corresponding e-commerce institution, and other e-commerce institutions cannot obtain these additional interaction information.

[0003] Federated graph learning as a privacy-preserving distributed graph learning paradigm aims to solve the problems of distributed graph data discretization, distribution imbalance and graph heterogeneity under the federated learning framework. At present, a mainstream strategy is to train a neighbor generator locally before federated communication to predict the latent neighbor features, thereby alleviating the model negative transfer problem caused by the feature bias in the graph data between different clients. Some other researches try to design different personalized parameter aggregation strategies on the server side, use local topological knowledge to measure the correlation between clients, and achieve personalized biased aggregation of the model based on this. Although some progress has been made, it is still difficult to get rid of the traditional federated learning paradigm, that is, the server customizes the global model architecture, and each client updates the global model by transmitting parameters / gradient. The above strategy brings two problems: (1) Lack of special consideration of graph structure information in the client, and topological bias makes the learned node representation have large differences between different clients, making it difficult to train collaboratively. (2) Limited by the unified model architecture. SUMMARY

[0004] The technical problem to be solved by the application is the lack of special consideration of graph structure information in the client and the limitation of the unified model architecture, and a heterogeneous federated graph learning method and system are provided.

[0005] The content of the application includes: in a first aspect, the application provides a heterogeneous federated graph learning method, comprising:

[0006] In each client, based on the preliminary semantic prototype of each client, a globally shared co-occurrence space is constructed, and a unified target semantic prototype is generated through a label propagation mechanism;

[0007] Based on the unified target semantic prototype, a prototype allocation matrix is generated and uploaded to the server side;

[0008] On the server side, a global prototype is generated by aggregating the prototype allocation matrix of each client, and then the global prototype is distributed to each client.

[0009] Based on the global prototype, each client uses a hash function to embed nodes into hash buckets, generating aligned local hash codes and local anchor embeddings, and then uploads the aligned local hash codes and local anchor embeddings to the server.

[0010] On the server side, the aligned local hash codes and local anchor embeddings are aggregated to obtain global hash codes and global anchor embeddings, and then the global hash codes and global anchor embeddings are distributed to the clients.

[0011] In the client, the global hash encoding and the global anchor embedding are optimized through graph autoencoders and consistency constraints to complete the alignment of the optimized condensed graph.

[0012] Furthermore, in each client, a globally shared symbiotic space is constructed, and a unified target semantic prototype is generated through a tag propagation mechanism, including:

[0013] In each client, a preliminary semantic prototype is generated based on the topology information and the original characteristics of the nodes, and a globally shared symbiotic space is constructed based on the preliminary semantic prototype of each client.

[0014] The initial semantic prototypes of each client are merged through a tag propagation mechanism, and combined with the adjacency information of the nodes, and multiple rounds of iteration are performed to generate a unified target semantic prototype.

[0015] Furthermore, the label propagation mechanism includes:

[0016] Calculate the initial label for each node to obtain the label vector of the node;

[0017] Based on the initial label of each node, the label vector of each node is updated through multiple rounds of iteration using the label propagation mechanism to obtain the final label of each node;

[0018] After each iteration, the previous label is updated according to the final label of each node, so that the label of each node is consistent with the labels of its neighboring nodes, and the final label propagation result is obtained. The label propagation is a matrix that records the label information of all nodes. The final label propagation result includes the final label of each node and the updated label propagation matrix.

[0019] Furthermore, the step of generating a prototype assignment matrix based on the unified target semantic prototype and uploading it to the server includes:

[0020] Based on the relationship between the unified target semantic prototype and the initial semantic prototype of each client, the prototype assignment matrix of each client is calculated.

[0021] The prototype allocation matrix for each client is uploaded to the server.

[0022] Further, the step of mapping node embeddings to hash buckets using a hash function on each client based on the global prototype, generating aligned local hash codes and local anchor embeddings, and uploading the aligned local hash codes and local anchor embeddings to the server includes:

[0023] Each client receives the global prototype, calculates the node embedding based on Euclidean distance, maps the node embedding to a hash bucket using a hash function, and generates the local hash code. The calculation formula for mapping the node embedding to the hash bucket using the hash function is as follows:

[0024]

[0025] Among them, H m E represents the local hash code. m This indicates different node embeddings, where m represents the client's sequence number. Let represent the transpose of the global prototype, where 1 represents a column vector of all 1s and b represents the bias. This indicates a transpose operation, where r represents the hash bucket width;

[0026] The local hash code is deduplicated to obtain the global hash codebook;

[0027] Generate local anchor embeddings based on the unified target semantic prototype;

[0028] The local hash code and the local anchor embedding are aggregated using the global hash codebook to obtain aligned local hash code and local anchor embedding, and then the aligned local hash code and local anchor embedding are uploaded to the server.

[0029] Further, on the server side, the aligned local hash code and local anchor embedding are aggregated to obtain the global hash code and global anchor embedding, including:

[0030] On the server side, based on the global hash codebook, the aligned local hash codes are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local hash codes and the global hash codes to obtain the global hash codes. The optimization algorithm is any algorithm used to minimize the alignment error.

[0031] Based on the global hash codebook, the aligned local anchor embeddings are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local anchor embeddings and the global anchor embeddings to obtain the global anchor embeddings.

[0032] The global hash encoding and global anchor embedding are distributed to the client.

[0033] Furthermore, in the client-side, optimizing the global hash encoding and the global anchor embedding through graph autoencoders and consistency constraints to optimize the alignment of the condensed graph includes:

[0034] In each client, based on the distance between the local hash code and the global hash code, the first loss function of the hash code is minimized using a graph autoencoder model, so that the local hash code is aligned with the global hash code;

[0035] Based on the alignment of the local hash code and the global hash code, and based on the distance between the local anchor embedding and the global anchor embedding, the first loss function of the anchor embedding is minimized using the graph autoencoder model, so that the local anchor embedding and the global anchor embedding are aligned.

[0036] Based on the alignment of the local anchor embedding with the global anchor embedding, the local anchor embedding is optimized by constraining the second loss function of anchor embedding through structural consistency, so that the optimized local anchor embedding is consistent with the global anchor embedding, thus completing the alignment of the optimized condensation map.

[0037] Furthermore, the formula for calculating the first loss function of anchor point embedding is as follows:

[0038]

[0039] Among them, L s K represents the first loss function for anchor point embedding. m This represents the hash code number of the m-th client, where k represents a positive integer. This indicates the embedding of the k-th anchor point in the m-th client. This indicates the embedding of the k-th global anchor point.

[0040] Furthermore, the formula for calculating the anchor embedding second loss function is as follows:

[0041]

[0042] Among them, L a The anchor embedding second loss function, P m Anchor embedding on the local client, This indicates that the anchor point of the local client is embedded in the transpose matrix. This represents the condensed graph structure of the local client, σ(·) represents the sigmoid function, and F represents the matrix norm.

[0043] Secondly, the present invention provides a heterogeneous federated graph learning system, comprising:

[0044] A symbiotic module is constructed to build a globally shared symbiotic space in each client based on the initial semantic prototype of each client, and to generate a unified target semantic prototype through a label propagation mechanism.

[0045] The upload matrix module is used to generate a prototype allocation matrix based on the unified target semantic prototype and upload it to the server.

[0046] Generate a global module, which is used on the server side to generate a global prototype by aggregating the prototype allocation matrix of each client, and then distribute the global prototype to each client.

[0047] The embedding mapping module is used to embed nodes into hash buckets on each client based on the global prototype using a hash function, generate aligned local hash codes and local anchor embeddings, and upload the aligned local hash codes and local anchor embeddings to the server.

[0048] Anchor embedding module is used on the server side to aggregate the aligned local hash code and local anchor embedding respectively to obtain global hash code and global anchor embedding, and distribute the global hash code and global anchor embedding to the client;

[0049] An optimization alignment module is used in the client to optimize the alignment of the condensed graph by optimizing the global hash encoding and the global anchor embedding through graph autoencoders and consistency constraints.

[0050] The beneficial effects of this invention are

[0051] 1. This invention significantly improves the flexibility and efficiency of federated graph learning by sharing a global prototype among clients and using a hash function for node embedding mapping;

[0052] 2. This invention employs anchor embedding and hash encoding for cross-client knowledge sharing, avoiding direct transmission of large-scale graph data and ensuring data privacy and security;

[0053] 3. By employing graph autoencoders and consistency constraint optimization algorithms, the alignment of local and global embeddings is effectively achieved, eliminating alignment errors caused by topological deviations and data heterogeneity between clients, thereby improving the model's generalization ability and collaborative training effect.

[0054] 4. This invention reduces the amount of data that needs to be transmitted by using hash encoding and further controls the association between the client and the global prototype by adjusting the hash bucket width, effectively reducing communication overhead. Attached Figure Description

[0055] Figure 1 This is a schematic diagram illustrating the process of a heterogeneous federated graph learning method according to the present invention;

[0056] Figure 2 This is a schematic diagram comparing the communication costs of the present invention;

[0057] Figure 3 This is a schematic diagram illustrating the accuracy comparison of the present invention; wherein, Figure 3 (a) is the Louvain partition of the product dataset. Figure 3 (b) is the Metis partition of the commodity dataset.

[0058] Figure 4 This is a block diagram of a heterogeneous federated graph learning system according to the present invention. Detailed Implementation

[0059] like Figures 1-3 As shown, this invention includes a heterogeneous federated graph learning method, as detailed below:

[0060] S1. In each client, a globally shared symbiotic space is constructed, and a unified target semantic prototype is generated through a tag propagation mechanism;

[0061] S11. In each client, a preliminary semantic prototype of the client is generated based on the topology information and the original characteristics of the nodes, and a globally shared symbiotic space is constructed based on the preliminary semantic prototype of each client.

[0062] In one embodiment, each client first constructs a globally shared symbiotic space. This symbiotic space is a cross-client shared representation space used to ensure that graph nodes of all clients are aligned in the same semantic space. During the construction process, each client needs to utilize its own topological information and the original features of its nodes to generate an initial semantic prototype. The initial semantic prototype is a semantic representation initially generated by the client based on its local graph structure and node features. It provides high-level semantic information of the local graph within the shared symbiotic space and serves as the basis for subsequent semantic fusion and alignment. The client's own topological information refers to the connection relationships between nodes, and the original features of the nodes include node attributes, feature vectors, etc.

[0063] The construction of the globally shared symbiotic space includes:

[0064] (a1) For the m-th client, there exists its original feature X. m , diagram structure Am Task-related tags Y m And arbitrary graph neural network encoders (GNNs) m Its predicted pseudo-labels and node embedding E m Calculate using the following formula:

[0065]

[0066] (a2) Subsequently, the pseudo-labels were obtained. Topological knowledge is further enhanced through a label propagation mechanism with a smoothing factor α to form a prototype assignment matrix. The prototype assignment matrix describes the association between the client's semantic prototype and the global target prototype, and is used to share semantic information in the global model. Its calculation formula is as follows:

[0067]

[0068] in, Let l represent the prototype assignment matrix during the l-th propagation. When l is 0, the prototype assignment matrix is ​​consistent with the pseudo-label, i.e. l represents the number of propagation layers.

[0069] The smoothing factor is set to 0.5 to balance the influence of a node's own prediction and its neighbors. L is the number of iterations controlling the label propagation. Graph neural networks are inherently well-suited for semi-supervised scenarios with very few labels, so this scheme remains applicable even in scenarios with missing or sparse labels. Even if none of the clients have labels, hash encoding can still be performed using traditional locality-sensitive hashing (LSH) settings, where the global prototype is set to a set of vectors randomly sampled from a Gaussian distribution by the server. Therefore, this scheme maintains robustness even with missing or sparse labels.

[0070] (a3) Assign matrix according to prototype After transposing and normalizing it by row, based on the node embedding E m Forming a local prototype Q m And upload it to the server based on its category prior distribution. Aggregation forms a global prototype The calculation formula is as follows:

[0071]

[0072] in, express The inverse of the degree matrix, The transpose of the prototype assignment matrix. Represents the global prototype of category c. This represents the node sample belonging to category c. This represents the proportion of samples from client m in category c. This represents the prototype of type c in the m-th client, where c represents the specific category.

[0073] Among them, an improvement was made to the locality-sensitive hashing based on Euclidean distance, replacing the traditional random Gaussian vector with a global prototype to capture the differences between different categories.

[0074] S12. The initial semantic prototypes of each client are merged through a label propagation mechanism, and combined with the adjacency information of the nodes, multiple rounds of iteration are performed to generate a unified target semantic prototype.

[0075] In one embodiment, after each client generates an initial semantic prototype, these prototypes need to be merged through a label propagation mechanism. This mechanism ensures that adjacent nodes can share semantic information and gradually reach consensus by propagating the node's label information—its initial semantic prototype—between nodes. During this process, the client undergoes multiple iterations. In each iteration, the node's label information depends not only on its own features but also on the influence of its neighboring nodes, thus gradually adjusting and optimizing the semantic representation of each node. Finally, after multiple iterations, the nodes of each client converge to a unified target semantic prototype, which shares commonalities across clients.

[0076] The label propagation mechanism includes:

[0077] (i1) Calculate the initial label of each node to obtain the label vector of the node;

[0078] In one embodiment, during the initial stage of the label propagation mechanism, an initial label needs to be calculated for each node in the graph. The initial label is typically based on the node's original features. For example, each node's initial label might be the result of clustering based on its features, or a predefined category label. Each node's initial label is represented as a label vector, where each dimension represents a portion of the node's features, or the probability distribution of the node's category. The node's original features include: node attributes, feature values, etc.

[0079] (i2) Based on the initial label of each node, multiple iterations are performed through the label propagation mechanism to update the label vector of each node and obtain the final label of each node;

[0080] In one embodiment, the core idea of ​​the label propagation mechanism is to propagate label information between nodes iteratively. At the start of the first iteration, each node's label vector consists only of its own initial label. Then, in each iteration, the node's label vector is influenced by the labels of its neighboring nodes. Specifically, each node calculates a new label vector based on the label information of its neighboring nodes and the propagation rules. Nodes integrate the label information of their neighboring nodes into their own label vector, typically using a weighted average method. The weights are based on the adjacency relationship between nodes or other factors such as similarity. After multiple iterations, the node's label vectors gradually converge, resulting in each node's final label, which reflects the node's final position and category in the global semantic space.

[0081] (i3) After each iteration, update the previous label according to the final label of each node, so that the label of each node is consistent with the labels of its neighboring nodes, and obtain the final label propagation result, which includes the final label of each node and the updated label propagation matrix.

[0082] In one embodiment, the label propagation process needs to be updated after each iteration. Label propagation is a matrix recording the label information of all nodes, with each row representing a node's label vector. The update process mainly involves adjusting the data in the matrix based on the final label of each node. During the update, the matrix is ​​adjusted according to the similarity and connection relationships between nodes, making the labels of adjacent nodes more consistent. This ensures consistency in the label propagation process, and that each node's label is not only related to its own initial label but also consistent with the labels of its neighboring nodes. Through iteration and matrix updates, the final label propagation result includes the final label of each node and the updated label propagation matrix. The entire label propagation process continues until the labels of all nodes are stable.

[0083] S2. Based on the unified target semantic prototype, generate a prototype assignment matrix and upload it to the server.

[0084] S21. Based on the relationship between the unified target semantic prototype and the preliminary semantic prototype of each client, calculate the prototype assignment matrix for each client;

[0085] In one embodiment, the system calculates a prototype assignment matrix for each client based on the relationship between a unified target semantic prototype and each client's initial semantic prototype. Specifically, the unified target semantic prototype is a global semantic prototype shared by all clients, while each client's initial semantic prototype is a semantic representation of its local graph. To align the client's initial semantic prototype with the global semantic prototype, the similarity or distance between them needs to be quantified first. This process uses methods such as Euclidean distance or cosine similarity to calculate the relationship, i.e., the similarity or matching degree, between each client's initial semantic prototype and the global target semantic prototype. The result is a prototype assignment matrix, which records the relative position or similarity value between the client and the global semantic prototype. Each matrix element represents how a node of the client matches or is associated with the global target semantic prototype, allowing each client to adjust and optimize its semantic representation based on its association with the global prototype.

[0086] S22. Upload the prototype allocation matrix for each client to the server.

[0087] In one embodiment, when each client calculates its prototype assignment matrix with respect to the global prototype, these matrices need to be uploaded to the server. This uploading is to centrally store the association information between each client and the global prototype, enabling the server to perform global data aggregation and processing. Each client's prototype assignment matrix contains the client's relative position in the global semantic space and its matching relationship with the global prototype. After uploading this information, the server uses these matrices to calculate an optimized version of the global semantic prototype, further promoting knowledge sharing and alignment among different clients.

[0088] The information mentioned above includes not only the prototype assignment matrix itself, but may also include additional information for subsequent optimization, such as the client's graph structure and node characteristics.

[0089] S3. On the server side, a global prototype is generated by aggregating the prototype allocation matrix of each client, and the global prototype is distributed to each client;

[0090] In one embodiment, the server receives prototype allocation matrices from all clients and aggregates the prototype data from each client to generate a global prototype. The global prototype contains semantic information across the entire system, providing a more general global representation by fusing the semantic prototypes from various clients. The generated global prototype is then distributed back to each client by the server, allowing the clients to use the global semantic information to further optimize their local models.

[0091] The experimental environment and setup included an Intel Xeon Silver 4314 processor with a clock speed of 2.40GHz, an NVIDIA GeForce RTX 4090 graphics card with 24GB of video memory, an Ubuntu 20.04.6 operating system, and 384GB of system memory.

[0092] S4. On each client, based on the global prototype, a hash function is used to embed and map the node into a hash bucket, generating an aligned local hash code and a local anchor embedding, and the aligned local hash code and local anchor embedding are uploaded to the server.

[0093] Anchor points are representative nodes generated through graph data processing, such as hash encoding and compression. Anchor points are usually compressed representations of nodes in the graph, representing certain specific graph structures or features.

[0094] S41. Each client receives the global prototype, calculates the node embedding based on the Euclidean distance, maps the node embedding to a hash bucket using a hash function, and generates the local hash code. The calculation formula for mapping the node embedding to the hash bucket using the hash function is as follows:

[0095]

[0096] Among them, H m E represents the local hash code. m This indicates different node embeddings, where m represents the client's sequence number. The transpose matrix represents the global prototype, where 1 represents a column vector of all 1s, b represents the bias, T represents the transpose operation, and r represents the hash bucket width, which is an adjustable hyperparameter in implementation.

[0097] Obtain local hash code Where i represents the node index, N m This indicates the number of local client nodes.

[0098] In one embodiment, the client first receives a global prototype already computed on the server side. Then, each client calculates the node embedding using Euclidean distance based on the global prototype and the node features of its local graph. Node embeddings are low-dimensional representations of node features learned through a graph neural network model, effectively capturing the relationships between nodes. Next, the client uses a hash function to map each node embedding into a hash bucket. The hash bucket groups similar nodes together, facilitating matching in subsequent aggregation operations. Through these steps, each node embedding is mapped to a hash bucket, generating a corresponding local hash code. This hash code is an efficient way to represent node embeddings and measure the similarity between nodes.

[0099] S42. Deduplicate the local hash code to obtain the global hash codebook;

[0100] In one embodiment, after the client-calculated local hash code is uploaded to the server, the server performs deduplication on the local hash code to obtain a global hash codebook. The purpose of deduplication is to ensure that when identical nodes are embedded and mapped to the same hash bucket, only one unique hash code is retained, reducing redundant information and ensuring encoding consistency. The global hash codebook contains all the unique hash codes uploaded by clients, representing the unique identifier of each node in the global graph.

[0101] The formula for deduplication is as follows:

[0102]

[0103] Where U represents the global hash codebook, M represents the total number of clients, u1 represents the first hash code after deduplication, and k represents a positive integer.

[0104] S43. Generate local anchor embeddings based on the unified target semantic prototype;

[0105] In one embodiment, the client generates local anchor embeddings based on a unified target semantic prototype, i.e., a global semantic prototype. Anchor embeddings are generated by combining the topological and feature information of the local graph with the global semantic prototype. Specifically, anchor embeddings can be viewed as a localized representation of the global prototype, adapting to the characteristics of the local graph while maintaining consistency in global information.

[0106] S44. Aggregate the local hash code and the local anchor embedding using the global hash codebook to obtain aligned local hash code and local anchor embedding, and upload the aligned local hash code and local anchor embedding to the server.

[0107] In one embodiment, the client aggregates its locally computed local hash code and local anchor embedding with hash codes from the global hash codebook. Specifically, the client uses a hash matching operation to find global hash codes that are identical or similar to its local hash code. In this way, the mapping relationship between the client's local hash code and the global hash code is clarified. Then, the client uses a hash matching operation to find global anchor embeddings that are identical or similar to its local anchor embedding, aligning the local and global anchor embeddings to ensure semantic and topological consistency. The client then uploads the aligned local hash code and local anchor embedding to the server.

[0108] S5. On the server side, the aligned local hash code and local anchor embedding are aggregated to obtain the global hash code and global anchor embedding, and the global hash code and global anchor embedding are distributed to the client.

[0109] S51. On the server side, based on the global hash codebook, the aligned local hash codes are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local hash codes and the global hash codes to obtain the global hash codes, wherein the optimization algorithm is any algorithm used to minimize the alignment error;

[0110] In one embodiment, on the server side, a global hash codebook serves as a global reference, providing a unified hash encoding standard. Client-side local hash encodings can then be adjusted and aggregated based on this standard. First, the aligned local hash encodings are weighted and averaged, then combined with data from each client to obtain a global hash encoding. An optimization algorithm is used in this process to reduce the discrepancy between the local and global hash encodings, ensuring they match as closely as possible. The data from each client includes node feature data, topological information, etc.

[0111] S52. Based on the global hash codebook, the aligned local anchor embeddings are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local anchor embeddings and the global anchor embeddings to obtain the global anchor embeddings;

[0112] In one embodiment, the server also performs a weighted average of the aligned local anchor embeddings, combining this with data from each client to obtain a global anchor embedding. This process also uses optimization algorithms to reduce the differences between local and global anchor embeddings, ensuring better alignment.

[0113] S53. Distribute the global hash encoding and global anchor embedding to the client.

[0114] S6. In the client, the global hash encoding and the global anchor embedding are optimized through graph autoencoder and consistency constraints to complete the alignment of the optimized condensed graph.

[0115] S61. In each client, based on the distance between the local hash code and the global hash code, the first loss function of the hash code is minimized using a graph autoencoder model, so that the local hash code is aligned with the global hash code;

[0116] In one embodiment, the hash encoding first loss function is a metric that measures the difference between local and global hash encodings. At each client, by comparing the gap between local and global hash encodings, a graph autoencoder model is used to continuously adjust and optimize the encodings, making them increasingly consistent, thus achieving alignment. This process is achieved by minimizing the differences between hash encodings, ensuring they are structurally matched.

[0117] S62. Based on the alignment of the local hash code and the global hash code, and based on minimizing the distance between the local anchor embedding and the global anchor embedding, the first loss function of the anchor embedding is minimized using the graph autoencoder model, so that the local anchor embedding is aligned with the global anchor embedding.

[0118] In one embodiment, the first loss function for anchor embedding is the loss function minimized by an optimization algorithm during the training of the graph autoencoder. It directly reflects the difference between local and global anchor embeddings, typically using alignment error methods such as Euclidean distance or cosine similarity. Local anchor embeddings represent the node embeddings learned by the client within the local map, while global anchor embeddings are reference embeddings in the global model. By minimizing this loss function, the graph autoencoder can learn an optimal node embedding, minimizing the distance between local and global embeddings and achieving higher alignment.

[0119] S63. Based on the alignment of the local anchor embedding and the global anchor embedding, the local anchor embedding is optimized by constraining the second loss function of anchor embedding through structural consistency, so that the optimized local anchor embedding is consistent with the global anchor embedding, thus completing the alignment of the optimized condensation map.

[0120] In one embodiment, the anchor embedding second loss function is an optimization objective for structural consistency, typically used to measure the alignment of embeddings while preserving structure. Its role is to ensure that structural information, such as adjacency relationships, is well preserved during local embedding optimization, and to align the local embedding with the global embedding. Local anchor embeddings are further optimized through structural consistency constraints. Structural consistency constraints, in addition to minimizing the distance between local and global embeddings during optimization, also require maintaining the structural consistency of the embeddings, i.e., maintaining consistent topological relationships between nodes. By using the anchor embedding second loss function with structural consistency constraints, the optimized embeddings are numerically aligned, while also preserving connection patterns and adjacency relationships in the graph structure.

[0121] The concentration process is as follows:

[0122] (b1) For each client, an allocation matrix is ​​constructed based on the global hash codebook U to bridge the mapping relationship between nodes and semantic codes. For client m, its allocation matrix Π m The calculation formula is as follows:

[0123]

[0124] in, Let δ represent the element in the i-th row and k-th column of the allocation matrix for the m-th client, and let δ represent the Kronecker function, where δ is the hash code of the i-th node. The k-th hash code u after deduplication k When consistent, Otherwise, it is 0.

[0125] (b2) Subsequently, the anchor point of the local client is embedded in P m and condensed diagram structure It can be derived from the allocation matrix, and its calculation formula is as follows:

[0126]

[0127] in, This represents the assignment matrix after transpose.

[0128] The alignment process for the condensed image is as follows:

[0129] (c1) In order to capture global anchor embedding The knowledge is acquired through global embedding in a manner similar to the generation of global prototypes. Subsequently, the knowledge of other clients is captured by aligning the consistency between local anchor embeddings and global anchor embeddings. The formula for calculating the first loss function of anchor embedding is as follows:

[0130]

[0131] Among them, L s K represents the first loss function for anchor point embedding. m This represents the hash code number of the m-th client, where k represents a positive integer. This indicates the embedding of the k-th anchor point in the m-th client. This indicates the embedding of the k-th global anchor point.

[0132] (c2) Correction of local topological bias is achieved by aligning the pseudomap and condensed map reconstructed through anchor point embedding. The calculation formula for the second loss function of anchor point embedding is as follows:

[0133]

[0134] Among them, L a The anchor embedding second loss function, P m Anchor embedding on the local client, This represents the transpose operation of the anchor embedding on the local client. denoted as the condensed graph structure, σ(·) represents the sigmoid function, and F represents the matrix norm.

[0135] Figure 2 In order to compare the communication cost of this invention with existing mainstream federated learning or federated graph learning algorithms, since this invention only transmits anchor embeddings and a small number of prototypes, its client-server communication cost is much lower than the existing transmission cost. Figure 3 This paper presents a performance comparison between the present invention and existing mainstream federated graph learning algorithms, demonstrating the generalization ability of the present invention by simulating different numbers of clients ranging from 20 to 40. Experimental results show that the present invention consistently outperforms existing mainstream methods under various settings.

[0136] This invention significantly improves the flexibility and efficiency of federated graph learning by sharing a global prototype among clients and using hash functions for node embedding mapping. It employs anchor embedding and hash encoding for cross-client knowledge sharing, avoiding direct transmission of large-scale graph data and ensuring data privacy and security. By using graph autoencoders and consistency constraint optimization algorithms, it effectively aligns local and global embeddings, eliminating alignment errors caused by topological deviations and data heterogeneity between clients, thus improving the model's generalization ability and collaborative training performance. Furthermore, by using hash encoding to reduce the amount of data that needs to be transmitted and adjusting the hash bucket width, it further controls the association between clients and the global prototype, effectively reducing communication overhead.

[0137] Figure 4 This is a block diagram of a heterogeneous federated graph learning system according to the present invention.

[0138] Based on the same concept, the present invention also provides a heterogeneous federated graph learning system, comprising:

[0139] A symbiotic module 41 is constructed to build a globally shared symbiotic space in each client based on the initial semantic prototype of each client, and to generate a unified target semantic prototype through a label propagation mechanism.

[0140] Upload matrix module 42 is used to generate a prototype allocation matrix based on the unified target semantic prototype and upload it to the server.

[0141] Generate global module 43, which is used on the server side to generate a global prototype by aggregating the prototype allocation matrix of each client, and distribute the global prototype to each client;

[0142] The embedding mapping module 44 is used to embed and map nodes into hash buckets on each client based on the global prototype using a hash function, generate aligned local hash codes and local anchor embeddings, and upload the aligned local hash codes and local anchor embeddings to the server.

[0143] Anchor embedding module 45 is used on the server side to aggregate the aligned local hash code and local anchor embedding respectively to obtain global hash code and global anchor embedding, and distribute the global hash code and global anchor embedding to the client;

[0144] The alignment optimization module 46 is used in the client to optimize the alignment of the condensed graph by optimizing the global hash encoding and the global anchor embedding through graph autoencoder and consistency constraints.

Claims

1. A heterogeneous federated graph learning method, characterized in that, include: In each client, a globally shared symbiotic space is constructed based on the initial semantic prototype of each client, and a unified target semantic prototype is generated through a label propagation mechanism; Based on the unified target semantic prototype, a prototype assignment matrix is ​​generated and uploaded to the server. On the server side, a global prototype is generated by aggregating the prototype allocation matrix of each client, and then the global prototype is distributed to each client. Based on the global prototype, each client uses a hash function to embed nodes into hash buckets, generating aligned local hash codes and local anchor embeddings, and then uploads the aligned local hash codes and local anchor embeddings to the server. On the server side, the aligned local hash codes and local anchor embeddings are aggregated to obtain global hash codes and global anchor embeddings, and then the global hash codes and global anchor embeddings are distributed to the clients. In the client, the global hash encoding and the global anchor embedding are optimized through graph autoencoders and consistency constraints to complete the alignment of the optimized condensed graph.

2. The heterogeneous federated graph learning method as described in claim 1, characterized in that, In each client, a globally shared symbiotic space is constructed, and a unified target semantic prototype is generated through a tag propagation mechanism, including: In each client, a preliminary semantic prototype is generated based on the topology information and the original characteristics of the nodes, and a globally shared symbiotic space is constructed based on the preliminary semantic prototype of each client. The initial semantic prototypes of each client are merged through a tag propagation mechanism, and combined with the adjacency information of the nodes, and multiple rounds of iteration are performed to generate a unified target semantic prototype.

3. The heterogeneous federated graph learning method as described in claim 1 or 2, characterized in that, The label propagation mechanism includes: Calculate the initial label for each node to obtain the label vector of the node; Based on the initial label of each node, the label vector of each node is updated through multiple rounds of iteration using the label propagation mechanism to obtain the final label of each node; After each iteration, the previous label is updated according to the final label of each node, so that the label of each node is consistent with the labels of its neighboring nodes, and the final label propagation result is obtained. The label propagation is a matrix that records the label information of all nodes. The final label propagation result includes the final label of each node and the updated label propagation matrix.

4. The heterogeneous federated graph learning method as described in claim 1, characterized in that, The step of generating a prototype assignment matrix based on the unified target semantic prototype and uploading it to the server includes: Based on the relationship between the unified target semantic prototype and the initial semantic prototype of each client, the prototype assignment matrix of each client is calculated. The prototype allocation matrix for each client is uploaded to the server.

5. The heterogeneous federated graph learning method as described in claim 1, characterized in that, The process of mapping node embeddings to hash buckets using a hash function on each client based on the global prototype, generating aligned local hash codes and local anchor embeddings, and uploading the aligned local hash codes and local anchor embeddings to the server includes: Each client receives the global prototype, calculates the node embedding based on Euclidean distance, maps the node embedding to a hash bucket using a hash function, and generates the local hash code. The calculation formula for mapping the node embedding to the hash bucket using the hash function is as follows: Among them, H m Indicates local hash encoding, E m This indicates different node embeddings, where m represents the client's sequence number. The transpose matrix represents the global prototype, where 1 represents a column vector of all 1s, b represents the bias, T represents the transpose operation, and r represents the hash bucket width. The local hash code is deduplicated to obtain the global hash codebook; Generate local anchor embeddings based on the unified target semantic prototype; The local hash code and the local anchor embedding are aggregated using the global hash codebook to obtain aligned local hash code and local anchor embedding, and then the aligned local hash code and local anchor embedding are uploaded to the server.

6. The heterogeneous federated graph learning method as described in claim 5, characterized in that, On the server side, the aligned local hash codes and local anchor embeddings are aggregated to obtain global hash codes and global anchor embeddings, including: On the server side, based on the global hash codebook, the aligned local hash codes are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local hash codes and the global hash codes to obtain the global hash codes. The optimization algorithm can be any algorithm used to minimize the alignment error. Based on the global hash codebook, the aligned local anchor embeddings are aggregated by weighted average, and an optimization algorithm is used to minimize the alignment error between the local anchor embeddings and the global anchor embeddings to obtain the global anchor embeddings. The global hash encoding and global anchor embedding are distributed to the client.

7. The heterogeneous federated graph learning method as described in claim 1, characterized in that, In the client-side, the global hash encoding and global anchor embedding are optimized through graph autoencoders and consistency constraints to optimize the alignment of the condensed graph, including: In each client, based on the distance between the local hash code and the global hash code, the first loss function of the hash code is minimized using a graph autoencoder model, so that the local hash code is aligned with the global hash code; Based on the alignment of the local hash code and the global hash code, and based on the distance between the local anchor embedding and the global anchor embedding, the first loss function of the anchor embedding is minimized using the graph autoencoder model, so that the local anchor embedding and the global anchor embedding are aligned. Based on the alignment of the local anchor embedding with the global anchor embedding, the local anchor embedding is optimized by constraining the second loss function of anchor embedding through structural consistency, so that the optimized local anchor embedding is consistent with the global anchor embedding, thus completing the alignment of the optimized condensation map.

8. The heterogeneous federated graph learning method as described in claim 7, characterized in that the calculation formula for the anchor embedding first loss function is as follows: in, L s K represents the first loss function for anchor point embedding. m This represents the hash code number of the m-th client, where k represents a positive integer. This indicates the embedding of the k-th anchor point in the m-th client. This indicates the embedding of the k-th global anchor point.

9. The method according to claim 7, characterized in that, The formula for calculating the second loss function of anchor point embedding is as follows: Among them, L a The anchor embedding second loss function, P m Anchor embedding on the local client, This indicates that the anchor point of the local client is embedded in the transpose matrix. This represents the condensed graph structure of the local client, σ(·) represents the sigmoid function, and F represents the matrix norm.

10. A heterogeneous federated graph learning system, characterized in that, include: Construct a symbiotic module (41) to build a globally shared symbiotic space in each client based on the initial semantic prototype of each client, and generate a unified target semantic prototype through a label propagation mechanism; The upload matrix module (42) is used to generate a prototype allocation matrix based on the unified target semantic prototype and upload it to the server. Generate a global module (43) to generate a global prototype on the server side by aggregating the prototype allocation matrix of each client and distribute the global prototype to each client; The embedding mapping module (44) is used to embed and map nodes into hash buckets on each client based on the global prototype using a hash function, generate aligned local hash codes and local anchor embeddings, and upload the aligned local hash codes and local anchor embeddings to the server. Anchor embedding module (45) is used on the server side to aggregate the aligned local hash code and local anchor embedding respectively to obtain global hash code and global anchor embedding, and distribute the global hash code and global anchor embedding to the client; An optimization alignment module (46) is used in the client to optimize the alignment of the condensed graph by optimizing the global hash encoding and the global anchor embedding through a graph autoencoder and consistency constraints.

Images