A federated graph model training system and method based on graph decomposition

Abstract

The specification provides a federated graph model training system and method based on graph decomposition, the system comprising: a server and at least two clients; the server is used to send an initial graph model and a global graph atom associated with the initial graph model to at least two clients; a target client in at least two clients is used to combine a graph atom weight and a global graph atom into global structure information; determine a similarity matrix corresponding to a local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; train the initial graph model into an intermediate graph model based on the global structure information and the similarity matrix, and calculate the graph atom gradient of the global graph atom; send the graph atom gradient and the intermediate graph model parameter of the intermediate graph model to the server; the server is used to perform federated aggregation based on the intermediate graph model parameters and the graph atom gradient respectively submitted by at least two clients, and the initial graph model, and determine a target graph model meeting a training stop condition according to an aggregation result.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a training system and method for federated graph models based on graph decomposition. Background Technology

[0002] In an era of rapid growth in distributed computing and graph-based data across multiple domains, effectively training graph neural networks remains a significant challenge. Traditional federated learning methods have enabled collaborative model training without centralized data aggregation, but these methods often exhibit limitations in fully capturing and utilizing the inherent global structural knowledge within distributed graph data. This limitation significantly hinders the ability to effectively model local structures, leading to unsatisfactory learning outcomes.

[0003] Existing technologies utilize graph neural networks to extract potential shared information from heterogeneous data. Different sharing methods are selected among different clients based on parameter size to adaptively aggregate models. New implicit graph structures are constructed to improve the initial graph structure, and different parameters are used to capture shared and differential information in graph features. While attempting to utilize shared structural information among clients, these technologies primarily focus on the local structure of nodes and their surrounding nodes, lacking the capture of global structural knowledge. Consequently, the trained models have weak representational capabilities. Therefore, a federated graph model training system based on graph decomposition is urgently needed to address these issues. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a federated graph model training system based on graph decomposition. This specification also relates to a method for training federated graph models based on graph decomposition, a computing device, a computer-readable storage medium, and a computer program product, to address the aforementioned problems in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a federated graph model training system based on graph decomposition is provided, including a server and at least two clients; The server is used to send the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients is configured to combine graph atom weights and global graph atoms into global structure information; determine the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atoms and the initial graph model; train the initial graph model into an intermediate graph model based on the global structure information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; and send the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server. The server is used to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and to determine the target graph model that meets the training stopping condition based on the aggregation result.

[0006] According to a second aspect of the embodiments of this specification, a method for training a federated graph model based on graph decomposition is provided, including a server and at least two clients; The server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients combines the graph atom weights and the global graph atoms into global structure information; determines the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atoms and the initial graph model; trains the initial graph model into an intermediate graph model based on the global structure information and the similarity matrix, and calculates the graph atom gradients of the global graph atoms; and sends the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and determines the target graph model that meets the training stopping condition based on the aggregation result.

[0007] According to a third aspect of the embodiments of this specification, a computing device is provided, including a memory, a processor, and a computer program or instructions stored in the memory and executable on the processor, wherein the processor executes the computer program or instructions to implement the steps of the federated graph model training method based on graph decomposition.

[0008] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program or instructions which, when executed by a processor, implement the steps of the federated graph model training method based on graph decomposition.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described federated graph model training method based on graph decomposition.

[0010] The federated graph model training system based on graph decomposition provided in this specification includes a server and at least two clients. The server sends an initial graph model and global graph atoms associated with the initial graph model to at least two clients. A target client among the at least two clients combines graph atom weights and global graph atoms into global structure information. It determines the similarity matrix corresponding to the local subgraphs based on the local subgraphs associated with the global graph atoms and the initial graph model. Based on the global structure information and the similarity matrix, it trains the initial graph model into an intermediate graph model and calculates the graph atom gradients of the global graph atoms. It sends the graph atom gradients and intermediate graph model parameters to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and determines the target graph model that meets the training stopping condition based on the aggregation result.

[0011] In one embodiment of this specification, multiple clients each hold different local subgraphs. These client-held local subgraphs are decomposed into trainable graph atoms, which the clients can then combine to train an intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client and performs federated training based on these parameters to obtain the target graph model. During the training of the target graph model, potential graph node connections within the local subgraphs held by each client are mined, enabling global-perspective graph model training, enhancing the modeling and representation capabilities of the graph model, and efficiently training the model while protecting privacy. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of a federated graph model training system based on graph decomposition provided in one embodiment of this specification; Figure 2 This is a structural diagram of a federated graph model training system based on graph decomposition, provided in one embodiment of this specification. Figure 3 This is an interactive schematic diagram of a federated graph model training system based on graph decomposition provided in one embodiment of this specification; Figure 4 This is a flowchart of a federated graph model training method based on graph decomposition provided in one embodiment of this specification; Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0013] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0014] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0016] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0017] The Fused Gromov-Wasserstein (FGW) structural reconstructor is a method for comparing between metric spaces. The FGW method allows for the simultaneous consideration of the probability distributions and topological structures of nodes in two different metric spaces, making it extremely useful when dealing with complex datasets, especially when it is necessary to model both the feature space and its intrinsic geometry simultaneously.

[0018] Figure 1The diagram illustrates a federated graph model training system based on graph decomposition according to an embodiment of this specification. The system includes a server and at least two clients. The server sends an initial graph model and global graph atoms associated with the initial graph model to the at least two clients. A target client among the at least two clients combines graph atom weights and global graph atoms into global structure information. It determines the similarity matrix corresponding to the local subgraphs based on the local subgraphs associated with the global graph atoms and the initial graph model. Based on the global structure information and the similarity matrix, it trains the initial graph model into an intermediate graph model and calculates the graph atom gradients of the global graph atoms. It sends the graph atom gradients and intermediate graph model parameters to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and determines a target graph model that meets the training stopping condition based on the aggregation result.

[0019] During model training, multiple clients each hold different local subgraphs. These client-held local subgraphs are decomposed into trainable graph atoms, which the clients can then combine to train an intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client and performs federated training based on these parameters to obtain the target graph model. During the training of the target graph model, the potential graph node connections within each client's local subgraph are mined, enabling global-perspective graph model training, enhancing the modeling and representation capabilities of the graph model, and achieving efficient model training while protecting privacy.

[0020] This specification provides a federated graph model training system based on graph decomposition, and also relates to a method for training federated graph models based on graph decomposition, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail in the following embodiments.

[0021] Figure 2The diagram illustrates a structural diagram of a federated graph model training system based on graph decomposition according to an embodiment of this specification. The federated graph model training system 200 includes a server 210 and at least two clients 220. The server is configured to send an initial graph model and global graph atoms associated with the initial graph model to the at least two clients. A target client among the at least two clients is configured to combine graph atom weights and the global graph atoms into global structural information; determine a similarity matrix corresponding to the local subgraphs associated with the global graph atoms and the initial graph model; train the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; send the graph atom gradients and intermediate graph model parameters of the intermediate graph model to the server; the server is configured to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, and the initial graph model, and determine a target graph model that meets the training stopping condition based on the aggregation result.

[0022] Specifically, the server and client collaborate on training the target graph model using federated learning. The server provides the client with an initial graph model and global graph atoms. The initial graph model refers to a graph neural network model, a type of federated graph model, where the graph neural network is trained collaboratively by the server. A global graph atom is a learnable graph structure unit used to capture and represent global structural information shared across all clients. The server holds at least one global graph atom, or a series of multiple global graph atoms. Graph atom weights are personalized weights for each client, representing the relevance and importance of the corresponding atom within that client. The global structural information represents the optimal personalized representation of the global structural information shared by all clients within the target client.

[0023] Global structural information can be computed using a fused Gromov-Wasserstein (FGW) structural reconstructor. A local subgraph refers to a private subgraph held by the target client, reflecting its local graph structure. The similarity matrix reflects the potential associations and structural similarities between different nodes in the target client's local subgraph. Two nodes will have high values ​​in the similarity matrix if their representations in the embedding space (node ​​embeddings) are very close. This similarity can reveal latent connections not explicitly labeled in the original data corresponding to the target client's local subgraph, but which may exist in other clients. In this way, the similarity matrix helps capture more complete local structural information and improves the learning and inference capabilities of the initial graph model. An intermediate graph model refers to the graph model obtained by the client after one round of model training based on the initial graph model. Graph atom gradients are used to update global graph atoms using stochastic gradient descent, and the updated global graph atoms are used for the next round of graph model training. The target graph model is the model obtained by the server after one round of training on the initial graph model.

[0024] Based on this, the federated graph model training system based on graph decomposition includes a server and at least two clients. The server holds an initial graph model and global graph atoms associated with the initial graph model. The server sends the initial graph model and global graph atoms associated with the initial graph model to at least two clients to begin model training. Each of the at least two clients holds a personalized graph atom weight. The target client among the at least two clients performs combined calculations based on the graph atom weights and global graph atoms to obtain global structure information, completes global structure reconstruction, and then calculates the similarity matrix corresponding to the local subgraphs associated with the global graph atoms and the initial graph model to complete local structure mining. Based on the global structure information and the similarity matrix, the initial graph model is trained into an intermediate graph model, and the graph atom gradients of the global graph atoms are calculated to align the global structure with the local structure. Finally, the intermediate graph model parameters are extracted, and the graph atom gradients and intermediate graph model parameters are sent to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, until the target graph model that meets the training stopping condition is determined based on the aggregation result.

[0025] For example, in a product recommendation scenario, various product sales platforms can cooperate with the server for federated training, thereby completing the training of the graph model without leaking the platform's private user and product data. The server provides an initial graph model to be trained, as well as global structure information shared by all clients, i.e., global graph atoms. The server sends the initial graph model and global graph atoms to each client. Each client performs combined calculations based on personalized graph atom weights and global graph atoms to obtain global structure information, complete global structure reconstruction, and local subgraphs composed of local user and product data associated with global graph atoms, as well as the similarity matrix corresponding to the local subgraphs calculated by the initial graph model, completing local structure mining. Based on the global structure information and similarity matrix, the initial graph model is trained into an intermediate graph model, and the graph atom gradients of the global graph atoms are calculated. The graph atom gradients and intermediate graph model parameters are sent to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by at least two clients to obtain the target graph model that meets the training stopping condition.

[0026] In summary, during model training, multiple clients each hold different local subgraphs. These client-held local subgraphs are decomposed into trainable graph atoms, which the clients can then combine to train an intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client and performs federated training based on these parameters to obtain the target graph model. During the training of the target graph model, the potential graph node connections within the local subgraphs held by each client are mined, enabling global-perspective graph model training, enhancing the modeling and representation capabilities of the graph model, and achieving efficient model training while protecting privacy.

[0027] Furthermore, the target client can use the reconstructor to construct global structural information. In specific implementation, the target client among the at least two clients is used to determine the graph atom weights, calculate the graph atom embeddings of the global graph atoms, determine the distance parameters, node feature parameters, and preset centroid parameters, and use the reconstructor to generate the global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters, and the preset centroid parameters.

[0028] Specifically, graph atom embedding refers to the multidimensional embedding vector of global graph atoms. The distance parameter is the distance metric between the client and the global graph atom. The node feature parameter represents the node features of the centroid; the preset centroid parameter is a predefined centroid. The centroid contains collective global structural knowledge. It is a weighted average of at least one global graph atom, minimizing the weighted sum of differences between each global graph atom and the centroid. Personalized centroids are obtained by calculating global structural information using graph atom weights. The reconstructor refers to a fused Gromov-Wasserstein (FGW) structural reconstructor.

[0029] Based on this, the target client in at least two clients determines the graph atom weights held by the target client and calculates the graph atom embeddings of the global graph atoms. Distance parameters, node feature parameters, and preset centroid parameters are determined, and a reconstructor is used to generate global structural information based on the graph atom weights, graph atom embeddings, distance parameters, node feature parameters, and preset centroid parameters.

[0030] Global structure information can be calculated based on the following formula (1).

[0031]

[0032] in, This refers to the distance metric, i.e., the distance parameter; The node features representing the centroid, i.e., the node feature parameters. This indicates the fusion of the Gromov-Wasestein distance metric. The size of the centroid is predefined. N global graph atoms are represented as... The embedding of N global graph atoms is represented as ,in, Represents the set of real numbers. Indicates the first atom in the global graph The number of nodes per atom, Indicates the embedding dimension. Each client... Randomly initialize the personalized weights for these atoms, i.e., graph atom weights. .

[0033] In summary, by using a reconstructor to generate global structural information based on graph atom weights, graph atom embeddings, distance parameters, node feature parameters, and preset centroid parameters, the accuracy of the global structural information determination is ensured.

[0034] Furthermore, in a specific implementation, the target client among the at least two clients is used to determine the local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information.

[0035] Specifically, node embedding information refers to the multidimensional node embedding of nodes contained in the local subgraph.

[0036] Based on this, the target client in at least two clients determines private client data, and constructs a local subgraph related to the global graph atoms based on the private client data. The local subgraph is then input into the initial graph model, and the feature extractor of the initial graph model generates node embedding information corresponding to the local subgraph. A similarity matrix corresponding to the local subgraph is calculated based on the node embedding information.

[0037] Following the previous example, Indicates client The owned private graph data, i.e., the local subgraph. The local subgraph contains i nodes, and the nodes are... Mapped to Dimensional Embedding This allows obtaining node embedding information. For the client... and There may be overlapping edges and nodes, i.e. and . and These represent the sets of nodes and edges, respectively. This represents the cardinality of the set. Federated model training involves collaboratively training a graph neural network using servers, with the overall goal of minimizing the global loss function.

[0038] In summary, the node embedding information corresponding to the local subgraph is generated using the feature extractor of the initial graph model, and the accuracy of the node embedding information is determined.

[0039] Furthermore, in a specific implementation, the target client among the at least two clients is used to input the local subgraph into the initial graph model, extract node embedding information using the feature extractor in the initial graph model, determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix, and the one-hot vector.

[0040] Specifically, the initial graph model includes a feature extractor. and predictor Feature extractor Include A message passing layer will input nodes Mapped to Dimensional Embedding The weight matrix refers to the weight matrix corresponding to the number of node categories in the client's local subgraph. The one-hot vector represents the input node. and Node labels.

[0041] Based on this, the target client in at least two clients inputs its local subgraph into the initial graph model, and uses the feature extractor in the initial graph model to extract node embedding information. The weight matrix corresponding to the number of node categories in the local subgraph, and the weight matrix representing the input nodes, are then determined. and The node label one-hot vector is used to calculate the similarity matrix corresponding to the local subgraph based on node embedding information, weight matrix, and one-hot vector. In the similarity matrix S, the node label one-hot vector is used to calculate the similarity matrix corresponding to the local subgraph. row and number The elements of the column can be calculated based on the following formula (2).

[0042]

[0043] Continuing with the previous example, where, Representing vectors The average value of each element in the local subgraph of the target client. The node embedding of all nodes in the local subgraph of the target client is represented as... For a specific pair of nodes within the target client subgraph and Their embedding and The correlation between them is given by the Pearson correlation coefficient, i.e. and The covariance is divided by the product of their standard deviations. Meanwhile, class-specific weights are assigned through the representative... and One-hot vector of node labels and The vectors are then derived and trained with a weight matrix. Interaction, in which It represents the number of node categories.

[0044] In practical applications, this formula allows each client to create a similarity matrix, uncovering potential but unrevealed connections indicated by node embeddings, and integrating the importance of class-specific interactions defined by the weight matrix. In the product sales process, the similarity matrix can be used to identify potential but unrevealed relationships between users (such as friendships). For two user nodes within the client's local subgraph... and The similarity between two nodes can be determined by calculating the Pearson correlation coefficient between their embedding vectors. The correlation coefficient measures the linear relationship between embedding vectors; nodes with similar preferred items are likely to have highly correlated embedding vectors, resulting in a higher similarity score. Furthermore, this similarity score can be obtained by utilizing their respective class labels (node ​​class labels might represent user groups, such as "student" or "teacher") and trainable weight matrices. Elements in the similarity matrix Further consideration will be given to class-specific interactions; for example, weight matrices might increase the importance of similarity between nodes of the same class. If A high similarity value means that even if the direct connection between the two nodes is not marked in the original data, the similarity matrix can still reveal their potential relationship, thus providing a basis for relationship discovery and recommendation in product sales.

[0045] In summary, by calculating the similarity matrix corresponding to the local subgraph based on node embedding information, weight matrix, and one-hot vector, the correlation between nodes in the local graph atom is fully explored, which facilitates the subsequent training of the initial graph model.

[0046] Furthermore, after determining the global structure information and similarity matrix, an alternating optimization strategy can be used to train the initial graph model. Specifically, the target client among the at least two clients is used to determine the local training period. Within the local training period, a first loss function is constructed based on the similarity matrix and the global structure information. The predictor in the initial graph model is used to predict node scores based on the node embedding information. A second loss function is constructed based on the node scores and the category labels corresponding to the local subgraph. The initial graph model is trained into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function, and the second loss function.

[0047] Specifically, the local training period refers to the model training period for the client to train the initial graph model. The first loss function is a mapping function, which is the optimal transfer mapping from the local centroid of the global structural information to the local similarity of the similarity matrix. The second loss function can be the cross-entropy loss. The second loss function can be constructed based on the predicted node scores obtained by the predictor of the local subgraphs from the initial graph model's predictor, and the corresponding category labels of the local subgraphs. It calculates the loss value of the model training within the local training period and is used to evaluate the predictive ability of the initial graph model. Based on the first loss function, the second loss function, and the loss parameters, a target loss function can be constructed to train the initial graph model and obtain the intermediate graph model.

[0048] Based on this, the target client in at least two clients determines its local training period. Within this local training period, a first loss function is constructed based on the similarity matrix and global structural information. The predictor in the initial graph model predicts node scores based on node embedding information, and a second loss function is constructed based on the node scores and the category labels corresponding to the local subgraphs. A target loss function is constructed based on the first loss function, the second loss function, and the loss parameters. The initial graph model is then trained into an intermediate graph model based on the target loss function, global graph atoms, and graph atom weights.

[0049] Following the previous example, the FGW difference between local centroids and local similarity is used as a metric. For stability during training, an alternating optimization strategy is employed. First, graph atoms and personalized weights are fixed to generate local centroids. During each local training epoch of training the local graph neural network model, the difference is added to the general cross-entropy loss to guide local modeling. After the client completes graph neural network training, it generates similarity again. Then, a new mapping is established between local centroids and local similarity, and the global graph atoms and the graph atom weights of the target client are updated.

[0050] Feature extractor of the initial graphical model Include A message passing layer will input nodes Mapped to Dimensional Embedding Then, the predictor Node embedding is used to obtain the prediction score. , representing nodes The unnormalized probability of belonging to each class. Given a class label. The general cross-entropy loss is given by the following formula (3).

[0051]

[0052] Local training cycle During this period, the target client First, the received global graph atoms and graph atom weights are fixed to train the initial graph model. Fixed local centroids are then generated using the above formula (1). During each local training period In the above formula (2), the local similarity is calculated. This is from arrive The optimal transport mapping. The loss function for training the initial graph model is given by the following formula (4).

[0053]

[0054] Combining formulas (3) and (4), the total loss of training the initial graphical model can be obtained as shown in formula (5).

[0055]

[0056] Where t represents the communication round between the server and the client; the server sends the initial graph model and the global graph atoms associated with the initial graph model to the target client, and the target client processes the graph atom gradients and intermediate graph model parameters obtained from processing the initial graph model and global graph atoms and sends them back to the server. This process is considered as one round of communication.

[0057] By optimizing this loss function, the client can iteratively calculate the local similarity, thereby improving the node embeddings generated by the local graph neural network.

[0058] In summary, an alternating optimization strategy combined with the objective loss function is adopted to train the initial graph model, thereby enhancing the training effect of the initial graph model.

[0059] Furthermore, after training the initial graph model, an alternating optimization strategy is adopted to calculate the graph atom gradient of the global graph atoms. Specifically, the target client among the at least two clients is used to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; determine a third loss function with an inverse mapping to the first loss function based on the similarity matrix; generate target graph atom weights based on the weight update parameters, the third loss function, and the graph atom weights; and calculate the graph atom gradient of the global graph atoms based on the weight update parameters, the third loss function, the global graph atoms, and the graph atom embeddings.

[0060] Specifically, the intermediate similarity matrix is ​​generated based on the intermediate graph model and local subgraphs, using the same generation method as the similarity matrix. The third loss function has an inverse mapping compared to the second loss function. The target graph atomic weights are obtained after updating the graph atomic weights.

[0061] Based on this, the target client in at least two clients calculates the intermediate similarity matrix corresponding to the local graph atoms based on the local graph atoms and the intermediate graph model. A third loss function with an inverse mapping to the first loss function is determined based on the similarity matrix. The graph atom weights are updated based on the weight update parameters, the third loss function, and the graph atom weights to obtain the target graph atom weights. The graph atom gradients of the global graph atoms are then calculated based on the weight update parameters, the third loss function, the global graph atoms, and the graph atom embeddings. The target graph atom weights and graph atom gradients are used for federated training on the server side.

[0062] Following the previous example, after completing the local graph modeling and obtaining the intermediate graph model, the target client... A local graph neural network, i.e., an intermediate graph model, is fixed, and the weights of global graph atoms and graph atoms are optimized. The trained intermediate graph model generates new local similarities based on local subgraphs. Determine the opposite mapping of formula (4), that is, the following formula (6).

[0063]

[0064] Based on the above formula (6), optimize the weights of global graph atoms and target client graph atoms. The target graph atom weights are determined through... Obtained directly by calculation, while the graph atomic gradient is based on... The calculated graph atomic gradients are then transmitted to the server for federated processing. It is a constant parameter.

[0065] In summary, by employing an alternating optimization strategy, after completing the initial graph model training, the graph atom gradients of global graph atoms are calculated, and the graph atom weights are updated, so that the server and client can continue to train the graph model based on the target graph atom weights and global graph atoms.

[0066] Furthermore, the server plays a crucial role in the federated training of the initial graph model. It needs to combine the data submitted by at least two clients. In specific implementation, the server is used to determine the graph sample weight of each client among the at least two clients, and to perform federated aggregation of the initial graph model based on the graph sample weight of each client and the intermediate graph model parameters. The target graph model is then determined based on the model aggregation result.

[0067] Specifically, the graph sample weight refers to the proportion of the local subgraphs held by the target client to the total local subgraphs held by at least two clients. The target graph model refers to the updated graph model obtained by the server after performing federated aggregation on the initial graph model based on the intermediate graph model parameters submitted by at least two clients.

[0068] Based on this, the server determines the graph sample weight of each client in at least two clients, and after receiving the intermediate graph model parameters submitted by each client in at least two clients, it performs federated aggregation on the initial graph model based on the graph sample weight of each client and the submitted intermediate graph model parameters, and determines the target graph model based on the model aggregation result.

[0069] Continuing with the previous example, after receiving the intermediate graph model parameters submitted by each client, the server determines the proportion of each client's local subgraph relative to the total local subgraphs held by at least two clients, and uses this proportion as the graph sample weight. Based on the intermediate graph model parameters and the graph sample weight, the initial graph model is federated and aggregated to obtain the target graph model. The initial graph model is aggregated into... ,in, The percentage of the sample in the graph is indicated.

[0070] In summary, the server performs federated aggregation of the initial graph model based on the graph sample weights of each client and the parameters of the submitted intermediate graph model, and determines the target graph model based on the aggregation results, thus completing the graph model training while protecting privacy.

[0071] Furthermore, considering that it is difficult for the server to obtain a target graph model with good predictive ability in a single round of graph model training, the server needs to combine with at least two clients to conduct multiple rounds of graph model training to improve the predictive ability of the target graph model. In specific implementation, when the target graph model does not meet the training stopping condition, the server is used to determine the graph atom gradients submitted by each of the at least two clients, update the global graph atoms to target global graph atoms based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients; use the target global graph atoms as the global graph atoms, use the target graph model as the initial graph model, and execute the step of sending the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients.

[0072] Based on this, training stopping conditions can include training epochs, training time, and the prediction accuracy of the initial graph model after one training epoch. If the target graph model does not meet the training stopping conditions, it means that the intermediate graph model obtained by the client cannot be used, and the server needs to continue federated training of the graph model to update the intermediate graph model. The server determines the graph atom gradients submitted by each of at least two clients, and updates the global graph atoms to the target global graph atoms based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients. The server can aggregate the graph atom gradients of the global graph atoms submitted by each client, use the target global graph atom as the global graph atom, use the target graph model as the initial graph model, and return to execute the step of sending the initial graph model and the associated global graph atoms to at least two clients. The next round of graph model training is then performed based on the target global graph atoms and the target graph model until a target graph model that meets the training stopping conditions is obtained.

[0073] Continuing with the previous example, the server receives graph atomic gradients submitted by each client, performs federated aggregation of the graph atomic gradients submitted by each client, and aggregates them into... Using the optimizer included in the initial graph model, the gradient gradient of the aggregated graph atoms is used to update the global graph atoms held by the server using gradient descent, thus obtaining the target global graph atoms. The target graph model and the target global graph atoms can be sent to each client as the initial graph model and global graph atoms for the next round of graph model training. Among them, stochastic gradient descent can be performed using the following formula (7).

[0074]

[0075] in, The learning rate is represented by t, and the number of communication rounds between the server and the client is represented by t.

[0076] In summary, by aggregating the graph atom gradients submitted by each client, the server can initiate subsequent rounds of communication with the clients based on the target global graph atoms and the target graph model. Through multiple rounds of communication, a graph model that meets the training stopping criteria is obtained, thereby improving the predictive ability of the graph model.

[0077] In one embodiment of this specification, multiple clients each hold different local subgraphs. These client-held local subgraphs are decomposed into trainable graph atoms, which the clients can then combine to train an intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client and performs federated training based on these parameters to obtain the target graph model. During the training of the target graph model, potential graph node connections within the local subgraphs held by each client are mined, enabling global-perspective graph model training, enhancing the modeling and representation capabilities of the graph model, and efficiently training the model while protecting privacy.

[0078] The following is in conjunction with the appendix Figure 3 Taking the application of the graph decomposition-based federated graph model training system provided in this specification in social graph models as an example, the graph decomposition-based federated graph model training system will be further explained. Figure 3 This specification illustrates an interactive diagram of a federated graph model training system based on graph decomposition, according to an embodiment of this specification. The system specifically includes the following steps: Step 302: The server sends the initial social graph model and the global graph atoms associated with the initial social graph model to at least two clients.

[0079] The global graph atom G is distributed decentralizedly among K clients, i.e. ,in, Indicates client Possesses private graph data. For the client. and There may be overlapping edges and nodes, i.e. and . and These represent the sets of nodes and edges, respectively. The cardinality of the set is represented. Federated graph model training involves collaboratively training the graph neural network on the server side.

[0080] Step 304: For the target client in at least two clients, determine the graph atom weights and calculate the graph atom embeddings of the global graph atoms; determine the distance parameters, node feature parameters, and preset centroid parameters; and use the reconstructor to generate global structure information based on the graph atom weights, graph atom embeddings, distance parameters, node feature parameters, and preset centroid parameters.

[0081] The target client combines global graph atoms and personalized graph atom weights in the global structure reconstruction module, and uses the Fused Gromov-Wasserstein (FGW) structure reconstructor to compute a personalized centroid. This centroid represents the optimal personalized representation of the global structure information shared by all clients within that client.

[0082] Step 306: The target client inputs the private graph data into the initial social graph model and uses the feature extractor in the initial social graph model to extract node embedding information; determines the weight matrix and one-hot vector of the private graph data, and calculates the similarity matrix corresponding to the private graph data based on the node embedding information, weight matrix and one-hot vector.

[0083] The target client trains a local graph neural network model received from the server in the local structure exploration module, and generates a local similarity matrix, i.e., a similarity matrix, through a graph generator. This generator uses node embeddings calculated from the feature extractor of the local graph neural network model to construct a similarity matrix that reflects the potential associations between nodes, further revealing and mining the implicit structural relationships in the client's subgraph.

[0084] Step 308: The target client determines the local training period. Within the local training period, a first loss function is constructed based on the similarity matrix and global structure information. The predictor in the initial social graph model is used to predict node scores based on node embedding information. A second loss function is constructed based on the node scores and the category labels corresponding to the private graph data. The initial social graph model is trained into an intermediate social graph model based on global graph atoms, graph atom weights, the first loss function, and the second loss function.

[0085] Step 310: The target client generates an intermediate similarity matrix based on the local graph atoms and the intermediate social graph model; based on the similarity matrix, a third loss function with an inverse mapping to the first loss function is determined; target graph atom weights are generated based on the weight update parameters, the third loss function, and the graph atom weights; and the graph atom gradient of the global graph atoms is calculated based on the weight update parameters, the third loss function, the global graph atoms, and the graph atom embedding.

[0086] The target client calculates the FGW difference between personalized centroids and local similarities in the global-local structure alignment module, and alternately updates the weights of local graph neural networks, global graph atoms, and personalized graph atoms.

[0087] Step 312: At least two clients send the graph atomic gradient and intermediate social graph model parameters to the server respectively.

[0088] The updated graph neural network parameters and gradients of graph atoms are sent to the server for aggregation.

[0089] Step 314: If the target social graph model does not meet the training stopping condition, the server updates the global graph atoms to the target global graph atoms based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients; the target global graph atoms are used as global graph atoms, the target social graph model is used as the initial social graph model, and step 302 is executed.

[0090] The updated graph neural network parameters and gradients of graph atoms are sent to the server for aggregation. This communication between the server and the client iterates until the performance converges, resulting in a trained social graph model.

[0091] This embodiment provides a federated graph model training system based on graph decomposition, which decomposes the client's graph into trainable graph atoms, thereby improving the modeling and representation capabilities of the federated graph model by utilizing global graph atoms while protecting graph data privacy.

[0092] Corresponding to the above system embodiments, this specification also provides embodiments of a federated graph model training method based on graph decomposition. Figure 4 A flowchart illustrating a federated graph model training method based on graph decomposition, as provided in one embodiment of this specification, is shown. Figure 4 As shown, the federated graph model training method based on graph decomposition includes a server and at least two clients, and specifically includes the following steps: Step 402: The server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; Step 404: The target client among the at least two clients combines the graph atom weights and the global graph atoms into global structure information; determines the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atoms and the initial graph model; trains the initial graph model into an intermediate graph model based on the global structure information and the similarity matrix, and calculates the graph atom gradients of the global graph atoms; sends the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server. Step 406: The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and determines the target graph model that meets the training stopping condition based on the aggregation result.

[0093] Optionally, the target client among the at least two clients is used to determine the graph atom weights, calculate the graph atom embeddings of the global graph atoms, determine the distance parameters, node feature parameters, and preset centroid parameters, and use a reconstructor to generate the global structure information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters, and the preset centroid parameters.

[0094] Optionally, the target client among the at least two clients is used to determine the local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information.

[0095] Optionally, the target client among the at least two clients is configured to input the local subgraph into the initial graph model, extract node embedding information using the feature extractor in the initial graph model, determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix, and the one-hot vector.

[0096] Optionally, the target client among the at least two clients is used to determine a local training period, construct a first loss function based on the similarity matrix and the global structure information within the local training period, predict node scores based on the node embedding information using the predictor in the initial graph model, construct a second loss function based on the node scores and the category labels corresponding to the local subgraph, and train the initial graph model into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function, and the second loss function.

[0097] Optionally, the target client among the at least two clients is configured to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; determine a third loss function with an inverse mapping to the first loss function based on the similarity matrix; generate target graph atom weights based on weight update parameters, the third loss function, and the graph atom weights; and calculate the graph atom gradient of the global graph atoms based on the weight update parameters, the third loss function, the global graph atoms, and the graph atom embeddings.

[0098] Optionally, the server is configured to determine the graph sample weight of each of the at least two clients, and perform federated aggregation of the initial graph model based on the graph sample weight of each client and the intermediate graph model parameters, and determine the target graph model based on the model aggregation result.

[0099] Optionally, if the target graph model does not meet the training stopping condition, the server is configured to determine the graph atom gradients submitted by each of the at least two clients, update the global graph atoms to target global graph atoms based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients; use the target global graph atoms as the global graph atoms, use the target graph model as the initial graph model, and execute the step of sending the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients.

[0100] In one embodiment of this specification, multiple clients each hold different local subgraphs. These client-held local subgraphs are decomposed into trainable graph atoms, which the clients can then combine to train an intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client and performs federated training based on these parameters to obtain the target graph model. During the training of the target graph model, potential graph node connections within the local subgraphs held by each client are mined, enabling global-perspective graph model training, enhancing the modeling and representation capabilities of the graph model, and efficiently training the model while protecting privacy.

[0101] The above is an illustrative scheme of a federated graph model training method based on graph decomposition according to this embodiment. It should be noted that the technical solution of this federated graph model training method based on graph decomposition belongs to the same concept as the technical solution of the federated graph model training system based on graph decomposition described above. Details not described in detail in the technical solution of the federated graph model training method based on graph decomposition can be found in the description of the technical solution of the federated graph model training system based on graph decomposition described above.

[0102] Figure 5 A structural block diagram of a computing device 500 according to an embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 550 is used to store data.

[0103] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0104] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0105] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 500 can also be a mobile or stationary server.

[0106] The processor 520 implements the steps of the federated graph model training method based on graph decomposition when executing the computer program or instructions.

[0107] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the federated graph model training system based on graph decomposition described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the federated graph model training system based on graph decomposition described above.

[0108] An embodiment of this specification also provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the federated graph model training method based on graph decomposition as described above.

[0109] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the federated graph model training system based on graph decomposition described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the federated graph model training system based on graph decomposition described above.

[0110] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described federated graph model training method based on graph decomposition.

[0111] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above-described federated graph model training system based on graph decomposition belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described federated graph model training system based on graph decomposition.

[0112] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0113] The computer program or instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0114] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0115] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0116] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. These embodiments have been selected and specifically described in this specification to better explain the principles and practical applications of this specification, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A federated graph model training system based on graph decomposition, characterized in that, Includes a server and at least two clients; The server is used to send the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients is used to determine the graph atom weights and calculate the graph atom embeddings of the global graph atoms; Determine the distance parameter, node feature parameter, and preset centroid parameter; use a reconstructor to generate global structure information based on the graph atom weights, graph atom embeddings, distance parameter, node feature parameter, and preset centroid parameter; Based on the local subgraphs associated with the global graph atoms and the initial graph model, a similarity matrix corresponding to the local subgraph is determined; based on the global structure information and the similarity matrix, the initial graph model is trained into an intermediate graph model, and the graph atom gradients of the global graph atoms are calculated; the graph atom gradients and the intermediate graph model parameters of the intermediate graph model are sent to the server; the reconstructor is a fusion Gromov-Wasestein structure reconstructor, and the reconstructor is used to calculate the centroid, which represents the optimal personalized representation of the global structure information shared by all clients on the client. The server is used to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and to determine the target graph model that meets the training stopping condition based on the aggregation result.

2. The federated graph model training system based on graph decomposition according to claim 1, characterized in that, The target client among the at least two clients is used to determine the local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information.

3. The federated graph model training system based on graph decomposition according to claim 2, characterized in that, The target client among the at least two clients is used to input the local subgraph into the initial graph model, extract node embedding information using the feature extractor in the initial graph model, determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix and the one-hot vector.

4. The federated graph model training system based on graph decomposition according to claim 2, characterized in that, The target client among the at least two clients is used to determine the local training period, construct a first loss function based on the similarity matrix and the global structure information within the local training period, and predict node scores based on the node embedding information using the predictor in the initial graph model. A second loss function is constructed based on the node scores and the category labels corresponding to the local subgraphs; the initial graph model is trained into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function, and the second loss function.

5. The federated graph model training system based on graph decomposition according to claim 4, characterized in that, The target client among the at least two clients is used to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; Based on the similarity matrix, a third loss function with an inverse mapping to the first loss function is determined. Target graph atom weights are generated based on the weight update parameters, the third loss function, and the graph atom weights. The graph atom gradients of the global graph atoms are calculated based on the weight update parameters, the third loss function, the global graph atoms, and the graph atom embeddings.

6. The federated graph model training system based on graph decomposition according to claim 1, characterized in that, The server is used to determine the graph sample weight of each of the at least two clients, and to perform federated aggregation of the initial graph model based on the graph sample weight of each client and the intermediate graph model parameters, and to determine the target graph model based on the model aggregation result.

7. The federated graph model training system based on graph decomposition according to claim 1, characterized in that, If the target graph model does not meet the training stopping condition, the server is used to determine the graph atom gradients submitted by each of the at least two clients, and update the global graph atoms to the target global graph atoms based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients. The target global graph atom is used as the global graph atom, the target graph model is used as the initial graph model, and the step of sending the initial graph model and the global graph atom associated with the initial graph model to the at least two clients is performed.

8. A method for training a federated graph model based on graph decomposition, characterized in that, Includes a server and at least two clients; The server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients determines the graph atom weights and calculates the graph atom embeddings of the global graph atoms; Determine the distance parameter, node feature parameter, and preset centroid parameter; use a reconstructor to generate global structure information based on the graph atom weights, graph atom embeddings, distance parameter, node feature parameter, and preset centroid parameter; Based on the local subgraphs associated with the global graph atoms and the initial graph model, a similarity matrix corresponding to the local subgraph is determined; based on the global structure information and the similarity matrix, the initial graph model is trained into an intermediate graph model, and the graph atom gradients of the global graph atoms are calculated; the graph atom gradients and the intermediate graph model parameters of the intermediate graph model are sent to the server; the reconstructor is a fusion Gromov-Wasestein structure reconstructor, and the reconstructor is used to calculate the centroid, which represents the optimal personalized representation of the global structure information shared by all clients on the client. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by the at least two clients, as well as the initial graph model, and determines the target graph model that meets the training stopping condition based on the aggregation result.

9. A computing device, comprising a memory, a processor, and a computer program or instructions stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program or instructions, it implements the steps of the method of claim 8.

10. A computer-readable storage medium storing a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method of claim 8.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method of claim 8.

Images