Federal map model training system and method based on graph decomposition

Abstract

The invention provides a federated graph model training system and method based on graph decomposition. The system comprises a server and at least two clients, the server is used for sending the initial graph model and the global graph atom associated with the initial graph model to the at least two clients; the target client in the at least two clients is used for combining the graph atom weight and the global graph atom into global structure information; determining a similarity matrix corresponding to the local sub-graph based on the local sub-graph and the initial graph model associated with the global graph atoms; training the initial graph model into an intermediate graph model based on the global structure information and the similarity matrix, and calculating a graph atom gradient of global graph atoms; sending the graph atom gradient and intermediate graph model parameters of the intermediate graph model to a server; and the server is used for performing federal aggregation on the basis of the intermediate graph model parameters and the graph atom gradients submitted by the at least two clients respectively and the initial graph model, and determining a target graph model meeting a training stop condition according to an aggregation result.

Description

Technical Field

[0001] The present specification relates to the field of computer technology, and in particular to a federated graph model training system and method based on graph decomposition. Background Art

[0002] In an era of distributed computing and the rapid growth of graph-based data in multiple fields, effectively training graph neural networks remains a major challenge. Traditional federated learning approaches promote collaborative model training without centralized data aggregation, but they often show limitations in fully capturing and leveraging the global structural knowledge inherent in distributed graph data. This limitation significantly hinders the ability to effectively model local structures, leading to unsatisfactory learning outcomes.

[0003] In the prior art, graph neural networks are used to extract potential shared information from heterogeneous data, and different sharing methods are selected according to the size of parameters between different clients to form an adaptive aggregation model. The initial graph structure is improved by building a new implicit graph structure, and different parameters are used to capture the shared and different information in the graph features. Although attempts have been made to utilize the structural information shared between clients, it mainly focuses on the local structure of the node and its surrounding nodes, and lacks the capture of global structural knowledge. Therefore, the representation ability of the trained model is weak. Therefore, there is an urgent need for a federated graph model training system based on graph decomposition to solve the above problems. Summary of the invention

[0004] In view of this, the embodiments of this specification provide a federated graph model training system based on graph decomposition. This specification also relates to a federated graph model training method based on graph decomposition, a computing device, a computer-readable storage medium, and a computer program product to solve the above problems existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a federated graph model training system based on graph decomposition is provided, including a server and at least two clients; The server is used to send the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients is used to combine the graph atom weights and the global graph atoms into global structural information; determine the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; train the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; send the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server; The server is used to perform federated aggregation based on the intermediate graph model parameters and graph atomic gradients respectively submitted by the at least two clients, and the initial graph model, and determine a target graph model that meets the training stop condition according to the aggregation result.

[0006] According to a second aspect of an embodiment of this specification, a method for training a federated graph model based on graph decomposition is provided, comprising a server and at least two clients; The server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients combines the graph atom weights and the global graph atoms into global structural information; determines a similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; trains the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculates the graph atom gradients of the global graph atoms; sends the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server; The server performs federated aggregation based on the intermediate graph model parameters and graph atomic gradients respectively submitted by the at least two clients and the initial graph model, and determines a target graph model that meets the training stop condition according to the aggregation result.

[0007] According to the third aspect of the embodiments of this specification, a computing device is provided, including a memory, a processor, and a computer program or instructions stored in the memory and executable on the processor, wherein the processor implements the steps of the graph decomposition-based federated graph model training method when executing the computer program or instructions.

[0008] According to a fourth aspect of an embodiment of this specification, a computer-readable storage medium is provided, which stores a computer program or instructions, and when the computer program or instructions are executed by a processor, the steps of the graph decomposition-based federated graph model training method are implemented.

[0009] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-mentioned graph decomposition-based federated graph model training method.

[0010] The graph decomposition-based federated graph model training system provided in this specification includes a server and at least two clients; the server is used to send the initial graph model and the global graph atoms associated with the initial graph model to at least two clients; the target client among the at least two clients is used to combine the graph atom weights and the global graph atoms into global structural information; determine the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atoms and the initial graph model; train the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; send the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server; the server is used to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by at least two clients respectively, and the initial graph model, and determine the target graph model that meets the training stop conditions according to the aggregation results.

[0011] In one embodiment of the present specification, multiple clients hold different local subgraphs respectively, and the local subgraphs held by the clients are decomposed into trainable graph atoms. The clients can obtain an intermediate graph model in combination with the global graph atom training. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client, and performs federated training based on the graph atom gradients and intermediate graph model parameters submitted by each client to obtain a target graph model. During the training of the target graph model, the potential graph node connection relationships in the local subgraphs held by each client are mined to achieve graph model training from a global perspective, enhance the modeling and representation capabilities of the graph model, and efficiently train the model while protecting privacy. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a federated graph model training system based on graph decomposition provided by an embodiment of this specification; Figure 2 It is a structural diagram of a federated graph model training system based on graph decomposition provided by an embodiment of this specification; Figure 3 It is an interactive schematic diagram of a federated graph model training system based on graph decomposition provided by an embodiment of this specification; Figure 4 It is a flowchart of a method for training a federated graph model based on graph decomposition provided in an embodiment of this specification; Figure 5 It is a structural block diagram of a computing device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0013] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0014] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0015] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0016] First, the terms involved in one or more embodiments of this specification are explained.

[0017] Fused Gromov-Wasserstein (FGW) structure reconstructor: is a method for comparison between metric spaces. The FGW method allows the probability distribution and topological structure of nodes in two different metric spaces to be considered simultaneously, which makes it very useful when dealing with complex datasets, especially when it is necessary to model both the feature space and its intrinsic geometric structure.

[0018] Figure 1A schematic diagram of a federated graph model training system based on graph decomposition provided according to an embodiment of the present specification is shown, and the federated graph model training system based on graph decomposition includes a server and at least two clients; the server is used to send an initial graph model and global graph atoms associated with the initial graph model to at least two clients; a target client among the at least two clients is used to combine graph atom weights and global graph atoms into global structural information; determine a similarity matrix corresponding to a local subgraph based on local subgraphs associated with the global graph atoms and the initial graph model; train the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; send the graph atom gradients and intermediate graph model parameters of the intermediate graph model to the server; the server is used to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by at least two clients respectively, and the initial graph model, and determine a target graph model that meets the training stop condition according to the aggregation result.

[0019] During the model training process, multiple clients hold different local subgraphs respectively. The local subgraphs held by the clients are decomposed into trainable graph atoms. The clients can combine the global graph atom training to obtain the intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client, and performs federated training based on the graph atom gradients and intermediate graph model parameters submitted by each client to obtain the target graph model. During the training of the target graph model, the potential graph node connection relationships in the local subgraphs held by each client are mined to achieve graph model training from a global perspective, enhance the modeling and representation capabilities of the graph model, and efficiently train the model while protecting privacy.

[0020] In this specification, a federated graph model training system based on graph decomposition is provided. This specification also involves a federated graph model training method based on graph decomposition, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0021] Figure 2A structural diagram of a federated graph model training system based on graph decomposition provided according to an embodiment of the present specification is shown, and the federated graph model training system 200 based on graph decomposition includes a server 210 and at least two clients 220; the server is used to send an initial graph model and global graph atoms associated with the initial graph model to the at least two clients; a target client among the at least two clients is used to combine graph atom weights and the global graph atoms into global structural information; a similarity matrix corresponding to the local subgraph is determined based on the local subgraph associated with the global graph atom and the initial graph model; the initial graph model is trained into an intermediate graph model based on the global structural information and the similarity matrix, and the graph atom gradient of the global graph atom is calculated; the graph atom gradient and the intermediate graph model parameters of the intermediate graph model are sent to the server; the server is used to perform federated aggregation based on the intermediate graph model parameters and graph atom gradients respectively submitted by the at least two clients, and the initial graph model, and determine the target graph model that meets the training stop condition according to the aggregation result.

[0022] Specifically, the server and the client jointly complete the training of the target graph model based on federated learning. The server provides the client with an initial graph model and global graph atoms. The initial graph model refers to a graph neural network model, which is a federated graph model. The federated model training is performed by the server to collaboratively train the graph neural network. Global graph atoms refer to a learnable graph structure unit that is used to capture and represent global structural information shared among all clients. The server holds at least one global graph atom, or a series of multiple global graph atoms. The graph atom weight is the personalized weight of the client, indicating the relevance and importance of the corresponding atom on the client. The global structural information represents the optimal personalized representation of the global structural information shared by all clients on the target client.

[0023] The global structure information can be calculated using the Fused Gromov-Wasserstein (FGW) structure reconstructor. The local subgraph refers to the private subgraph held by the target client, which can reflect the local graph structure of the target client. The similarity matrix reflects the potential associations and structural similarities between different nodes in the local subgraph held by the target client. If the representations (node ​​embeddings) of two nodes in the embedding space are very close, then their values ​​in the similarity matrix will be high. This similarity can be used to reveal potential connections that are not visible in the local subgraph, even if these connections are not explicitly marked in the original data corresponding to the local subgraph of the target client, but may exist in other clients. In this way, the similarity matrix helps capture more complete local structure information and improve the learning and inference capabilities of the initial graph model. The intermediate graph model refers to the graph model obtained by the client after a round of model training based on the initial graph model. The graph atom gradients are used to update the global graph atoms based on the stochastic gradient descent method, and the updated global graph atoms are used for the next round of graph model training. The target graph model is the model obtained by the server after a round of training on the initial graph model.

[0024] Based on this, a federated graph model training system based on graph decomposition includes a server and at least two clients, and the server holds an initial graph model and global graph atoms associated with the initial graph model. The server sends the initial graph model and the global graph atoms associated with the initial graph model to at least two clients to start model training. At least two clients each hold a personalized graph atom weight. The target client among the at least two clients performs combined calculation based on the graph atom weight and the global graph atom to obtain global structure information, complete global structure reconstruction, and then calculates the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model to complete local structure mining. Based on the global structure information and the similarity matrix, the initial graph model is trained as an intermediate graph model, and the graph atom gradient of the global graph atom is calculated to achieve alignment of the global structure and the local structure. Finally, the intermediate graph model parameters of the intermediate graph model are extracted, and the graph atom gradient and the intermediate graph model parameters of the intermediate graph model are sent to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by at least two clients, as well as the initial graph model, until the target graph model that meets the training stop condition is determined according to the aggregation result.

[0025] For example, in the commodity recommendation scenario, each commodity sales platform can cooperate with the server for federated training, so as to complete the training of the graph model without leaking the private user data and commodity data of the commodity sales platform. The server provides the initial graph model to be trained, as well as the global structure information shared by each client, that is, the global graph atom. The server sends the initial graph model and the global graph atom to each client respectively. Each client performs a combined calculation based on the personalized graph atom weight and the global graph atom to obtain the global structure information, complete the global structure reconstruction, and calculate the local subgraph composed of local user data and local commodity data based on the global graph atom association, and the initial graph model to calculate the similarity matrix corresponding to the local subgraph, and complete the local structure mining. Based on the global structure information and the similarity matrix, the initial graph model is trained as an intermediate graph model, and the graph atom gradient of the global graph atom is calculated. The graph atom gradient and the intermediate graph model parameters of the intermediate graph model are sent to the server. The server performs federated aggregation based on the intermediate graph model parameters and graph atom gradients submitted by at least two clients respectively, and obtains the target graph model that meets the training stop condition.

[0026] In summary, during the model training process, multiple clients hold different local subgraphs respectively. The local subgraphs held by the clients are decomposed into trainable graph atoms. The clients can combine the global graph atom training to obtain the intermediate graph model. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client, and performs federated training based on the graph atom gradients and intermediate graph model parameters submitted by each client to obtain the target graph model. During the training of the target graph model, the potential graph node connection relationships in the local subgraphs held by each client are mined to achieve graph model training from a global perspective, enhance the modeling and representation capabilities of the graph model, and efficiently train the model while protecting privacy.

[0027] Furthermore, the target client can use the reconstructor to construct global structural information. In specific implementation, the target client among the at least two clients is used to determine the graph atom weights, calculate the graph atom embeddings of the global graph atoms; determine the distance parameters, node feature parameters and preset centroid parameters; and use the reconstructor to generate the global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters and the preset centroid parameters.

[0028] Specifically, graph atom embedding refers to the multidimensional embedding vector of global graph atoms. Distance parameter refers to the distance measure of the client to the global graph atoms. Node feature parameter represents the node feature of the center of mass, and preset center of mass parameter is the predefined center of mass. The center of mass contains the collective global structural knowledge. It is the weighted average of at least one global graph atom, minimizing the weighted sum of the differences between each global graph atom and the center of mass. The global structural information is calculated using graph atom weights to obtain a personalized center of mass. Reconstructor refers to the Fused Gromov-Wasserstein (FGW) structure reconstructor.

[0029] Based on this, the target client among the at least two clients determines the graph atom weights held by the target client, calculates the graph atom embeddings of the global graph atoms, determines the distance parameters, the node feature parameters and the preset centroid parameters, and uses a reconstructor to generate global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters and the preset centroid parameters.

[0030] The global structure information can be calculated based on the following formula (1).

[0031]

[0032] in, represents the distance metric, i.e., the distance parameter; Represents the node characteristics of the centroid, that is, the node characteristic parameters, represents the fused Gromov-Wasserstein distance metric. represents the size of the center of mass, which is predefined. The N global graph atoms are represented as , the embedding of N global graph atoms is expressed as ,in, represents the set of real numbers, Represents the first atom in the global graph The number of nodes of atoms, Represents the embedding dimension. Each client Randomly initialize its personalized weights for these atoms, i.e., graph atom weights .

[0033] In summary, the reconstructor is used to generate global structural information based on graph atom weights, graph atom embeddings, distance parameters, node feature parameters and preset centroid parameters to ensure the accuracy of the global structural information.

[0034] Furthermore, in a specific implementation, the target client among the at least two clients is used to determine the local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information.

[0035] Specifically, the node embedding information refers to the multi-dimensional node embedding of the nodes contained in the local subgraph.

[0036] Based on this, the target client among the at least two clients determines the private client data, and constructs a local subgraph associated with the global graph atoms based on the private client data, inputs the local subgraph into the initial graph model, and uses the feature extractor of the initial graph model to generate node embedding information corresponding to the local subgraph. Based on the node embedding information, a similarity matrix corresponding to the local subgraph is calculated.

[0037] Using the above example, Represents the client The private graph data owned by , that is, the local subgraph. The local subgraph contains i nodes. Map to a Dimensional Embedding , get the node embedding information. For the client and , there may be overlapping edges and nodes, i.e. and . and Represent the collection of nodes and edges respectively. Indicates the cardinality of a set. Federated model training trains graph neural networks through server collaboration, and the overall goal is to minimize the global loss function.

[0038] In summary, the feature extractor of the initial graph model is used to generate node embedding information corresponding to the local subgraph, and the accuracy of the node embedding information is determined.

[0039] Furthermore, in a specific implementation, the target client among the at least two clients is used to input the local subgraph into the initial graph model, and use the feature extractor in the initial graph model to extract the node embedding information; determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix and the one-hot vector.

[0040] Specifically, the initial graph model includes a feature extractor and predictor , feature extractor Include The message passing layer takes the input node Map to a Dimensional Embedding The weight matrix refers to the weight matrix corresponding to the number of node categories in the client's local subgraph. The one-hot vector represents the input node and Node label.

[0041] Based on this, the target client of at least two clients inputs the local subgraph into the initial graph model, and uses the feature extractor in the initial graph model to extract the node embedding information. Determine the weight matrix corresponding to the number of node categories in the local subgraph, and the weight matrix representing the input node and The node label one-hot vector is used to calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, weight matrix and one-hot vector. Row and The elements of the column can be calculated based on the following formula (2).

[0042]

[0043] Using the above example, Representation vector The node embedding of all nodes in the local subgraph of the target client is expressed as For a specific pair of nodes within the target client subgraph and , their embedding and The correlation between is given by the Pearson correlation coefficient, which is and The covariance of is divided by the product of their standard deviations. Meanwhile, the class-specific weights are represented by and One-hot vector of node labels and These vectors are then combined with the trainable weight matrix Interaction, where is the number of node categories.

[0044] In practical applications, using this formula, each client can create a similarity matrix to mine the potential but unrevealed connections indicated by node embeddings and integrate the importance of class-specific interactions defined by the weight matrix. In the process of commodity sales, the similarity matrix can be used to identify potential but unrevealed relationships between users (such as friend relationships). For two user nodes in the client's local subgraph and , the degree of similarity between the two nodes can be obtained by calculating the Pearson correlation coefficient between the embedding vectors of the two nodes. The correlation coefficient measures the linear relationship between the embedding vectors. If they have similar favorite items, their embedding vectors may be highly correlated, resulting in a high similarity score. In addition, by utilizing their respective class labels (the class labels of the nodes may represent user groups, such as "students" or "teachers") and a trainable weight matrix , the elements in the similarity matrix We further consider class-specific interactions, for example, the weight matrix may increase the importance of similarities between nodes of the same class. If the value is high, even if the direct connection between the two nodes is not marked in the original data, the similarity matrix can still reveal their potential association, thus providing a basis for relationship discovery and recommendation in commodity sales.

[0045] In summary, the similarity matrix corresponding to the local subgraph is calculated based on the node embedding information, weight matrix and one-hot vector, which fully explores the association relationship between nodes in the local graph atoms, facilitating the subsequent training of the initial graph model.

[0046] Furthermore, after determining the global structural information and the similarity matrix, the alternating optimization strategy can be used to train the initial graph model. In a specific implementation, the target client among the at least two clients is used to determine a local training cycle, construct a first loss function based on the similarity matrix and the global structural information within the local training cycle, and use the predictor in the initial graph model to predict the node score based on the node embedding information; construct a second loss function based on the node score and the category label corresponding to the local subgraph; and train the initial graph model into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function and the second loss function.

[0047] Specifically, the local training cycle refers to the model training cycle in which the client trains the initial graph model. The first loss function is a mapping function, which is the optimal transmission mapping from the local centroid of the global structural information to the local similarity of the similarity matrix. The second loss function can be a cross entropy loss. The second loss function can be constructed based on the predicted node scores obtained by predicting the local subgraph based on the predictor of the initial graph model, and the category labels corresponding to the local subgraph, and calculate the loss value of the model training within the local training cycle to evaluate the prediction ability of the initial graph model. Based on the first loss function, the second loss function and the loss parameter, a target loss function can be constructed to train the initial graph model and obtain an intermediate graph model.

[0048] Based on this, the target client among at least two clients determines a local training cycle, and constructs a first loss function based on the similarity matrix and global structural information during the local training cycle. The predictor in the initial graph model is used to predict the node score based on the node embedding information, and a second loss function is constructed based on the node score and the category label corresponding to the local subgraph. A target loss function is constructed based on the first loss function, the second loss function and the loss parameter, and the initial graph model is trained as an intermediate graph model based on the target loss function, the global graph atoms and the graph atom weights.

[0049] Continuing with the above example, the FGW difference between the local centroid and the local similarity is used as the metric. For the stability of the training process, an alternating optimization strategy is adopted. First, the graph atoms and personalized weights are fixed to generate the local centroid. In each local training cycle of training the local graph neural network model, the difference is added to the universal cross entropy loss to guide local modeling. After the client completes the graph neural network training, it generates the similarity again. Then a new mapping is made between the local centroid and the local similarity, and the global graph atoms and the graph atom weights of the target client are updated.

[0050] Feature Extractor for Initial Graph Model Include The message passing layer takes the input node Map to a Dimensional Embedding Then, the predictor Using node embeddings to get prediction scores , representing the node The unnormalized probability of belonging to each class. Given a class label , the general cross entropy loss is the following formula (3).

[0051]

[0052] In local training cycle During this period, the target client First, fix the received global graph atoms and graph atom weights to train the initial graph model. Use the above formula (1) to generate the fixed local centroid In each local training epoch According to the above formula (2), the local similarity is calculated This is a arrive The optimal transmission mapping of . The loss function of training the initial graph model is as follows (4).

[0053]

[0054] Combining formula (3) and formula (4), the total loss of training the initial graph model can be obtained as the following formula (5).

[0055]

[0056] Among them, t represents the communication round between the server and the client; the server sends the initial graph model and the global graph atoms associated with the initial graph model to the target client, and the target client sends the graph atom gradients and intermediate graph model parameters obtained by processing the initial graph model and the global graph atoms to the server after processing. This process is regarded as one round of communication.

[0057] By optimizing this loss function, the client is able to iteratively compute local similarities and thus improve the node embeddings generated by the local graph neural network.

[0058] In summary, the alternating optimization strategy is adopted and combined with the target loss function to train the initial graph model to enhance the training effect of the initial graph model.

[0059] Furthermore, an alternating optimization strategy is adopted to calculate the graph atom gradients of the global graph atoms after completing the training of the initial graph model. In specific implementation, the target client among the at least two clients is used to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; determine a third loss function having an opposite mapping with the first loss function based on the similarity matrix, generate target graph atom weights based on weight update parameters, the third loss function and the graph atom weights, and calculate the graph atom gradients of the global graph atoms based on the weight update parameters, the third loss function, the global graph atoms and the graph atom embedding.

[0060] Specifically, the intermediate similarity matrix is ​​generated based on the intermediate graph model and the local subgraph, and the generation method is the same as the generation method of the similarity matrix. The third loss function has a loss function with an opposite mapping compared to the second loss function. The target graph atom weight is obtained after the graph atom weight is updated.

[0061] Based on this, the target client among at least two clients calculates the intermediate similarity matrix corresponding to the local graph atoms based on the local graph atoms and the intermediate graph model. Based on the similarity matrix, a third loss function with an inverse mapping with the first loss function is determined, and the graph atom weights are updated based on the weight update parameters, the third loss function and the graph atom weights to obtain the target graph atom weights, and the graph atom gradients of the global graph atoms are calculated based on the weight update parameters, the third loss function, the global graph atoms and the graph atom embedding. The target graph atom weights and the graph atom gradients are used for federated training on the server side.

[0062] Following the above example, after completing the local graph modeling and obtaining the intermediate graph model, the target client The local graph neural network, i.e. the intermediate graph model, is fixed, and the global graph atoms and graph atom weights are optimized. The intermediate graph model obtained through training generates new local similarity based on the local subgraph. The inverse mapping of formula (4) is determined, that is, the following formula (6).

[0063]

[0064] Based on the above formula (6), the weights of the global graph atoms and the target client graph atoms are optimized. The target graph atom weight is calculated by Directly calculated, the atomic gradient of the graph is based on The calculated graph atomic gradients are transmitted to the server for federated processing, where: is a constant parameter.

[0065] In summary, after completing the training of the initial graph model, the alternating optimization strategy is used to calculate the graph atom gradients of the global graph atoms and update the graph atom weights, so that the subsequent server and client can continue to train the graph model based on the target graph atom weights and the global graph atoms.

[0066] Furthermore, the server, as an important role in the federated training of the initial graph model, needs to combine the submissions of at least two clients. In the specific implementation, the server is used to determine the proportion of graph samples of each client among the at least two clients, and to federate the initial graph model based on the proportion of graph samples of each client and the intermediate graph model parameters, and determine the target graph model according to the model aggregation result.

[0067] Specifically, the graph sample ratio refers to the ratio of the local subgraphs held by the target client to the total local subgraphs held by at least two clients. The target graph model refers to the updated graph model obtained by the server after federating the initial graph model based on the intermediate graph model parameters submitted by at least two clients.

[0068] Based on this, the server determines the proportion of graph samples of each of at least two clients. After receiving the intermediate graph model parameters submitted by each of at least two clients, the server performs federated aggregation on the initial graph model based on the graph sample proportion of each client and the submitted intermediate graph model parameters, and determines the target graph model based on the model aggregation result.

[0069] Following the above example, after receiving the intermediate graph model parameters submitted by each client, the server determines the ratio of the local subgraphs held by each client to the total local subgraphs held by at least two clients, and uses this ratio as the graph sample ratio. Based on the intermediate graph model parameters and graph sample ratios, the initial graph model is federated and aggregated to obtain the target graph model. The initial graph model is aggregated as ,in, Indicates the sample weight of the graph.

[0070] In summary, the server performs federated aggregation on the initial graph model based on the proportion of graph samples of each client and the submitted intermediate graph model parameters, determines the target graph model according to the model aggregation results, and completes the graph model training while protecting privacy.

[0071] Furthermore, considering that it is difficult for the server to obtain a target graph model with better prediction ability through one round of graph model training, the server needs to perform multiple rounds of graph model training in combination with at least two clients to improve the prediction ability of the target graph model. In specific implementation, when the target graph model does not meet the training stop condition, the server is used to determine the graph atom gradients submitted by each of the at least two clients, and update the global graph atom to the target global graph atom based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients; use the target global graph atom as the global graph atom, use the target graph model as the initial graph model, and execute the step of sending the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients.

[0072] Based on this, the training stop condition can be the training round, training time, prediction accuracy of the initial graph model after one round of training and other conditions. When the target graph model does not meet the training stop condition, it means that the intermediate graph model obtained by the client training cannot be put into use, and the server needs to continue the federated training of the graph model to update the intermediate graph model. The server determines the graph atom gradients submitted by each of the at least two clients, and updates the global graph atoms to the target global graph atoms based on the stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients. The server can aggregate the graph atom gradients of the global graph atoms submitted by each client, take the target global graph atom as the global graph atom, take the target graph model as the initial graph model, and return to execute the step of sending the initial graph model and the global graph atoms associated with the initial graph model to at least two clients, and perform the next round of graph model training based on the target global graph atom and the target graph model until the training obtains the target graph model that meets the training stop condition.

[0073] Using the above example, the server receives the graph atomic gradients submitted by each client, and performs federated aggregation on the graph atomic gradients submitted by each client. The aggregation is . The graph atom gradient obtained by aggregation is used by the optimizer included in the initial graph model to update the global graph atom held by the server using the gradient descent method to obtain the target global graph atom. The target graph model and the target global graph atom can be sent to each client as the initial graph model and global graph atom for the next round of graph model training. Among them, the stochastic gradient descent can be used as the following formula (7).

[0074]

[0075] in, represents the learning rate, and t represents the communication round between the server and the client.

[0076] In summary, by aggregating the graph atom gradients submitted by each client, the server can communicate with the client in the next round based on the target global graph atom and the target graph model. After multiple rounds of communication, the training obtains a graph model that meets the training stop conditions, thereby improving the prediction ability of the graph model.

[0077] In one embodiment of the present specification, multiple clients hold different local subgraphs respectively, and the local subgraphs held by the clients are decomposed into trainable graph atoms. The clients can obtain an intermediate graph model in combination with the global graph atom training. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client, and performs federated training based on the graph atom gradients and intermediate graph model parameters submitted by each client to obtain a target graph model. During the training of the target graph model, the potential graph node connection relationships in the local subgraphs held by each client are mined to achieve graph model training from a global perspective, enhance the modeling and representation capabilities of the graph model, and efficiently train the model while protecting privacy.

[0078] The following combination Figure 3 , taking the application of the federated graph model training system based on graph decomposition provided in this specification in the social graph model as an example, the federated graph model training system based on graph decomposition is further described. Among them, Figure 3 An interactive schematic diagram of a federated graph model training system based on graph decomposition provided in an embodiment of this specification is shown, which specifically includes the following steps: Step 302: The server sends the initial social graph model and global graph atoms associated with the initial social graph model to at least two clients.

[0079] The global graph atom G is decentralized among K clients, that is, ,in, Represents the client Private graph data owned by the client. and , there may be overlapping edges and nodes, i.e. and . and Represent the collection of nodes and edges respectively. Indicates the cardinality of a set. Federated graph model training is done by co-training graph neural networks on the server.

[0080] Step 304: The target client among at least two clients determines the graph atom weights, calculates the graph atom embeddings of the global graph atoms; determines the distance parameters, the node feature parameters and the preset centroid parameters; and generates the global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters and the preset centroid parameters using a reconstructor.

[0081] The target client combines the global graph atoms and personalized graph atom weights in the global structure reconstruction module and uses the Fused Gromov-Wasserstein (FGW) structure reconstructor to calculate a personalized centroid. This centroid represents the optimal personalized representation of the global structure information shared by all clients on this client.

[0082] Step 306: The target client inputs the private graph data into the initial social graph model, and uses the feature extractor in the initial social graph model to extract the node embedding information; determines the weight matrix and one-hot vector of the private graph data, and calculates the similarity matrix corresponding to the private graph data based on the node embedding information, the weight matrix and the one-hot vector.

[0083] The target client trains the local graph neural network model received from the server in the local structure exploration module, and generates a local similarity matrix, i.e., similarity matrix, through the graph generator. The generator uses the node embeddings calculated from the feature extractor of the local graph neural network model to construct a similarity matrix reflecting the potential associations between nodes, further revealing and mining the implicit structural relationships in the client subgraph.

[0084] Step 308: The target client determines a local training cycle, constructs a first loss function based on the similarity matrix and global structure information within the local training cycle, and uses the predictor in the initial social graph model to predict node scores based on node embedding information; constructs a second loss function based on the node scores and category labels corresponding to the private graph data; and trains the initial social graph model into an intermediate social graph model based on global graph atoms, graph atom weights, the first loss function, and the second loss function.

[0085] Step 310: The target client generates an intermediate similarity matrix based on local graph atoms and the intermediate social graph model; determines a third loss function having an inverse mapping with the first loss function based on the similarity matrix, generates target graph atom weights based on weight update parameters, the third loss function and graph atom weights, and calculates graph atom gradients of global graph atoms based on weight update parameters, the third loss function, global graph atoms and graph atom embeddings.

[0086] The target client calculates the FGW difference between the personalized centroid and the local similarity in the global-local structure alignment module, and alternately updates the local graph neural network and the global graph atoms and personalized graph atom weights.

[0087] Step 312: At least two clients send graph atom gradients and intermediate social graph model parameters to the server respectively.

[0088] The updated graph neural network parameters and the gradients of graph atoms are transmitted to the server for aggregation.

[0089] Step 314: When the target social graph model does not meet the training stop condition, the server updates the global graph atom to the target global graph atom based on stochastic gradient descent and the graph atom gradients submitted by each of at least two clients; uses the target global graph atom as the global graph atom, uses the target social graph model as the initial social graph model, and executes step 302.

[0090] The updated graph neural network parameters and the gradients of graph atoms are transmitted to the server for aggregation. This communication between the server and the client is iterated until the performance converges and a trained social graph model is obtained.

[0091] This embodiment provides a federated graph model training system based on graph decomposition, which decomposes the client's graph into trainable graph atoms, thereby utilizing global graph atoms while protecting the privacy of graph data to improve the modeling and representation capabilities of the federated graph model.

[0092] Corresponding to the above system embodiment, this specification also provides an embodiment of a federated graph model training method based on graph decomposition. Figure 4 FIG. 1 shows a flowchart of a method for training a federated graph model based on graph decomposition provided by an embodiment of this specification. Figure 4 As shown, the method for training a federated graph model based on graph decomposition includes a server and at least two clients, and specifically includes the following steps: Step 402: the server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; Step 404: The target client among the at least two clients combines the graph atom weights and the global graph atoms into global structural information; determines a similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; trains the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculates the graph atom gradient of the global graph atom; sends the graph atom gradient and the intermediate graph model parameters of the intermediate graph model to the server; Step 406: The server performs federated aggregation based on the intermediate graph model parameters and graph atomic gradients respectively submitted by the at least two clients, and the initial graph model, and determines a target graph model that meets the training stop condition according to the aggregation result.

[0093] Optionally, the target client among the at least two clients is used to determine the graph atom weights, calculate the graph atom embeddings of the global graph atoms; determine the distance parameters, the node feature parameters and the preset centroid parameters; and use a reconstructor to generate the global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters and the preset centroid parameters.

[0094] Optionally, the target client among the at least two clients is used to determine a local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate a similarity matrix corresponding to the local subgraph based on the node embedding information.

[0095] Optionally, the target client among the at least two clients is used to input the local subgraph into the initial graph model, and use the feature extractor in the initial graph model to extract node embedding information; determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix and the one-hot vector.

[0096] Optionally, the target client among the at least two clients is used to determine a local training cycle, construct a first loss function based on the similarity matrix and the global structural information within the local training cycle, and use the predictor in the initial graph model to predict node scores based on the node embedding information; construct a second loss function based on the node scores and category labels corresponding to the local subgraph; and train the initial graph model into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function and the second loss function.

[0097] Optionally, the target client among the at least two clients is used to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; determine a third loss function having an inverse mapping with the first loss function based on the similarity matrix, generate target graph atom weights based on weight update parameters, the third loss function and the graph atom weights, and calculate the graph atom gradients of the global graph atoms based on the weight update parameters, the third loss function, the global graph atoms and the graph atom embedding.

[0098] Optionally, the server is used to determine the proportion of graph samples of each client among the at least two clients, and federally aggregate the initial graph model based on the proportion of graph samples of each client and the intermediate graph model parameters, and determine the target graph model according to the model aggregation result.

[0099] Optionally, when the target graph model does not meet the training stop condition, the server is used to determine the graph atom gradients submitted by each of the at least two clients, and update the global graph atom to the target global graph atom based on stochastic gradient descent and the graph atom gradients submitted by each of the at least two clients; use the target global graph atom as the global graph atom, use the target graph model as the initial graph model, and execute the step of sending the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients.

[0100] In one embodiment of the present specification, multiple clients hold different local subgraphs respectively, and the local subgraphs held by the clients are decomposed into trainable graph atoms. The clients can obtain an intermediate graph model in combination with the global graph atom training. The server receives the graph atom gradients and intermediate graph model parameters submitted by each client, and performs federated training based on the graph atom gradients and intermediate graph model parameters submitted by each client to obtain a target graph model. During the training of the target graph model, the potential graph node connection relationships in the local subgraphs held by each client are mined to achieve graph model training from a global perspective, enhance the modeling and representation capabilities of the graph model, and efficiently train the model while protecting privacy.

[0101] The above is a schematic scheme of a federated graph model training method based on graph decomposition of this embodiment. It should be noted that the technical scheme of the federated graph model training method based on graph decomposition and the technical scheme of the federated graph model training system based on graph decomposition belong to the same concept. For details not described in detail in the technical scheme of the federated graph model training method based on graph decomposition, please refer to the description of the technical scheme of the federated graph model training system based on graph decomposition.

[0102] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0103] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0104] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0105] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 may also be a mobile or stationary server.

[0106] Among them, when the processor 520 executes the computer program or instructions, it implements the steps of the graph decomposition-based federated graph model training method.

[0107] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned federated graph model training system based on graph decomposition belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the above-mentioned federated graph model training system based on graph decomposition.

[0108] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned method for training a federated graph model based on graph decomposition.

[0109] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned federated graph model training system based on graph decomposition belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned federated graph model training system based on graph decomposition.

[0110] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned graph decomposition-based federated graph model training method.

[0111] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned federated graph model training system based on graph decomposition belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the above-mentioned federated graph model training system based on graph decomposition.

[0112] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0113] The computer program or instruction includes a computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0114] It should be noted that, for the convenience of description, the aforementioned method embodiments are all described as a series of action combinations, but those skilled in the art should be aware that this specification is not limited by the order of the actions described, because according to this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this specification.

[0115] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0116] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A federated graph model training system based on graph decomposition, characterized in that: Includes a server and at least two clients; The server is used to send the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients is used to combine the graph atom weights and the global graph atoms into global structural information; determine the similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; train the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculate the graph atom gradients of the global graph atoms; send the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server; The server is used to perform federated aggregation based on the intermediate graph model parameters and graph atomic gradients respectively submitted by the at least two clients, and the initial graph model, and determine a target graph model that meets the training stop condition according to the aggregation result.

2. The federated graph model training system based on graph decomposition according to claim 1, characterized in that: The target client among the at least two clients is used to determine the graph atom weights, calculate the graph atom embeddings of the global graph atoms; determine the distance parameters, the node feature parameters and the preset centroid parameters; and use a reconstructor to generate the global structural information based on the graph atom weights, the graph atom embeddings, the distance parameters, the node feature parameters and the preset centroid parameters.

3. The graph decomposition-based federated graph model training system according to claim 1, characterized in that: The target client among the at least two clients is used to determine the local subgraph associated with the global graph atom, input the local subgraph into the initial graph model, obtain node embedding information, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information.

4. The federated graph model training system based on graph decomposition according to claim 3 is characterized in that: The target client among the at least two clients is used to input the local subgraph into the initial graph model, and use the feature extractor in the initial graph model to extract node embedding information; determine the weight matrix and one-hot vector of the local subgraph, and calculate the similarity matrix corresponding to the local subgraph based on the node embedding information, the weight matrix and the one-hot vector.

5. The federated graph model training system based on graph decomposition according to claim 3, characterized in that: The target client among the at least two clients is used to determine a local training cycle, construct a first loss function based on the similarity matrix and the global structure information within the local training cycle, and predict node scores based on the node embedding information using a predictor in the initial graph model; A second loss function is constructed based on the node score and the category label corresponding to the local subgraph; and the initial graph model is trained into an intermediate graph model based on the global graph atoms, the graph atom weights, the first loss function and the second loss function.

6. The federated graph model training system based on graph decomposition according to claim 5, characterized in that: A target client among the at least two clients is used to generate an intermediate similarity matrix based on the local graph atoms and the intermediate graph model; A third loss function having an inverse mapping with the first loss function is determined based on the similarity matrix, a target graph atom weight is generated based on a weight update parameter, the third loss function and the graph atom weight, and a graph atom gradient of the global graph atom is calculated based on the weight update parameter, the third loss function, the global graph atoms and the graph atom embedding.

7. The graph decomposition-based federated graph model training system according to claim 1, characterized in that: The server is used to determine the proportion of graph samples of each client among the at least two clients, and to perform federated aggregation on the initial graph model based on the proportion of graph samples of each client and the intermediate graph model parameters, and to determine the target graph model according to the model aggregation result.

8. The graph decomposition-based federated graph model training system according to claim 1, characterized in that: In the case where the target graph model does not satisfy the training stop condition, the server is used to determine the graph atom gradient submitted by each of the at least two clients, and update the global graph atom to the target global graph atom based on stochastic gradient descent and the graph atom gradient submitted by each of the at least two clients; The target global graph atom is used as the global graph atom, the target graph model is used as the initial graph model, and the step of sending the initial graph model and the global graph atom associated with the initial graph model to the at least two clients is performed.

9. A method for training a federated graph model based on graph decomposition, characterized in that: Includes a server and at least two clients; The server sends the initial graph model and the global graph atoms associated with the initial graph model to the at least two clients; The target client among the at least two clients combines the graph atom weights and the global graph atoms into global structural information; determines a similarity matrix corresponding to the local subgraph based on the local subgraph associated with the global graph atom and the initial graph model; trains the initial graph model into an intermediate graph model based on the global structural information and the similarity matrix, and calculates the graph atom gradients of the global graph atoms; sends the graph atom gradients and the intermediate graph model parameters of the intermediate graph model to the server; The server performs federated aggregation based on the intermediate graph model parameters and graph atomic gradients respectively submitted by the at least two clients and the initial graph model, and determines a target graph model that meets the training stop condition according to the aggregation result.

10. A computing device comprising a memory, a processor, and a computer program or instruction stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program or instructions, the steps of the method according to claim 9 are implemented.

11. A computer-readable storage medium storing a computer program or instruction, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to claim 9 are implemented.

12. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to claim 9 are implemented.

Images