Federal learning differential privacy graph neural network system and training method

Abstract

The invention relates to a federated learning differential privacy graph neural network system and a training method, and belongs to the field of artificial intelligence information security. The system is composed of a server and M client devices which are in data communication with each other through network connection. The method comprises the following steps: S1, initializing federal learning parameters; s2, initializing hierarchical pooling graph neural network model parameters; s3, sending the model parameters to the client device; s4, the client side calculates federal learning gradient data; s5, sending the federal learning gradient data to a server; s6, the server calculates new model parameters; and S7, repeating the steps S3-S8 until the maximum number of iterations is reached. According to the method, the representation capability is enhanced by introducing a hierarchical pooling strategy, the learning rate is dynamically adjusted by performing fusion and correction of first-order and second-order moment estimation on the gradient, the adaptability and convergence speed of the GNNs are remarkably improved under a federated learning framework, and the robustness of the GNNs to noise and class imbalance is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to a kind of federal learning differential privacy's graph neural network system and training method, belong to artificial intelligence information security field, especially suitable for a kind of federal learning differential privacy's graph neural network and training. BACKGROUND

[0002] As a powerful tool for processing non-Euclidean space data (such as graph structure data), graph neural networks (GNNs) have completely changed our way of processing data. Unlike traditional models that process vectorized data, GNNs can directly utilize node features and underlying graph topology through a message passing mechanism. In this mechanism, nodes are updated iteratively based on information exchanged along graph edges. This unique design enables GNNs to capture structural dependencies, thus excelling in tasks where relational data plays a key role. GNNs have made breakthrough progress in applications such as social network analysis, financial systems, biological structure modeling, and recommendation systems.

[0003] Despite their remarkable capabilities, GNNs face significant challenges, particularly in terms of data privacy. Since GNNs often handle highly sensitive data, such as brain connectivity graphs generated from functional magnetic resonance imaging data in medical health applications, these data contain personally identifiable medical information. Graph structure data provides richer contextual information than standard Euclidean data, including direct node interactions, community structures, and global topological patterns, which can be exploited by malicious actors.

[0004] Differential privacy (DP) as a strict privacy protection mechanism has been widely adopted to address privacy issues. DP ensures that the inclusion or exclusion of any single data point has a negligible and mathematically bounded impact on the model output, thereby preventing attackers from inferring sensitive information. DP has been successful in various machine learning domains such as medical image analysis, natural language processing, and reinforcement learning. However, the application of DP in graph neural networks has been relatively less explored, mainly due to the unique challenges posed by graph structures.

[0005] Mueller et al. successfully combined the differential privacy of stochastic gradient descent with GNNs for graph classification tasks to protect model parameters. They also proposed techniques such as adaptive optimization strategies based on this architecture to increase the learning rate. However, stochastic gradient descent has inherent limitations, such as unstable training and slow convergence. Advanced optimizers (such as Adam and Ranger) have shown superior performance compared to stochastic gradient descent. They use adaptive moment estimation to improve convergence speed and training stability. However, their direct integration with DP is not easy. The main challenge lies in the need to add noise to the gradient update under DP constraints. This noise destroys the second-order moment estimation that is critical to optimizers such as Adam, resulting in update bias and degraded optimization performance. Although researchers have tried to alleviate these problems by adjusting the noise distribution per iteration, its effectiveness in GNNs has not been widely verified, where the dependencies between nodes further exacerbate noise propagation. Summary of the Invention

[0006] In view of this, in order to address the privacy protection issues of graph neural networks with multi-party participation in training and learning when modeling structured data, and to solve the shortcomings of unstable training and slow convergence brought by the existing differential privacy stochastic gradient descent method, the present invention proposes a federated learning differential privacy graph neural network system and training method, which aims to reduce the bias introduced by noise based on the bias correction of the Ranger optimizer, and introduce a hierarchical pooling strategy to enhance the representation ability, perform moment estimation on the gradient, thereby realizing dynamic adjustment of the learning rate, and achieving improvements in the classification accuracy and convergence speed performance of GNNs under the federated learning framework.

[0007] The present invention provides the following technical solutions:

[0008] A federated learning differentially private graph neural network system consists of a server and M client devices, which communicate data with each other through a network connection; the server includes: a server data storage module and a server federated learning module; one of the client devices includes: a client data storage module and a client federated learning module; the server federated learning module and the client federated learning module respectively store a hierarchical pooling graph neural network model with exactly the same structure: where M is a positive integer greater than 1.

[0009] Preferably, the server data storage module is connected to the client data storage module, and is used to receive and store the federated learning gradient data sent by the client data storage module in the client device, and send the model parameters in the server to the client data storage module to update the model parameters of the client.

[0010] Preferably, the server federated learning module is connected to the server data storage module, updates the model parameters in the server according to the federated learning gradient data, and stores them in the server data storage module.

[0011] Preferably, the client data storage module is used to locally store the user's historical data, the client's model parameters and the federated learning gradient data.

[0012] Preferably, the client federated learning module is connected to the client data storage module, and uses the user's historical data and the client's model parameters in the client data storage module to calculate the federated learning gradient data and store it in the server data storage module.

[0013] Furthermore, the hierarchical pooling graph neural network is composed of multiple stacked layers, each layer contains two modules: a graph neural network layer (GNN) and a graph pooling layer connected in series. A readout layer is used between the layers to achieve feature fusion of the outputs of all layers, and finally a multi-layer perception layer (MLP) is used to achieve the classification output of the results.

[0014] Different from the existing hierarchical pooling network structure, the hierarchical pooling method of the present invention does not adopt a linear assignment matrix strategy, but adopts a nonlinear K-nearest neighbor strategy, which makes the expression more accurate.

[0015] Furthermore, for the user's historical data graph G = (A, X), the output of the i-th stacked graph convolution layer of the layered pooling graph neural network is: The output of the graph pooling layer is: H i+1 =H i [idx,:]; where idx=Top k (X i ), Top k (·) is the index function of the k nearest neighbor nodes, is a hyperparameter, graph feature matrix and the adjacency matrix is the weight parameter of the i-th stack, I is the identity matrix, and the degree matrix diag(·) is the diagonalization operation, d is the dimension of the feature vector, H1=X, n is the number of graph nodes, and i≥1.

[0016] Furthermore, the readout layer uses sorting and aggregation operations to maintain the relative order of node features at each level, and then connects the node features of all levels through Concat to form the final node features as the input of the multi-layer perception layer.

[0017] A federated learning differentially private graph neural network training method includes the following steps:

[0018] S1: The server federated learning module in the server initializes the federated learning parameters and stores them in the server data storage module;

[0019] S2: The server initializes the parameters of the layered pooling graph neural network model and stores them in the server data storage module;

[0020] S3: The server data storage module sends the layered pooling graph neural network model parameters to all client devices and stores them in the client data storage module;

[0021] S4: The client uses the user's historical data in the client data storage module to calculate the federated learning gradient data of the layered pooling graph neural network model and stores it in the client data storage module;

[0022] S5: The client data storage module sends the federated learning gradient data to the server data storage module;

[0023] S6: The server federated learning module uses the federated learning gradient data from the server data storage module to calculate the parameters of the new layered pooling graph neural network model and stores it in the server data storage module.

[0024] S7: Repeat steps S3 to S8 until the maximum number of iterations, at which point all client devices have completed the training of the layered pooling graph neural network model.

[0025] Furthermore, the federated learning parameters described in step S1 include: the maximum number of iterations t max , learning rate warm-up iteration number t u , learning rate cooling iteration number t d , gradient norm upper bound C, initial first-order moment estimate m0, initial second-order moment estimate v0, maximum second-order moment estimate v max , learning rate η, weight decay coefficient λ, noise standard deviation σ, decay rates β0, β1, β2, β a , update frequency Sampling volume L, minimum numerical error ε; where, t max , t u , t d 、 L is a positive integer, v max ≥v0,t d >t u , η, λ, β0, β1, β2, β a ∈[0, 1].

[0026] Furthermore, the step S3 is updated according to the set frequency. Regularly revise the parameters of the layered pooling graph neural network model, that is, when the tth iteration update satisfies When Correcting the parameters of the layered pooling graph neural network model Where ← represents assignment and p is a positive integer.

[0027] Furthermore, the step S4 is specifically as follows:

[0028] S401: The client randomly samples L historical data from the user's historical data with equal probability as a sample set D;

[0029] S402: The client federated learning module uses the sample set to calculate the original gradient of the layered pooling graph neural network model; wherein, at the tth iteration, any i-th historical data x i The original gradient is denoted as g i (x i );

[0030] S403: The client federated learning module clips the original gradient of the sample set to obtain the clipped gradient Among them, ||·||2 is the two-norm;

[0031] S404: The client federated learning module adds noise to the clipped gradient to obtain federated learning gradient data Where N(0,σ 2 C 2 ) has a mean of 0 and a variance of o σ2 C 2 Normally distributed noise.

[0032] Furthermore, for the t-th iteration update, step S6 is specifically as follows:

[0033] S601: The server federated learning module calculates the first-order moment estimate using the federated learning gradient data stored in the server data storage module and second-order moment estimates in Federated learning gradient data calculated for all clients separately The mean of

[0034] S602: The server federated learning module updates the maximum second-order moment estimate v max =max(v t , v max );

[0035] S603: The server federated learning module calculates the first-order moment bias correction and second-order moment bias correction

[0036] S604: The server federated learning module calculates the deviation vector

[0037] S605: The server federated learning module calculates the current learning rate

[0038] S606: The server federated learning module calculates weight decay Among them, θ t-1 is the layered pooling graph neural network model parameter of the previous iteration, and mean(·) is the mean operation;

[0039] S607: The server federated learning module calculates the tth iteration to update the layered pooling graph neural network model parameters θ t =θ t-1 -η t ·u t -η t ·d t .

[0040] The beneficial effects of the present invention are: the present invention provides a federated learning differentially private graph neural network system and training method, expands the Ranger optimizer and differential privacy method of the first-order gradient benchmark, and introduces a hierarchical pooling strategy to enhance the representation capability. By fusing the first-order and second-order moment estimates of the gradient and correcting them, the learning rate is dynamically adjusted, which significantly improves the adaptability and convergence speed of GNNs under the federated learning framework and enhances the robustness of GNNs to noise and class imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to illustrate the purpose and technical solution of the present invention, the present invention provides the following drawings:

[0042] Figure 1 This is a framework diagram of federated learning in Example 1 of the present invention;

[0043] Figure 2 This paper presents a structural diagram of a graph neural network system with federated learning and differential privacy;

[0044] Figure 3 This is a flow chart of a graph neural network training method for federated learning with differential privacy according to the present invention;

[0045] Figure 4 This is a framework diagram of the layered pooling graph neural network model of Example 1 of the present invention. DETAILED DESCRIPTION

[0046] Example 1: In order to achieve high-precision and efficient training of graph neural networks with the participation of multiple parties while ensuring user privacy protection, considering the complexity of ciphertext calculation, the following method can usually be used: Figure 1The federated learning framework shown. Furthermore, to address the shortcomings of existing graph neural network training accuracy, this embodiment intends to improve the accuracy of graph neural networks within a federated learning framework. The present invention provides a “graph neural network system and training method with federated learning differential privacy.”

[0047] like Figure 2 As shown, a federated learning differentially private graph neural network system is composed of a server (1) and M client devices (2), which communicate data with each other through a network connection; the server (1) includes: a server data storage module (11), a server federated learning module (12); one of the client devices (2) includes: a client data storage module (21), a client federated learning module (22); the server federated learning module (12) and the client federated learning module (22) respectively store a hierarchical pooling graph neural network model with exactly the same structure; wherein M is a positive integer greater than 1.

[0048] The server data storage module (11) is connected to the client data storage module (21) and is used to receive and store the federated learning gradient data sent by the client data storage module (21) in the client device (2), and send the model parameters in the server (1) to the client data storage module (21) to update the model parameters of the client.

[0049] The server federated learning module (12) is connected to the server data storage module (11), updates the model parameters in the server (1) according to the federated learning gradient data, and stores the updated data in the server data storage module (11).

[0050] The client data storage module (21) is used to locally store the user's historical data, the client's model parameters and the federated learning gradient data.

[0051] The user's historical data comes from data collection equipment.

[0052] The client federated learning module (22) is connected to the client data storage module (21), and uses the user's historical data in the client data storage module (21) and the client's model parameters to calculate the federated learning gradient data and store it in the server data storage module (11).

[0053] Further, if Figure 4 As shown in the figure, the hierarchical pooling graph neural network consists of three stacked layers. Each layer contains two modules, a graph neural network layer and a graph pooling layer, which are connected in series. A readout layer is used between the layers to achieve feature fusion of the outputs of all layers. Finally, a multi-layer perception layer is used to achieve the classification output of the results.

[0054] Furthermore, for the user's historical data graph G = (A, X), the output of the i-th stacked graph convolution layer of the layered pooling graph neural network is: The output of the graph pooling layer is: H i+1 =H i [idx,:]; where idx=Top k (X i ), Top k (·) is the index function of the k nearest neighbor nodes, is a hyperparameter, graph feature matrix and the adjacency matrix is the weight parameter of the i-th stack, I is the identity matrix, and the degree matrix diag(·) is the diagonalization operation, d is the dimension of the feature vector, H1=X, n is the number of graph nodes, and i≥1.

[0055] Furthermore, the readout layer uses sorting and aggregation operations to maintain the relative order of node features at each level, and then connects the node features of all levels through Concat to form the final node features as the input of the multi-layer perception layer.

[0056] A federated learning differential privacy graph neural network training method, combined with Figure 3 , including the following steps:

[0057] S1: The server federated learning module (12) in the server (1) initializes the federated learning parameters and stores them in the server data storage module (11).

[0058] Furthermore, the federated learning parameters described in step S1 include: the maximum number of iterations t max , learning rate warm-up iteration number t u , learning rate cooling iteration number t d , gradient norm upper bound C, initial first-order moment estimate m0, initial second-order moment estimate v0, maximum second-order moment estimate v max , learning rate η, weight decay coefficient λ, noise standard deviation σ, decay rates β0, β1, β2, β a , update frequency Sampling volume L, minimum numerical error ε; where, t max , t u , t d 、 L is a positive integer, v max ≥v0,t d >t u , η, λ, β0, β1, β2, β a ∈[0,1].

[0059] In this embodiment, t is selected respectively. max =1000, t u =220, C=1, t d =280, L=100, σ=1 / L, m0=0, v max =0, v0=0, η, λ=1e-4, β0=0.9, β1=0.9, β2=0.99, β a =0.5,ε=1e-8.

[0060] S2: The server (1) initializes the parameters of the layered pooling graph neural network model and stores them in the server data storage module (11).

[0061] S3: The server data storage module (11) sends the layered pooling graph neural network model parameters to all client devices (2) and stores them in the client data storage module (21).

[0062] According to the set update frequency Regularly revise the parameters of the layered pooling graph neural network model, that is, when the tth iteration update satisfies When Correcting the parameters of the layered pooling graph neural network model Where ← represents assignment, p = 1, 2, ... 200.

[0063] S4: The client (2) calculates the federated learning gradient data of the layered pooling graph neural network model using the user's historical data in the client data storage module (21), and stores it in the client data storage module (21).

[0064] Furthermore, the step S4 is specifically as follows:

[0065] S401: The client randomly samples L historical data from the user's historical data with equal probability as a sample set D;

[0066] S402: The client federated learning module uses the sample set to calculate the original gradient of the layered pooling graph neural network model; wherein, at the tth iteration, any i-th historical data x i The original gradient is denoted as g t (x i );

[0067] S403: The client federated learning module clips the original gradient of the sample set to obtain the clipped gradient Among them, ||·||2 is the two-norm;

[0068] S404: The client federated learning module adds noise to the clipped gradient to obtain federated learning gradient data Where N(0,σ 2 C 2 ) has a mean of 0 and a variance of σ 2 C 2 Normally distributed noise.

[0069] S5: The client data storage module (21) sends the federated learning gradient data to the server data storage module (11).

[0070] S6: The server federated learning module (12) uses the federated learning gradient data of the server data storage module (11) to calculate the parameters of the new layered pooling graph neural network model and stores it in the server data storage module (11).

[0071] Furthermore, iterative updating is performed starting from t=1. For the t-th iterative updating, step S6 is specifically as follows:

[0072] S601: The server federated learning module calculates the first-order moment estimate using the federated learning gradient data stored in the server data storage module and second-order moment estimates in Federated learning gradient data calculated for all clients separately The mean of

[0073] S602: The server federated learning module updates the maximum second-order moment estimate v max =max(v t , v max );

[0074] S603: The server federated learning module calculates the first-order moment bias correction and second-order moment bias correction

[0075] S604: The server federated learning module calculates the deviation vector

[0076] S605: The server federated learning module calculates the current learning rate

[0077] S606: The server federated learning module calculates weight decay Where θt-1 is the model parameter of the layered pooling graph neural network in the previous iteration, and mean(·) is the mean operation;

[0078] S607: The server federated learning module calculates the tth iteration to update the layered pooling graph neural network model parameters θ t =θ t-1 -η t ·u t-η t ·d t .

[0079] S7: Repeat steps S3 to S8 until the maximum number of iterations t = t max ,At this point, all client devices (2) have completed the training of the layered pooling graph neural network model.

[0080] Example 2: In order to better demonstrate the beneficial effects of the method of the present invention, this example will conduct comparative experiments based on five existing biomedical graph datasets (AIDS[1], Fingerprints[2], Molbace[1], Molbbbp[3], and Proteins[4]). The specific dataset features are shown in Table 1. These five datasets all represent complex structures (compound molecules, fingerprint images, proteins) as graph structures (nodes represent atoms / fingerprint details / amino acids, and edges represent chemical bonds / spatial relationships / spatial proximity). Their purpose is to complete classification tasks: AIDS and Molbace predict the ability of molecules to resist HIV or inhibit HIV replication, Molbbbp predicts the biological activity of molecules to penetrate the blood-brain barrier, Proteins predicts whether a protein is an enzyme, and Fingerprints is used for identity recognition. For this type of classification task, graph neural networks (GNNs) are ideal tools for processing this type of graph structure data. They can effectively learn the structural patterns and relationships contained in the graph. Given the high sensitivity of this data (involving personal medical information, unique biometrics, drug molecular structure, and protein function research), it is crucial to apply differential privacy (DP) technology when training GNN models. It can train models while protecting the privacy of original data, which is crucial for promoting secure medical research, drug discovery, and identity authentication.

[0081] Table 1 Overview of the dataset and its characteristics

[0082] Dataset Average number of nodes Number of images Average number of edges Number of categories AIDS 15.69 2000 16.20 2 fingerprint 7.06 2149 5.76 4 Molbace 34 1513 73.70 2 Molbbbp 24.06 2039 51.90 2 protein 39.06 1113 72.82 2

[0083] [1] C. Morris, NM Kriege, F. Bause, K. Kersting, P. Mutzel, M. Neumann, Tudataset: A collection of benchmark datasets for learning with graphs, arXivpreprint arXiv: 2007.08663 (2020).

[0084] [2] W Hu, M. Fey, M. Zitnik, Y. Dong, H. Ren, B. Liu, M. Catasta, J. Leskovec, Opengraph benchmark: Datasets for machine learning on graphs, arXiv preprint arXiv: 2005.00687 (2020).

[0085] [3] Z. Wu, B. Ramsundar, EN Feinberg, J. Gomes, C. Geniesse, ASPappu, K. Leswing, V. Pande, Moleculenet: a benchmark for molecular machine learning, Chemical science 9 (2018) 513-530.

[0086] [4] KM Borgwardt, CSOng, S. S.Vishwanathan, AJSmola, H.-P.Kriegel, Protein function prediction via graph kernels, Bioinformatics 21(2005)i47-i56.

[0087] In order to predict the classification task of the above data set, the present invention provides "a federated learning differentially private graph neural network system and training method".

[0088] like Figure 2 As shown, a federated learning differentially private graph neural network system is composed of a server (1) and M client devices (2), which communicate data with each other through a network connection; the server (1) includes: a server data storage module (11), a server federated learning module (12); one of the client devices (2) includes: a client data storage module (21), a client federated learning module (22); the server federated learning module (12) and the client federated learning module (22) respectively store a hierarchical pooling graph neural network model with exactly the same structure; wherein M is a positive integer greater than 1.

[0089] The hierarchical pooling graph neural network consists of three stacked layers, each layer contains two modules: a graph neural network layer and a graph pooling layer connected in series. A readout layer is used between the layers to achieve feature fusion of all layer outputs, and finally a multi-layer perception layer is used to achieve the classification output of the results.

[0090] The graph neural network layer adopts a typical representative graph convolutional neural network (GCN) among GNNs.

[0091] A federated learning differential privacy graph neural network training method, combined with Figure 3 , including the following steps:

[0092] S1: The server federated learning module (12) in the server (1) initializes the federated learning parameters and stores them in the server data storage module (11).

[0093] S2: The server (1) initializes the parameters of the layered pooling graph neural network model and stores them in the server data storage module (11).

[0094] S3: The server data storage module (11) sends the layered pooling graph neural network model parameters to all client devices (2) and stores them in the client data storage module (21).

[0095] S4: The client (2) calculates the federated learning gradient data of the layered pooling graph neural network model using the user's historical data in the client data storage module (21), and stores it in the client data storage module (21).

[0096] S5: The client data storage module (21) sends the federated learning gradient data to the server data storage module (11).

[0097] S6: The server federated learning module (12) uses the federated learning gradient data of the server data storage module (11) to calculate the parameters of the new layered pooling graph neural network model and stores it in the server data storage module (11).

[0098] S7: Repeat steps S3 to S8 until the maximum number of iterations t = t max ,At this point, all client devices (2) have completed the training of the layered pooling graph neural network model.

[0099] More specific details are consistent with the description in Example 1 and will not be repeated here.

[0100] Table 2 Performance comparison experimental results of each data set

[0101]

[0102] In order to better show the beneficial effects of the present application, the method of the present application and the method of reference [6] are compared on all data sets. The hierarchical pooling graph neural network adopted in the system model adopts the same structure and initial parameters, and the federal learning parameters and the sample data set of the client are the same. The present embodiment is compared in six aspects of ROC-AUC curve, accuracy, sensitivity, specificity and F1-Score, and the experimental results are shown in Table 2. v

[0103] [6] T.T. Mueller, J.C. Paetzold, C. Prabhakar, D. Usynin, D. Rueckert, G. Kaissis, Differentially private graph neural networks for whole-graph classification, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (2022) 7308-7318.

[0104] From Table 2, it is not difficult to see that the method of the present application is better than the prior art in almost all performance comparisons.

[0105] Finally, it should be pointed out that the above preferred embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made in form and details without departing from the scope defined by the claims of the present application.​

Claims

1. A federated learning differentially private graph neural network system, comprising a server (1) and M client devices (2), which communicate with each other via a network connection; the server (1) comprises: a server data storage module (11), a server federated learning module (12); one of the client devices (2) comprises: a client data storage module (21), a client federated learning module (22); the server federated learning module (12) and the client federated learning module (22) respectively store a hierarchical pooling graph neural network model with exactly the same structure; wherein M is a positive integer greater than 1; The server data storage module (11) is connected to the client data storage module (21) and is used to receive and store the federated learning gradient data sent by the client data storage module (21) in the client device (2), and send the model parameters in the server (1) to the client data storage module (21) to update the model parameters of the client; The server federated learning module (12) is connected to the server data storage module (11), updates the model parameters in the server (1) according to the federated learning gradient data, and stores the updated data in the server data storage module (11); The client data storage module (21) is used to locally store the user's historical data, the client's model parameters and the federated learning gradient data; The client federated learning module (22) is connected to the client data storage module (21), and uses the user's historical data in the client data storage module (21) and the client's model parameters to calculate the federated learning gradient data and store it in the server data storage module (11).

2. A federated learning differentially private graph neural network system according to claim 1, characterized in that: The hierarchical pooling graph neural network is composed of multiple stacked layers, each layer contains two modules: a graph neural network layer (GNN) and a graph pooling layer connected in series. A readout layer is used between the layers to achieve feature fusion of the outputs of all layers, and finally a multi-layer perception layer (MLP) is used to achieve the classification output of the results.

3. A federated learning differentially private graph neural network system according to claim 2, characterized in that: For the user's historical data graph G = (A, X), the output of the i-th stacked graph convolution layer of the layered pooling graph neural network is: The output of the graph pooling layer is: H i+1 =H i [idx,:]; where idx=Top k (X i ), Top k (·) is the index function of the k nearest neighbor nodes, is a hyperparameter, graph feature matrix and the adjacency matrix is the weight parameter of the i-th stack, I is the identity matrix, and the degree matrix diag(·) is the diagonalization operation, d is the dimension of the feature vector, H1=X, n is the number of graph nodes, and i≥1.

4. A federated learning differentially private graph neural network system according to claim 2, characterized in that: The readout layer uses sorting and aggregation operations to maintain the relative order of node features at each level, and then connects the node features of all levels through Concat to form the final node features as the input of the multi-layer perception layer.

5. A federated learning differentially private graph neural network training method applied to a federated learning differentially private graph neural network system according to any one of claims 1 to 4, characterized in that: The steps include: S1: The server federated learning module (12) in the server (1) initializes the federated learning parameters and stores them in the server data storage module (11); S2: The server (1) initializes the parameters of the layered pooling graph neural network model and stores them in the server data storage module (11); S3: The server data storage module (11) sends the layered pooling graph neural network model parameters to all client devices (2) and stores them in the client data storage module (21); S4: The client uses the user's historical data in the client data storage module (21) to calculate the federated learning gradient data of the layered pooling graph neural network model and stores it in the client data storage module (21); S5: The client data storage module (21) sends the federated learning gradient data to the server data storage module (11); S6: The server federated learning module (12) calculates the parameters of the new layered pooling graph neural network model using the federated learning gradient data of the server data storage module (11), and stores the parameters in the server data storage module (11); S7: Repeat steps S3 to S8 until the maximum number of iterations, at which point all client devices (2) have completed the training of the layered pooling graph neural network model.

6. A federated learning differentially private graph neural network training method according to claim 5, characterized in that: The federated learning parameters described in step S1 include: maximum number of iterations t max , learning rate warm-up iteration number t u , learning rate cooling iteration number t d , gradient norm upper bound C, initial first-order moment estimate m0, initial second-order moment estimate v0, maximum second-order moment estimate v max , learning rate η, weight decay coefficient λ, noise standard deviation σ, decay rates β0, β1, β2, β a , update frequency Sampling volume L, minimum numerical error ε; where, t max , t u , t d 、 L is a positive integer, v max ≥v0,t d >t u , η, λ, β0, β1, β2, β a ∈[0, 1].

7. The method for training a graph neural network with federated learning differential privacy according to claim 5, characterized in that: The step S3 is updated according to the set frequency. Regularly revise the parameters of the layered pooling graph neural network model, that is, when the tth iteration update satisfies When Correcting the parameters of the layered pooling graph neural network model Where ← represents assignment and p is a positive integer.

8. The method for training a graph neural network with federated learning differential privacy according to claim 5, characterized in that: The step S4 is specifically as follows: S401: The client randomly samples L historical data from the user's historical data with equal probability as a sample set D; S402: The client federated learning module (22) uses the sample set to calculate the original gradient of the layered pooling graph neural network model; wherein, at the tth iteration, any i-th historical data x i The original gradient is denoted as g t (x i ); S403: The client federated learning module (22) clips the original gradient of the sample set to obtain the clipped gradient Among them, ||·||2 is the two-norm; S404: The client federated learning module (22) adds noise to the clipped gradient to obtain federated learning gradient data Where N(0, σ 2 C 2 ) has a mean of 0 and a variance of σ 2 C 2 Normally distributed noise.

9. The method for training a graph neural network with federated learning differential privacy according to claim 5, characterized in that: For the t-th iteration update, step S6 is specifically as follows: S601: The server federated learning module (12) calculates the first-order moment estimate using the federated learning gradient data of the server data storage module (11). and second-order moment estimates in Federated learning gradient data calculated for all clients separately The mean of: S602: The server federated learning module (12) updates the maximum second-order moment estimate v max =max(v t , v max ); S603: The server federated learning module (12) calculates the first-order moment bias correction and second-order moment bias correction S604: The server federated learning module (12) calculates the deviation vector S605: The server federated learning module (12) calculates the current learning rate S606: Server federated learning module (12) calculates weight decay Among them, θ t-1 is the layered pooling graph neural network model parameter of the previous iteration, and mean(·) is the mean operation; S607: The server federated learning module (12) calculates the t-th iteration to update the layered pooling graph neural network model parameters θ t =θ t-1 -η t ·u t -η t ·d t .

Images