Method for property inference of federated graph neural networks and applications thereof

Abstract

The application discloses a federal graph neural network attribute inference method and application thereof. The method comprises the following steps: generating a local node embedding at an inference end, and obtaining an inference end local embedding gradient corresponding to the local node embedding; generating a target user embedding gradient based on the inference end local embedding gradient, updating the target user embedding gradient, and obtaining a local node embedding of the target user; obtaining a shadow graph dataset, marking the shadow graph dataset with a preset target inference attribute, training a shadow classification model and an attribute classifier based on the shadow graph dataset; obtaining a feature vector corresponding to the target user local node embedding based on the shadow classification model; and obtaining a probability distribution of a predicted attribute of the target user local graph data based on the updated attribute classifier. The application can accurately measure the possibility of user privacy data leakage, and provides theoretical support for the development of a privacy protection method.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data privacy inference, and particularly relates to a property inference method of a federated graph neural network and application thereof. BACKGROUND

[0002] In recent years, federated graph neural networks (FedGNNs) have achieved remarkable success in representation learning of graph data. It can realize collaborative training of graph neural network (GNN) models among multiple participants without sharing their original graph data, thereby solving the data island problem faced by centralized GNNs in data-sensitive scenarios.

[0003] Graph data carries a large amount of personal attribute privacy, which is extremely easy to cause privacy leakage in the process of sharing, analysis and application, and faces potential threats. Although FedGNNs can protect the data privacy of users to a certain extent, they still face many privacy security problems.

[0004] Since the vulnerability of FedGNNs in the face of inference attacks has not been deeply explored in the prior art, there is an urgent need for a property inference method of a federated graph neural network for testing the security performance of the federated graph neural network.

[0005] Therefore, in view of the above technical problems, it is necessary to provide an improved property inference strategy for a federated graph neural network environment.

[0006] The information disclosed in this BACKGROUND section is only intended to increase an understanding of the general context in which the present application can be practiced. It is not admitted that this information constitutes prior art that is already known in the art. SUMMARY

[0007] The purpose of the present application is to provide a property inference method of a federated graph neural network and application thereof, which can provide an effective property inference method, can detect the defense ability of the federated graph neural network against property inference, and its own security performance.

[0008] In order to achieve the above-mentioned purpose, the technical scheme provided by an embodiment of the present application is as follows:

[0009] In a first aspect, the present application provides a property inference method of a federated graph neural network, comprising:

[0010] Based on the initial model, generating a local node embedding at the inference end, and obtaining an inference end local embedding gradient corresponding to the local node embedding based on forward propagation;

[0011] generate a target user embedding gradient based on the inference end local embedding gradient, recursively update the target user embedding gradient, and obtain a local node embedding of the target user based on the updated target user embedding gradient;

[0012] obtain a shadow map dataset based on the local node embedding of the target user, and label the shadow map dataset with a preset target reasoning attribute, train a shadow classification model and an attribute classifier based on the labeled shadow map dataset;

[0013] input the local node embedding of the target user, obtain a feature vector corresponding to the local node embedding of the target user based on the shadow classification model, and input the feature vector and the target reasoning attribute to obtain a probability distribution of the prediction attribute of the local map data of the target user based on the updated attribute classifier.

[0014] In one or more embodiments of the present application, the generating of the target user embedding gradient based on the local embedding gradient comprises:

[0015] generating a random noise vector with the same number of rows or columns as the local embedding gradient and obtaining a noise gradient vector corresponding to the random noise vector;

[0016] concatenating the local embedding gradient and the noise gradient vector by rows or by columns to generate the target user embedding gradient.

[0017] In one or more embodiments of the present application, the recursively updating of the target user embedding gradient comprises:

[0018] obtaining an aggregated global model and a global embedding gradient corresponding to the global model;

[0019] inputting the target user embedding gradient into a preset discriminator, updating parameters of the discriminator based on minimizing the first loss function;

[0020] updating parameters of a generator for generating local node embeddings and random noise vectors based on minimizing a second loss function;

[0021] generating the target user embedding gradient based on the updated generator.

[0022] In one or more embodiments of the present application, the obtaining of the local node embedding of the target user based on the updated target user embedding gradient is based on the following formula:

[0023]

[0024] wherein W (l) is a local model gradient weight of the target user, H CEmbedding for a local node of a target user.

[0025] In one or more embodiments of the present application, the shadow classification model and the attribute classifier are trained based on the labeled shadow map dataset, comprising:

[0026] The shadow classification model is trained based on the shadow map dataset and / or the local map dataset at the inference end, and a node embedding of the shadow model is obtained;

[0027] The parameters of the shadow classification model are updated based on minimizing the third loss function;

[0028] Based on the updated shadow classification model, the preset test dataset is classified, and a plurality of categories and probability values corresponding to each category are output;

[0029] The probability values corresponding to each category are aggregated into a feature vector, and a training dataset of an attribute classifier is generated based on the feature vector;

[0030] The attribute classifier is trained based on the training dataset of the attribute classifier.

[0031] In one or more embodiments of the present application, the probability values corresponding to each category are aggregated into a feature vector, comprising:

[0032] Based on a preset activation function, the probability values corresponding to each category are converted into posterior output probabilities, and the sum of the posterior output probabilities corresponding to each category is 1;

[0033] The Element-wise difference between each posterior output probability value p1, p2, …, p k is calculated, and each difference result is spliced into a feature vector

[0034] In one or more embodiments of the present application, the first loss function, the second loss function, and the third loss function are respectively:

[0035]

[0036] Wherein, N represents the dimension of the embedding vector or gradient, φ is the discriminator model parameter, represents the output of the discriminator model, is a global embedding gradient, K is the number of clients participating in federated training, is a local embedding gradient at the inference end, z fake is a random noise vector, θ is the generator model parameter, is a local embedding gradient at the non-inference end, is a target user embedding gradient; LD L1 is a first loss function, GRN L2 is a second loss function, D L3 is a third loss function; The shadow model embedding gradient is obtained.

[0037] In a second aspect, the present application provides an attribute inference system of a federated graph neural network, comprising:

[0038] A generation module is configured to generate a local node embedding at an inference end based on an initial model, and obtain an inference end local embedding gradient corresponding to the local node embedding based on forward propagation;

[0039] An update module is configured to generate a target user embedding gradient based on the inference end local embedding gradient, recursively update the target user embedding gradient, and obtain a local node embedding of a target user based on the updated target user embedding gradient;

[0040] A training module is configured to obtain a shadow graph dataset based on the local node embedding of the target user, label the shadow graph dataset with a preset target inference attribute, train a shadow classification model and an attribute classifier based on the labeled shadow graph dataset;

[0041] An inference module is configured to take the local node embedding of the target user as input, obtain a feature vector corresponding to the local node embedding of the target user based on the shadow classification model, and take the feature vector and the target inference attribute as input, obtain a probability distribution of a predicted attribute of local graph data of the target user based on the updated attribute classifier.

[0042] In a third aspect, the present application provides a security detection method of a federated graph neural network, which applies the attribute inference method of the federated graph neural network, comprising:

[0043] Attribute inference is performed on a target user in the federated graph neural network based on the attribute inference method of the federated graph neural network;

[0044] Attribute information of local graph data of the target user is obtained.

[0045] If an accuracy of the attribute inference of the federated graph neural network is greater than a preset security threshold, the target user is in a risk state;

[0046] If the accuracy of the attribute inference of the federated graph neural network is less than or equal to the preset security threshold, the target user is in a safe state.

[0047] In a fourth aspect, the present application provides a security detection system of a federated graph neural network, comprising:

[0048] The application module is configured to perform attribute inference on the target user in the federated graph neural network based on an attribute inference method of the federated graph neural network.

[0049] The acquisition module is configured to acquire attribute information of local graph data of the target user.

[0050] The first judgment module is configured to determine that the target user is in a risk state when the accuracy of the attribute inference of the local graph neural network is greater than a preset safety threshold.

[0051] The second judgment module is configured to determine that the target user is in a safe state when the accuracy of the attribute inference of the local graph neural network is less than or equal to the preset safety threshold.

[0052] Compared with the prior art, the attribute inference method of the federated graph neural network provided by the application can obtain the probability distribution of the local graph data of the target user by recursively updating the local node embedding of the user and combining the trained shadow model and the attribute classifier. The target local data can be inferred for the federated graph neural network, the possibility of user privacy data leakage can be accurately measured, and theoretical support is provided for the development of privacy protection methods. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments described in the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0054] Figure 1 is a schematic diagram of a federated graph neural network framework in an embodiment of the application;

[0055] Figure 2 is a flowchart of an attribute inference method of a federated graph neural network in an embodiment of the application;

[0056] Figure 3 is a structural block diagram of an attribute inference system of a federated graph neural network in an embodiment of the application;

[0057] Figure 4 is a structural block diagram of an electronic device in an embodiment of the application.

[0058] Figure 5 is a flowchart of a security detection method of a federated graph neural network in an embodiment of the application;

[0059] Figure 6 is a structural block diagram of a security detection system of a federated graph neural network in an embodiment of the application. DETAILED DESCRIPTION

[0060] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings, but the scope of protection of the present application is not limited by the specific embodiments.

[0061] Unless otherwise clearly indicated, throughout the specification and claims, the term "comprise" or variations such as "comprises" or "comprising" will be understood to imply the inclusion of a stated element or group of elements but not the exclusion of any other element or group of elements.

[0062] In order to facilitate the understanding of the technical solutions of the present application, the technical terms that may appear in the present application will be explained in detail below.

[0063] Forward propagation: also known as forward propagation algorithm, is a training method of neural network model. The input data passes through each layer of the neural network, the weights and biases of each layer linearly transform the input data, and then the non-linear transformation is performed through the activation function, and finally the output result of the model is obtained. The purpose of forward propagation is to calculate the predicted value of the model.

[0064] Federated graph neural network (Federated Graph Neural Networks): Federated graph neural network combines the advantages of federated learning and graph neural network model, and can learn and infer distributed graph data while protecting data privacy.

[0065] Please refer to Figure 2 Fig. 1 shows a flowchart of the attribute inference method of the federated graph neural network in an embodiment of the present application. The attribute inference method of the federated graph neural network specifically includes the following steps:

[0066] S201: based on the initial model, generating a local node embedding at the inference end, and obtaining an inference end local embedding gradient corresponding to the local node embedding based on forward propagation;

[0067] As shown in Figure 1 Fig. 2 is a schematic diagram of a federated graph neural network framework in an embodiment of the present application. In the FedGNNs framework, each client has its own graph data set D (i) :=(g (i) ,Y (i) ), where g (i) =(V (i) ,E (i) ), V (i) represents a set of nodes in the graph, E (i) represents a set of edges, Y (i)A set of labels representing the graph data. Each client owns a GNN model to learn the representation of the graph and make predictions. FedGNNs can enable multiple clients to collaborate through a server to improve their respective GNN models without necessarily disclosing the client's local graph dataset. For each node v i ∈V (i) , neighborhood aggregation is the same as traditional GNNs. In FedGNNs with node classification as the downstream task, the client C learns local node embeddings using the local GNN model, and uploads the trained local model gradient and node embedding gradient to the central server after completing l rounds of local training. The server aggregates the gradients received from the clients and delivers the aggregated global model and global node embedding gradient to each client. Finally, the client updates the local GNN classification model using the received global gradient until the model converges.

[0068] As described above, federated graph neural networks can enable multiple clients to collaborate through a server to improve their respective graph neural network models. In federated learning, only the server fits the generated global model gradient and global embedding gradient are shared among participants without exposing the local private data of the participants. But also because of the sharing of the above information, users may be disguised as an attack party of the client node participating in the model aggregation, infer and obtain their private information based on the shared global gradient. The present application focuses on federated graph neural networks and provides a more accurate attribute inference method. The ability of the corresponding federated graph neural network to resist attribute inference attacks can be judged based on the comparison between the feature inference result of the target user's local graph data and the actual feature.

[0069] It can be understood that in order to obtain the global model and global embedding gradient delivered by the server, the attack party needs to be one of the clients participating in federated learning, so the initial model is the initial GNN model corresponding to each client in the federated graph neural network. Further, the initial GNN model is used to complete the local graph data training to obtain a d-dimensional local node embedding H m , and a local embedding gradient is obtained through forward propagation for participating in federated training.

[0070] S202: Based on the inference end local embedding gradient, generate a target user embedding gradient, recursively update the target user embedding gradient, and obtain a local node embedding of the target user based on the updated target user embedding gradient;

[0071] In an exemplary embodiment, based on the local embedding gradient, the target user embedding gradient is generated, including: generating a random noise vector with the same number of rows or columns as the local embedding gradient and obtaining a noise gradient vector corresponding to the random noise vector; and splicing the local embedding gradient and the noise gradient vector by row or by column to generate the target user embedding gradient.

[0072] It should be noted that, in order to realize the generation of the target user embedding gradient, a generator for generating a local node embedding and a random noise vector is arranged at the inference end, and a discriminator for evaluating the gap between the generated embedding gradient estimate and the real global embedding gradient is arranged to make the generated node embedding of the generator closer to the global embedding.

[0073] For example, a multilayer perceptron can be arranged at the inference end. The multilayer perceptron (MLP, Multilayer Perceptron) is a forward structure artificial neural network, which can have multiple hidden layers in addition to the input and output layers. The layers of the multilayer perceptron are fully connected. The bottom layer of the multilayer perceptron is the input layer, the middle is the hidden layer, and the last is the output layer. In an embodiment of the present application, the GRN (Generative Regression Neutral Network) in the above-described perceptron is composed of a generator and a discriminator.

[0074] Further, the inference end uses the above-described generator to generate a random noise vector z fake and a corresponding noise gradient vector θ is the model parameter of the generator. Then, the inference end splices and the d-dimensional linking into a Kxd-dimensional target user embedding gradient as the input of the discriminator to facilitate the generator to obtain a more approximate target user embedding gradient from the real data. That is, the generator model can be represented as

[0075]

[0076] It can be understood that, if the generated noise gradient vector has the same number of rows as the local embedding gradient , the splicing can also be The present application can apply the above two splicing methods or change the splicing position based on the above two splicing methods, which is not limited in the embodiments.

[0077] Further, in an exemplary embodiment of the present application, the target user embedding gradient is recursively updated, specifically including: obtaining the aggregated global model and the global embedding gradient corresponding to the global model; inputting the target user embedding gradient into a preset discriminator, updating the parameters of the discriminator based on minimizing the first loss function; updating the parameters of the generator based on minimizing the second loss function; and generating the target user embedding gradient based on the updated generator.

[0078] It should be noted that the inference end obtains the global embedding gradient and the aggregated global model by querying the server . The acquisition of the target user embedding gradient can be converted into a regression problem. That is, the discriminator adjusts its parameters by minimizing the loss function L D , and feeds back the result to the generator to make the generated inference end local node embedding closer to the global embedding.

[0079] It can be understood that the server usually uses the mean method to fit the local embedding of each client to the global embedding, that is:

[0080]

[0081] Since the global embedding is approximately the average of all client node embeddings, the inference end can infer the embedding features of other clients (especially the target user) by generating an approximation of the global embedding and subtracting the known part (such as the local embedding of the inference end). The inference end optimizes the generator parameters θ by minimizing L GRN , so that the generated noise gradient is as close as possible to the actual node embedding gradient of the target user. After obtaining the embedding gradient of the target model, the attacker can obtain the local node embedding H C of the target user through the layer-by-layer back propagation mechanism of the GNN.

[0082] It should be noted that the first loss function and the second loss function are respectively:

[0083]

[0084] where N represents the dimension of the embedding vector or gradient, φ is the discriminator model parameter, represents the output of the discriminator model, is the global embedding gradient, K is the number of clients participating in federated training, is the local embedding gradient of the inference end, z fake is a random noise vector, θ is the generator model parameter, is the local embedding gradient of the non-inference end, embedding gradient of the target user; L D is a first loss function, L GRN is a second loss function.

[0085] Based on the application of the above loss function, the optimization of the generator parameter θ can be represented as follows:

[0086]

[0087] wherein η is a learning rate. After repeatedly adjusting θ and z fake so that the generator finally generates

[0088]

[0089] After obtaining the embedding gradient of the target model, the inference end can obtain the local node embedding H of the target user through the layer-by-layer back propagation mechanism of the GNN. C .

[0090]

[0091] wherein W (l) The local model gradient weight of the target user can be obtained from the local gradient of the target user.

[0092] It should be noted that the embodiments of the present application do not limit the method of obtaining the local gradient of the target user. In a specific embodiment, the method of obtaining the local gradient of the target user in federated learning can be applied to the present scheme, that is, the local gradient of the target user is obtained by the method of differential analysis. In another exemplary embodiment, the embedding gradient of the target user can be replaced by the model gradient based on the generator and discriminator configured on the inference end as described above, and the same recursive updating method is used to obtain.

[0093] S203: Obtain a shadow map data set based on the local node embedding of the target user, and label the shadow map data set with a preset target inference attribute, train a shadow classification model and an attribute classifier based on the labeled shadow map data set;

[0094] It should be noted that the shadow map data set refers to a graph data set with the same or similar distribution as the target graph data, which is obtained from the public data set and / or the local data set based on the features of the local node embedding of the target user. Training the shadow classification model based on the shadow map data set can better simulate the node embedding output by the target user.

[0095] In particular, a single shadow model usually outputs a single result in the subsequent inference stage. In order to understand the reaction of the target user in the face of different structural data and characteristics, thereby improving the prediction accuracy of the target user graph attribute, the shadow data set and the local data set at the inference end can be divided into multiple training data sets, and a shadow model is trained based on each training data set.

[0096] For example, by querying the characteristics (including but not limited to mean, variance, etc.) of the local node embedding H C of the target user from the public data set, an auxiliary data set D aux is obtained which has the same or similar distribution as the target graph data. S The shadow graph data set G S is marked in the local graph data set at the inference end and the shadow graph data set, and the target inference attribute that the inference end wants to infer is marked. The inference end trains x (x≥1) shadow models based on G Each shadow model is trained based on multiple subsets of the same size. The shadow graph is randomly sampled from the local data set G m or G S .

[0097] It should be noted that the target inference attribute, i.e., the graph attribute that the inference end expects to obtain in the target user's local graph data, can include but is not limited to node number, variable number, image category, etc. The selection of the inference attribute by the embodiment of the present application is not limited.

[0098] In an exemplary embodiment of the present application, the shadow classification model and the attribute classifier are trained based on the marked shadow graph data set, including: training the shadow classification model based on the shadow graph data set and / or the local graph data set at the inference end, obtaining the node embedding of the shadow model; updating the parameters of the shadow classification model based on minimizing the third loss function; classifying the predetermined test data set based on the updated shadow classification model, outputting multiple categories and the probability value corresponding to each category; aggregating the probability value corresponding to each category into a feature vector, generating a training data set for the attribute classifier based on the feature vector; training the attribute classifier based on the training data set of the attribute classifier.

[0099] The third loss function is used to directly compare the node embedding obtained by the shadow classification model from the shadow graph data set and / or the local data set at the inference end with the target node embedding, and as the loss function becomes smaller, the difference between the two becomes smaller, thereby achieving the purpose of optimizing the parameters of the shadow classification model.

[0100] In an exemplary embodiment, the third loss function can be:

[0101]

[0102] wherein L D is a third loss function; is a shadow model embedding gradient, denotes the output of the discriminator model, N denotes the dimension of the embedding vector or gradient, and φ denotes the parameters of the discriminator model.

[0103] Further, aggregating the probability values corresponding to each of the categories into a feature vector includes: converting the probability values corresponding to each of the categories into posterior output probabilities based on a preset activation function, and a sum of the posterior output probabilities corresponding to each category is 1; calculating an Element-wise difference between each of the posterior output probability values p1, p2, …, p k and concatenating the difference results into a feature vector

[0104] For example, in a specific embodiment, the inference end expects to infer the number of nodes in a target user's local map data set. The shadow classification model output result vector shows that the possibility of the number of nodes in the data set being 100 is 60%, the possibility of the number of nodes in the data set being 60 is 50%, and the possibility of the number of nodes in the data set being 30 is 90%. It can be understood that the sum of the probabilities in the above output result has exceeded 100%, and therefore it is necessary to process it so that the probability values corresponding to each of the categories are converted into posterior output probabilities. That is, the sum of the posterior output probabilities corresponding to each category is 1. Corresponding to the above specific embodiment, after conversion, it should show that the possibility of the number of nodes in the data set being 100 is 30%, the possibility of the number of nodes in the data set being 60 is 25%, and the possibility of the number of nodes in the data set being 30 is 45%.

[0105] It should be noted that the activation function refers to a function running on a neuron of an artificial neural network, which is responsible for mapping the input of the neuron to the output, and aims to help the network learn complex operations in the data. If an activation function is not used, the output of each layer is a linear function of the input of the upper layer, and the output is a linear combination of the input regardless of the number of layers of the neural network. Using an activation function can introduce a non-linear factor to the neuron, so that the neural network can arbitrarily approximate any non-linear function, and the expression ability of the deep neural network is more powerful, so that the neural network can be applied to numerous non-linear models. In the embodiments of the present application, the softmax function is preferably used as the activation function to convert the probability values corresponding to each of the categories into posterior output probabilities.

[0106] It should be noted that the k posterior output probabilities p1, p2, …, p k , the Element-wise difference p diff between all posterior probability values is calculated

[0107]

[0108] The p diff of each node is obtained diff After that, the inference end splices all p P into a feature vector Each shadow classification model training data set pre-labeled with target attributes, including node number, edge number, and the spliced feature vector are collectively used as the training data set of the attribute classifier P

[0109] In an exemplary embodiment, after the training data set of the attribute classifier is generated, the inference end trains the attribute classifier on the training data set. The type of the prediction layer model of the attribute classifier is not limited in the embodiment of the application. Since the random forest model can provide a higher prediction accuracy when processing complex data and high-dimensional data, preferably, the random forest (RF) is used as the prediction layer model in the embodiment.

[0110] The attribute classifier is generally composed of a feature extractor E and multiple parallel prediction layers M P Each M P is responsible for predicting an attribute, and the target is to optimize the parameters λ of the attribute classifier by minimizing the cross-entropy loss function, so that the attribute classifier can infer the attributes of interest of the inference end in the graph data. The optimization problem can be expressed as:

[0111]

[0112] Where F is the attribute set of interest of the attacker, such as the number of nodes, the number of edges, the node density, etc., f is an attribute in F, and for each attribute f ∈ F, there is a corresponding prediction layer M P , and represents the cross-entropy loss. The feature extractor E is responsible for extracting the embedding features in the training data set input to the attribute classifier. Here, all attributes share the same parameters for E, and different parameters are used for M P , so that it can handle the uniqueness of each attribute and ensure that each attribute is optimized specifically. Through this supervised learning method, the attribute classifier can learn the relationship between the feature vector and the attribute label, so as to be able to identify the specific attributes possessed by the target user in the local graph in the inference stage.

[0113] S204: Based on the shadow classification model, obtain the feature vector corresponding to the local node embedding of the target user with the local node embedding of the target user as input; based on the updated attribute classifier, obtain the probability distribution of the predicted attribute of the local map data of the target user with the feature vector and the target reasoning attribute as input.

[0114] Based on the foregoing steps of the present scheme, the reasoning end has trained the attribute classifier using the embedding and attribute label. In the reasoning phase, the reasoning end inputs the obtained local embedding of the target user into the attribute classifier. The input features can be statistical features (such as mean, variance, etc.) of the target node embedding or structural features of the relationship between nodes.

[0115] In addition, since the reasoning end has learned the behavior of the local model of the target user through the shadow classification model and other means in the training phase, the reasoning end can obtain the posterior output of the local embedding of the target user using the forward propagation of the shadow classification model and the corresponding activation function layer, and calculate the aggregated posterior probability of the target user using the same aggregation method as in the training phase of the attribute classifier. Subsequently, the reasoning end can obtain the predicted attribute of the target graph G Target user local embedding H C The statistical features (variance, mean, etc.), structural features, and optimized classifier parameters λ of the target user local embedding H P are input into the trained attribute classifier P, and the M C layer performs reasoning according to the input features, and the attacker can obtain the probability distribution of the predicted attribute of the target graph G

[0116]

[0117] To further illustrate the reasoning effect of the attribute reasoning method provided by the present application in the federated graph neural network, a set of experimental data is again provided as evidence.

[0118] The following experiments are performed on a Xeon(R) W-2133 CPU device equipped with 256 GB RAM and a NVIDIA GeForce RTX 2080Ti processor, and all algorithms are implemented based on the PyTorch framework. The number of iterations for federated training is set to 50, and the learning rate of the generator model and the GNN model is set to 0.01. This paper sets 5 clients participating in the training of FedGNNs, one of which is the reasoning client, or the client directly controlled by the reasoning end.

[0119] Three public datasets with different complexity in terms of the number of classes, edges and nodes are chosen to verify the performance of the method, including Cora, Citeseer, Pubmed. They are widely used in literature for graph learning. The entire graph dataset is evenly distributed to 5 clients, each of which has a part of the entire graph dataset as a local dataset to generate training and test data for PIAFGNN, and the local graph data of the client is divided into 80% training data set and 20% test data set. In the same experiment, two different privacy graph attributes are considered as the target of the attacker, namely the number of nodes and the number of edges, which are irrelevant to the main classification task of the model.

[0120] In order to compare the inference accuracy of different settings fairly, the training data of all inference ends is ensured to be the same in the experiment. Random forest (RF) is used as the training algorithm of the attribute classifier in this paper, and MaxPool is used as the embedding aggregation method of GNN in the pooling layer. At the same time as a reference, the attribute inference attack method for centralized GNNs and the random guess method for binary classification task are selected as the baseline.

[0121] Table 1 Inference accuracy of different local models and different datasets

[0122]

[0123] As can be seen from the above table, the present application has achieved excellent inference performance under different settings, and even on FedGNNs based on GCN, the inference accuracy is close to that of the centralized method, and the correct prediction rate reaches 79.40% in the inference of the target graph node number attribute. In addition, even on such a complex GNN model as GraphSage, the attack accuracy of PIAFGNN can reach more than 65% on such graph datasets as Pubemd and CiteSeer with complex structure and loose connection, which is much higher than the attack accuracy of the random guess baseline. This shows that the present application can implement high-precision attribute inference on different local model structures and different datasets.

[0124] Please refer to Figure 5 Fig. 1 shows a flowchart of a security detection method of a federated graph neural network in an embodiment of the present application. The attribute inference method of the federated graph neural network specifically includes the following steps:

[0125] S501: attribute inference of a target user in the federated graph neural network based on the attribute inference method of the federated graph neural network; S502: obtaining attribute information of local graph data of the target user; S503: if the accuracy of the attribute inference of the federated graph neural network is greater than a preset safety threshold, the target user is in a risk state; S504: if the accuracy of the attribute inference of the federated graph neural network is less than or equal to the preset safety threshold, the target user is in a safe state.

[0126] Please refer to Figure 3 Based on the same inventive concept as the attribute inference method of the federated graph neural network, an embodiment of the present application provides an attribute inference system 300 of a federated graph neural network, which comprises a generation module 301, an update module 302, a training module 303 and an inference module 304.

[0127] Specifically, the generation module 301 is configured to generate a local node embedding at an inference end based on an initial model, and obtain an inference end local embedding gradient corresponding to the local node embedding based on forward propagation; the update module 302 is configured to generate a target user embedding gradient based on the inference end local embedding gradient, recursively update the target user embedding gradient, and obtain a local node embedding of a target user based on the updated target user embedding gradient; the training module 303 is configured to obtain a shadow graph dataset based on the local node embedding of the target user, mark the shadow graph dataset with a preset target inference attribute, train a shadow classification model and an attribute classifier based on the marked shadow graph dataset; and the inference module 304 is configured to take the local node embedding of the target user as input, obtain a feature vector corresponding to the local node embedding of the target user based on the shadow classification model, and take the feature vector and the target inference attribute as input, obtain a probability distribution of a predicted attribute of local graph data of the target user based on the updated attribute classifier.

[0128] Please refer to Figure 6 Based on the same inventive concept as the attribute inference method of the federated graph neural network, an embodiment of the present application provides an attribute inference system 300 of a federated graph neural network, which comprises a generation module 301, an update module 302, a training module 303 and an inference module 304.

[0129] Specifically, the application module 601 is configured to perform attribute inference on a target user in the federated graph neural network based on the attribute inference method of the federated graph neural network; the acquisition module 602 is configured to acquire attribute information of local graph data of the target user; the first judgment module 603 is configured to determine that the target user is in a risk state when an accuracy of attribute inference of the federated graph neural network is greater than a preset safety threshold; and the second judgment module 604 is configured to determine that the target user is in a safe state when the accuracy of attribute inference of the federated graph neural network is less than or equal to the preset safety threshold.

[0130] Please refer to Figure 4 The embodiments of the present application also provide an electronic device 400, which comprises at least one processor 401, a memory 402 (for example, a non-volatile memory), a storage 403 and a communication interface 404, and the at least one processor 401, the memory 402, the storage 403 and the communication interface 404 are connected together via a bus 405. The at least one processor 401 is configured to invoke at least one program instruction stored or encoded in the memory 402, so as to enable the at least one processor 401 to perform various operations and functions of the attribute inference method of the federated graph neural network described in various embodiments of the present application.

[0131] In the embodiments of the present application, the electronic device 400 can include but is not limited to a personal computer, a server computer, a workstation, a desktop computer, a laptop computer, a notebook computer, a mobile electronic device, a smart phone, a tablet computer, a cellular phone, a personal digital assistant (PDA), a handheld device, a messaging device, a wearable electronic device, a consumer electronic device, and the like.

[0132] The embodiments of the present application also provide a computer readable medium, which carries computer execution instructions, and the computer execution instructions, when executed by a processor, can be used to implement various operations and functions of the attribute inference method of the federated graph neural network described in various embodiments of the present application.

[0133] The computer readable medium in the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device.

[0134] In the present application, the computer readable signal medium can include a data signal propagated in a baseband or as a carrier wave in a propagated data signal, in which computer readable program code is embodied. Such propagated data signal can take a variety of forms, including but not limited to electro-magnetic, optical or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a storage medium and that can communicate, propagate or transport program for use by or in connection with an instruction execution system, apparatus or device. Program code embodied on a computer readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the above.

[0135] Those skilled in the art should clearly understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage etc.) containing computer usable program code.

[0136] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, systems, and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processing machine, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus create a means for implementing the functions specified in the flowcharts and / or block diagrams.Figure 1 apparatus for performing the function specified by the one or more blocks Figure 1 apparatus for performing the function specified by the one or more blocks.

[0137] The foregoing description of specific exemplary embodiments of the application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form disclosed, and various modifications and variations are possible in light of the above teachings. It is intended that the application embrace all alternatives, modifications, and variations as can come within the scope of the description and claims. It is intended that the scope of the application be limited not with this detailed description, but rather determined by the claims and their equivalents.

Claims

1. A method for security detection of a federated graph neural network, the method comprising: The method comprises: generating a local node embedding at an inference end based on an initial model, and obtaining an inference end local embedding gradient corresponding to the local node embedding based on forward propagation; generating a target user embedding gradient based on the inference end local embedding gradient, recursively updating the target user embedding gradient, and obtaining a local node embedding of the target user based on the updated target user embedding gradient; obtaining a shadow graph dataset based on the local node embedding of the target user, and labeling the shadow graph dataset with a preset target inference attribute, training a shadow classification model and an attribute classifier based on the labeled shadow graph dataset; taking the local node embedding of the target user as input, obtaining a feature vector corresponding to the local node embedding of the target user based on the shadow classification model, and taking the feature vector and the target inference attribute as input, obtaining a probability distribution of a predicted attribute of the target user local graph data based on the updated attribute classifier; obtaining attribute information of the target user local graph data; if an accuracy of attribute inference of the bond graph neural network is greater than a preset safety threshold, the target user is in a risk state; if the accuracy of attribute inference of the bond graph neural network is less than or equal to the preset safety threshold, the target user is in a safe state; wherein the generating of the target user embedding gradient based on the local embedding gradient comprises: generating a random noise vector with the same number of rows or columns as the local embedding gradient, and obtaining a noise gradient vector corresponding to the random noise vector; concatenating the local embedding gradient and the noise gradient vector by row or by column to generate the target user embedding gradient; the recursive updating of the target user embedding gradient comprises: obtaining an aggregated global model and a global embedding gradient corresponding to the global model; inputting the target user embedding gradient into a preset discriminator, updating parameters of the discriminator based on minimizing a first loss function, updating parameters of a generator for generating local node embedding and random noise vector based on minimizing a second loss function, and generating the target user embedding gradient based on the updated generator.

2. The method of claim 1, wherein, The formula for obtaining the local node embedding of the target user based on the updated target user embedding gradient is: ； wherein, are local model gradient weights for the target user, are local node embeddings for the target user.

3. The method of claim 1, wherein, The training of the shadow classification model and the attribute classifier based on the labeled shadow graph dataset comprises: training the shadow classification model based on the shadow graph dataset and / or the inference end local graph dataset, obtaining a node embedding of the shadow classification model; updating parameters of the shadow classification model based on minimizing a third loss function; classifying a preset test dataset based on the updated shadow classification model, and outputting a plurality of categories and probability values corresponding to each of the categories; aggregating the probability values corresponding to each of the categories into a feature vector, and generating a training dataset of the attribute classifier based on the feature vector; training the attribute classifier based on the training dataset of the attribute classifier.

4. The method of claim 3, wherein, The aggregation of the probability values corresponding to each of the categories into a feature vector comprises: The probability values corresponding to the categories are converted into posterior output probabilities based on a preset activation function, and the sum of the posterior output probabilities corresponding to each category is 1; computing an element-wise difference between each of the posterior output probability values and concatenating each difference result into a feature vector .

5. The method of claim 3, wherein, The first loss function, the second loss function, and the third loss function are respectively: ； ； ； where N denotes the dimension of the embedding vector or gradient, are the discriminator model parameters, denotes the output of the discriminator model, is the global embedding gradient, K is the number of clients participating in federated training, is the local embedding gradient at the inference end, is a random noise vector, are the generator model parameters, is the local embedding gradient at the non-inference end, is the target user embedding gradient; is a first loss function, is a second loss function, is a third loss function; is the shadow model embedding gradient.

6. A secure detection system of a federated graph neural network, applied to the secure detection method of the federated graph neural network according to any one of claims 1-5, characterized in that, Comprise: The generation module is configured to generate a local node embedding at an inference end based on an initial model, and obtain an inference end local embedding gradient corresponding to the local node embedding based on forward propagation; The update module is configured to generate a target user embedding gradient based on the inference end local embedding gradient, recursively update the target user embedding gradient, and obtain a local node embedding of a target user based on the updated target user embedding gradient; The training module is configured to obtain a shadow map data set based on the local node embedding of the target user, label the shadow map data set with a preset target inference attribute, train a shadow classification model and an attribute classifier based on the labeled shadow map data set; The inference module is configured to take the local node embedding of the target user as input, obtain a feature vector corresponding to the local node embedding of the target user based on the shadow classification model, and take the feature vector and the target inference attribute as input, obtain a probability distribution of a predicted attribute of the local map data of the target user based on the updated attribute classifier; The application module is configured to perform attribute inference on a target user in a federated graph neural network based on the attribute inference method of the federated graph neural network, obtain attribute information of the local map data of the target user, determine that the target user is in a risk state when an accuracy rate of the attribute inference of the federated graph neural network is greater than a preset safety threshold, and determine that the target user is in a safe state when the accuracy rate of the attribute inference of the federated graph neural network is less than or equal to the preset safety threshold.

Images