Longitudinal federal map learning method and device satisfying differential privacy

Abstract

The invention provides a longitudinal federated graph learning method and device satisfying differential privacy, and the method comprises the steps: transmitting a learning request to a service party and other participants participating in longitudinal federated graph learning except local participants, obtaining graph structure coding information based on target speculation graph structure information and an updated community set, generating perturbed graph structure coding information meeting the differential privacy; obtaining target shared graph structure information satisfying differential privacy based on the disturbed graph structure coding information and the initial shared graph structure information, and sharing the target shared graph structure information to other participants; receiving each piece of target shared graph structure information from each other participant, and determining global graph structure information based on each piece of target shared graph structure information and the target speculative graph structure information so as to reconstruct a global graph structure; and based on the global graph structure and the local graph data, performing distributed training on a preset global graph model by sending the intermediate result to the service party to obtain a trained global graph model.

Description

Technical Field

[0001] The present invention relates to the technical field of vertical federated graph learning, and in particular to a vertical federated graph learning method and device that satisfies differential privacy. Background Art

[0002] Graph neural networks (GNNs) combine node attribute information and high-order neighborhood information to generate rich node embeddings. They achieve state-of-the-art performance on graph-based learning tasks such as node and edge classification, attracting widespread attention from both academia and industry. During computation, GNNs use a node's neighborhood information as a guide, aggregating the attributes and features of adjacent nodes to generate node embeddings that incorporate both attribute and structural information. Therefore, the accuracy of GNN models relies on the complete attribute and structural information provided by the graph data holder.

[0003] However, in real-world scenarios, due to data regulations and commercial competition, global graph data may be dispersed across multiple institutions in the form of subgraphs or local graphs. Each institution can collect graph data information from trusted users and build local subgraphs, but mutually untrusted institutions cannot directly share these subgraphs for training graph neural networks. Vertical federated graph learning is a typical approach for graph learning problems in multi-party graph data collaboration scenarios. Vertical federated graph learning assumes that all participants are node-aligned, meaning that each participant has all nodes in the global graph, but the edges and node attributes in the global graph are divided into subgraphs held by different participants. In vertical federated graph learning methods, the graph neural network computation process requires aggregating node features based on the complete global graph structure to calculate node embeddings. However, each participant only holds an incomplete subgraph of the global graph structure and cannot directly share their subgraph data with other participants. Therefore, each participant must collaborate to perform feature aggregation in a privacy-preserving manner.

[0004] Existing vertical federated graph learning methods that achieve differential privacy generally use secure multi-party computation to encrypt the subgraph structure and perform feature aggregation on the complete neighborhood of a node in a ciphertext state to ensure that no additional information is revealed during the feature aggregation process. However, secure computation under ciphertext has a significant computational overhead, resulting in low computational efficiency of existing methods. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for vertical federated graph learning that satisfies differential privacy, so as to eliminate or improve one or more defects existing in the prior art.

[0006] One aspect of the present invention provides a vertical federated graph learning method that satisfies differential privacy, the method comprising the following steps: Sending a vertical federated graph learning request to the service provider and all other participants except the local party among the participants participating in the vertical federated graph learning, obtaining graph structure encoding information based on the target inferred graph structure information and the updated community set to generate perturbed graph structure encoding information that satisfies differential privacy, wherein the target inferred graph structure information is obtained by iteratively optimizing the initialized inferred graph structure information based on the local local graph data using a local local graph model, and the updated community set is obtained by updating the initial community set obtained by community division of the local local graph data based on the target inferred graph structure information; The target shared graph structure information that satisfies differential privacy is obtained based on the perturbed graph structure encoding information and the initial shared graph structure information, and is shared with other participants. The perturbed initial shared graph structure information is the initial shared graph structure information that satisfies differential privacy and is generated based on the initial shared graph structure information obtained by merging the target inferred graph structure information and the local partial graph data. Receiving target shared graph structure information from other participants, and determining global graph structure information based on the target shared graph structure information and the target inferred graph structure information to reconstruct the global graph structure; Based on the global graph structure and local local graph data, the preset global graph model is distributedly trained by sending intermediate results to the service party to obtain a trained global graph model, wherein the global graph model includes a local feature extraction module, a global feature aggregation module located on the service party and a local classification module that are communicatively connected in sequence, and the intermediate results include the local features extracted in each iteration and the updated gradient. In the first iteration, the updated gradient is the initial gradient.

[0007] In some embodiments of the present invention, the local partial graph data includes a node set, an adjacency matrix representing an edge set, a node attribute set, and a node category label set representing real node categories.

[0008] In some embodiments of the present invention, obtaining target shared graph structure information satisfying differential privacy based on the perturbed graph structure encoding information and the initial shared graph structure information includes: The edge existence probability matrix is ​​estimated based on the perturbed graph structure encoding information and used as the prior probability; The initial shared graph structure information after perturbation is used as evidence, and Bayesian estimation is performed on the target inferred graph structure information based on the prior probability and evidence. The edge posterior existence probability matrix that satisfies differential privacy is obtained and used as the target shared graph structure information.

[0009] In some embodiments of the present invention, the edge existence probability matrix includes an updated intra-community edge existence probability matrix obtained based on the perturbed neighbor number sequence in the perturbed graph structure encoding information and an updated inter-community edge existence probability matrix obtained based on the perturbed connection number matrix in the perturbed graph structure encoding information.

[0010] In some embodiments of the present invention, the graph structure encoding information includes a neighbor number sequence and a connection number matrix, wherein the neighbor number sequence includes the number of neighbor nodes in each update community to which each node in the local partial graph data belongs in the update community set determined based on the target inferred graph structure information, and the connection number matrix includes the number of connections between each two update communities in the update community set determined based on the target inferred graph structure information.

[0011] In some embodiments of the present invention, determining global graph structure information based on the shared graph structure information of each target and the target inferred graph structure information to reconstruct the global graph structure includes: For the edges between each node in the target shared graph structure information of each participant participating in the vertical federated graph learning, if the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants is greater than or equal to 1, or the value of the corresponding edge in the target inferred graph structure information is 1, then the existence probability of the edge is determined to be 1. Otherwise, the existence probability of the edge is the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants; Based on the existence probability of each edge, the global edge existence probability matrix is ​​obtained and used as the global graph structure information; The global graph structure information is hard sampled according to the sampling threshold to reconstruct the global graph structure.

[0012] In some embodiments of the present invention, the distributed training of a preset global graph model based on the global graph structure and the local partial graph data by sending intermediate results to a service provider to obtain a trained global graph model includes: The local feature extraction module obtains the neighborhood of each node in the global graph structure based on the global graph structure, and performs feature aggregation based on the neighborhood of each node and the node attribute set in the local graph data to extract local features; Send the extracted local features to the service provider so that the service provider can receive local features from all participants participating in the vertical federated graph learning, perform feature aggregation based on the local features through the global feature aggregation module to obtain global features, and send the global features to all participants; Receive global features from the service party, perform classification based on the global features through the local classification module to output the predicted node category, and send the updated gradient in each iteration to the service party so that the service party receives the updated gradients from each participant and aggregates the updated gradients to obtain the aggregated gradients, until the loss between the predicted node category and the node category label set in the local local graph data is minimized to obtain the trained global graph model, wherein the updated gradients are obtained by back propagation update based on the aggregated gradients obtained in the previous iteration.

[0013] In some embodiments of the present invention, the intermediate result is a perturbed intermediate result generated by adding noise perturbation that satisfies differential privacy to the intermediate result.

[0014] In some embodiments of the present invention, the number of node categories in the initial community in the initial community set and the node set in the local partial graph data is the same; the number of neighbor nodes of each node in each update community in the update community set is the largest, and the number of neighbor nodes is calculated based on the target inferred graph structure information; the update community set is a perturbed update community set generated by adding noise perturbation that satisfies differential privacy to each update community in the update community set.

[0015] Another aspect of the present invention provides a longitudinal federated graph learning device that satisfies differential privacy, the device comprising: a computer device, the computer device comprising a processor and a memory, the memory storing computer instructions, the processor being used to execute the computer instructions stored in the memory, and the device implementing the steps of the aforementioned method when the computer instructions are executed by the processor.

[0016] Another aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which is used to implement the steps of the aforementioned method when executed by a processor.

[0017] Another aspect of the present invention provides a computer program product comprising computer instructions, which implement the steps of the above method when executed by a processor.

[0018] The differentially private longitudinal federated graph learning method and device of the present invention can be used by all participants in the longitudinal federated graph learning to collaboratively train graph neural network models in a differentially private manner. By sharing graph structure information that is critical to the utility of the graph neural network model for feature aggregation during the longitudinal federated graph learning process, the necessary feature aggregation calculations of the graph are achieved while avoiding the introduction of secure multi-party computing technology with huge computational overhead. This can achieve high computational efficiency while satisfying differential privacy constraints and maintaining high model accuracy, thereby greatly reducing computational overhead.

[0019] Additional advantages, objects, and features of the present invention will be set forth in part in the following description and will become apparent to those skilled in the art upon examination of the following or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the structures particularly pointed out in the description and drawings.

[0020] Those skilled in the art will understand that the purposes and advantages that can be achieved by the present invention are not limited to the above specific descriptions, and the above and other purposes that can be achieved by the present invention will be more clearly understood based on the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0022] Figure 1 Schematic diagram of the process of a vertical federated graph learning method that satisfies differential privacy in one embodiment of the present invention; Figure 2 Schematic diagram of the process of obtaining target inference graph structure information in step S110 in one embodiment of the present invention; Figure 3 Schematic diagram of the process of community division and obtaining graph structure coding information in step S110 in one embodiment of the present invention; Figure 4 Schematic diagram comparing the model accuracy effects of the present method and the prior art in one embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0024] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, the accompanying drawings only show structures and / or processing steps closely related to the solutions according to the present invention, while other details that are not closely related to the present invention are omitted.

[0025] It should be emphasized that the term "include / comprises" when used herein refers to the existence of features, elements, steps or components, but does not exclude the existence or addition of one or more other features, elements, steps or components.

[0026] It should also be noted that, unless otherwise specified, the term "connection" herein may refer not only to a direct connection but also to an indirect connection involving an intermediate.

[0027] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or similar components, or the same or similar steps.

[0028] In order to solve the problems of extremely high computational overhead and low training efficiency in the existing vertical federated graph learning method that satisfies differential privacy based on secure multi-party computing technology, the embodiments of the present invention propose a vertical federated graph learning method and device that satisfies differential privacy. The method and device can be used by all participants in the vertical federated graph learning to collaboratively train graph neural network models in a differentially private manner. The method not only satisfies the differential privacy constraints, but also greatly improves the computational efficiency while maintaining high model accuracy while avoiding the introduction of secure multi-party computing technology with huge computational overhead.

[0029] Figure 1 FIG. 1 is a flow chart of a vertical federated graph learning method that satisfies differential privacy in one embodiment of the present invention. Figure 1 As shown, the method is executed by any participant among the participants participating in the vertical federated graph learning, and includes the following steps: Step S110, sending a vertical federated graph learning request to the service provider and each participant other than the local among the participants participating in the vertical federated graph learning, obtaining graph structure encoding information based on the target inferred graph structure information and the updated community set to generate perturbed graph structure encoding information that satisfies differential privacy, wherein the target inferred graph structure information is obtained by iteratively optimizing the initialized inferred graph structure information based on the local local graph data using a local local graph model, and the updated community set is obtained by updating the initial community set obtained by community division of the local local graph data based on the target inferred graph structure information; Step S120: obtaining target shared graph structure information that satisfies differential privacy based on the perturbed graph structure encoding information and the initial shared graph structure information, and sharing the information with other participants, wherein the perturbed initial shared graph structure information is the initial shared graph structure information that satisfies differential privacy generated by merging the target inferred graph structure information and the local partial graph data; Step S130, receiving target shared graph structure information from other participants, and determining global graph structure information based on the target shared graph structure information and the target inferred graph structure information to reconstruct the global graph structure; Step S140: Distributed training is performed on the preset global graph model based on the global graph structure and the local partial graph data by sending intermediate results to the service provider to obtain a trained global graph model, wherein the global graph model includes a local feature extraction module, a global feature aggregation module located on the service provider, and a local classification module that are communicatively connected in sequence, and the intermediate results include the local features extracted in each iteration and the updated gradient. In the first iteration, the updated gradient is the initial gradient.

[0030] The graph learning approach described in this method primarily refers to the training of graph neural network models. This approach is currently the most advanced graph learning method and has achieved state-of-the-art performance in many graph-based tasks, such as graph data mining, recommender systems, and bioinformatics analysis. The basic principle of graph learning is to aggregate node attributes (e.g., user gender, age, etc.) or node embeddings of adjacent nodes based on the graph structure and map these aggregated embeddings through a parameterized nonlinear transformation to learn feature representations for all nodes in the graph. Taking the node classification task, a baseline task of graph neural networks, as an example, given a simple undirected graph, each layer of the graph neural network aggregates the node embeddings of the previous layer using a differentiable and permutation-invariant aggregation function (e.g., summation or averaging) based on the set of adjacent nodes described by the adjacency matrix representing the graph structure. A differentiable nonlinear transformation is then performed to obtain the node embedding of the node at that layer. At the input layer of the network, the node embeddings are initialized to the node attributes themselves. At the final layer, the model outputs a feature vector, the graph node embedding, which is used to perform downstream tasks. During the training process of the graph neural network model for the node classification task, the node embedding is compared with the node category label through the softmax layer, and the model parameters are updated through the backpropagation algorithm using a predefined loss function (such as the cross-entropy cost function), thereby completing the training of the graph neural network model.

[0031] In step S110, any participant first sends a vertical federated graph learning request to the service provider and other participants, so that the service provider and other participants and the participant issuing the request jointly participate in the vertical federated graph learning. The participant can be a client, etc., and the service provider can be a server, etc., which is a semi-honest server.

[0032] The local graph model can be pre-trained by the local graph data held. The local graph data held by the u-th participant can be G u =(V u ,A u ,X u ,Y u ), where V u Represents a node set, A u is the edge set E used to represent the local graph structureu The adjacency matrix, that is, if edge (i, j)∈E u ,but on the contrary i represents a node, j represents a neighboring node of node i; X u Represents a set of node attributes, Y u Represents a set of node category labels, which are real node categories. Node categories can be, for example, active users or inactive users, male or female user gender, etc. Participants are based on local graph data G u Pre-training to obtain the local local graph neural network model (abbreviated as local local graph model) f gnnl , and uses the pre-trained local graph model to infer graph structure information as the learning and optimization target through reverse learning. This makes the optimized inferred graph structure information have similar utility to the adjacency matrix in feature aggregation of the local map, that is, it can retain useful information on the graph while avoiding privacy leakage, thereby ensuring the utility of shared information in vertical federated graph learning. In order to effectively learn the inferred graph structure information, participants can use a two-layer network GCN (Graph Convolutional Network) model as the preset local graph model, which can be expressed as: Among them, Z u represents the node embedding set, ReLU represents the nonlinear activation function, Represents the first prediction node category set, σ represents the Sigmod function, and denote the learnable parameters of the first and second layers of the graph convolutional network, respectively. Represents the cost function, specifically the cross entropy cost function CE can be used to evaluate the difference between the first predicted node category set and the node category label set. The above local graph neural network is pre-trained by minimizing the cost function to obtain the trained parameters and That is, a pre-trained local graph model is obtained. The pre-trained local graph model is the local graph model obtained by the pre-training described above.

[0033] Afterwards, the inferred graph structure information is optimized through the local graph model and its cost function and gradient descent method during training, thereby extracting the key graph structure that has the most significant impact on the effectiveness of the graph neural network. Figure 2 As shown, each element in the inferred graph structure matrix representing the inferred graph structure information is first initialized to 0, that is, the initialized inferred graph structure information is obtained, and then the local local graph model is used based on the initialized inferred graph structure matrix Output the second prediction node category set It can be expressed as: After obtaining the second predicted node category set from the inferred graph structure matrix, the target inferred graph structure matrix is ​​finally obtained by iteratively optimizing the following inverse learning cost function It can be expressed as: Where ρ represents the cost function The weight parameter, represents the sparse regularization term weight parameter, Represents the inferred graph structure matrix The Frobenius norm (i.e., F-norm, which is used to measure the overall "size" or "magnitude" of the matrix). The cost function of inverse learning is achieved by forcing the matrix from the inferred graph structure to The second prediction node category set At the same time, the adjacency matrix A from the real local graph u The first prediction node category set and the true node category label set Y u Consistent, making the inferred graph structure matrix It can not only have good data utility in GNN learning tasks, but also evaluate which edges in the real local graph structure are more important for accurately predicting node categories. In addition, by penalizing the inferred graph structure matrix The Frobenius norm of is used to control its sparsity. Among them, in order to control the inferred graph structure matrix The range of elements in , the participants will be after each gradient update The element a in is projected into the value range (0, 1), which can be expressed as: Among them, M represents the projection function, t represents the number of training steps or iterations, ζ represents the learning rate of reverse learning, and g represents the reverse learning gradient.

[0034] After the reverse learning is completed, the participants obtain the local graph model f gnn1 Utility-critical target inference graph structure matrix And the adjacency matrix A of the local graph u and the target inference graph structure matrix After merging, the initial shared graph structure matrix is ​​obtained. The initial shared graph structure matrix serves as the initial shared graph structure information and can be expressed as: Among them, the initial shared graph structure information The local graph privacy information of the participants is included in the data. If it is directly shared with other participants, there may be a risk of privacy information leakage. Therefore, it needs to be shared in a way that satisfies the differential privacy constraint. To ensure that the shared graph structure information strictly satisfies the edge differential privacy, the participants first share the initial shared graph structure information. After binarization, it can be expressed as: This means that if If the value of the element representing the importance of the edge in is higher than the sampling threshold, the edge is selected as part of the initial shared graph structure. The sampling threshold can be 0.5. After that, the participants adopt a privacy budget of ∈ r The random response mechanism function initially shares the graph structure information Add random noise perturbation to perform differential privacy protection, which can be calculated as follows: Based on this, the initial shared graph structure information after perturbation Satisfy∈ r -Edge differential privacy.

[0035] In order to extract the graph structure features from the local graph data, that is, to obtain the graph structure encoding information, the participants first perform the initial community division based on the target inferred graph structure information, and then further adjust the initial community division results to obtain the final community division results, such as Figure 3 Specifically, we can first use the node attribute set and node category label set (X u , Y u ) pre-trains the multi-layer perceptron (MLP) to obtain a community partitioning network. The community partitioning network is used to divide the node set V of the local graph into multiple initial communities. The number of node categories in the initial community and the node set is the same, that is, the initial community set is obtained. Represents the set of nodes divided into the corresponding initial community, and c represents the number of initial communities or the number of node categories. Other community division methods can also be used for initial community division, and are not limited to the above methods. Since the community division network itself may have misclassifications that cause nodes to be divided into the wrong community, it is necessary to adjust the community to which each node belongs in order to more accurately calculate the graph structure encoding information. For each node in each initial community in the initial community set, the graph structure matrix is ​​inferred according to the target Calculate the number of neighbor nodes of the node in each initial community, take the initial community with the largest number of neighbor nodes as the candidate community, and add the node to the candidate community to obtain the updated community (both the communities before and after the node is added are updated), thus obtaining the updated community set obtained after iterative community update for each node.

[0036] Since the community adjustment process depends on the target inferred graph structure matrix In order to satisfy the edge differential privacy constraint, the privacy budget is ∈ e The utility function of the exponential mechanism adds noise perturbations to the results after each iterative update during the community adjustment process, where the number of neighbor nodes of the node in different communities is used as the exponential mechanism. Therefore, in this process, the final updated community set is It can also be a set of updated communities after perturbation that satisfies differential privacy, where represents the updated community after the cth perturbation. Specifically, since the target inference graph structure matrix Adding or deleting an edge will only change the number of neighbor nodes of the node included in the community to 1 at most, so the sensitivity ΔEM of the exponential mechanism is 1. Since adding or deleting an edge will affect the two endpoint nodes of the edge in the community adjustment process, the privacy budget used in each iterative update is set to

[0037] Participants obtain the final community division results The target inferred graph structure information is used together to calculate the coarse-grained graph structure encoding information, and noise perturbation is added to the graph structure encoding information so that the perturbed graph structure encoding information satisfies differential privacy. Based on the final community division result, the target inferred graph structure matrix can be The edges in are divided into intra-community edges and inter-community edges. The final community division result can be an updated community set without adding noise perturbation, or an updated community set with added noise perturbation that satisfies differential privacy. In this embodiment, Figure 3 As shown, the graph structure encoding information includes a neighbor number sequence and a connection number matrix, wherein the neighbor number sequence includes the number of neighbor nodes in each update community to which each node in the local partial graph data determined based on the target inferred graph structure information belongs in the update community set, and the connection number matrix includes the number of connections between each two update communities in the update community set determined based on the target inferred graph structure information. For each node i in the local partial graph, node i in the update community set Update community The calculation method of the number of neighbor nodes in can be expressed as: Then, the neighbor number sequence composed of the number of neighbor nodes of all nodes is n represents the number of nodes. Any pair of update communities in and The number of connections between the two communities can be calculated as follows: Among them, 1≤k<m≤c. Then, the connection matrix composed of the connection numbers of all updated community pairs is ε u ∈R c×c .

[0038] Afterwards, the participants use the Laplace mechanism function with a privacy budget of ∈1 to add noise perturbations to the neighbor number sequence and the connection number matrix, so that the perturbed neighbor number sequence and the connection number matrix satisfy ∈1-edge differential privacy. The change of one edge in the network will cause the number of neighbor nodes of the two nodes to change. Therefore, the sensitivity of the Laplace mechanism to the neighbor number sequence is 2. The number of neighbor nodes after the disturbance can be expressed as: In contrast, The change of one edge in the network will at most cause the number of connections between the corresponding two updated communities to change to 1. Therefore, the sensitivity of the Laplace mechanism to the connection matrix is ​​1. The number of connections after perturbation can be expressed as: Since the neighbor number sequence and the connection matrix ε u Respective statistical target inference graph structure matrix Therefore, the two graph structures can encode information using the same privacy budget ∈ 1. Moreover, considering that the number of neighbor nodes and the number of connections cannot be negative, the perturbed neighbor number sequence And the perturbed connection matrix Clipped to non-negative values.

[0039] In some embodiments, in step S120, obtaining target shared graph structure information satisfying differential privacy based on the perturbed graph structure encoding information and the initial shared graph structure information includes the following steps: The edge existence probability matrix is ​​estimated based on the perturbed graph structure encoding information and used as the prior probability; The initial shared graph structure information after perturbation is used as evidence, and Bayesian estimation is performed on the target inferred graph structure information based on the prior probability and evidence. The edge posterior existence probability matrix that satisfies differential privacy is obtained and used as the target shared graph structure information.

[0040] In step S120, the participants calculate the edge existence probability matrix as the prior probability of Bayesian estimation based on the coarse-grained graph structure encoding information after the disturbance, and perform Bayesian estimation with the fine-grained initial shared graph structure information after the disturbance as evidence, thereby calculating the edge posterior existence probability matrix of the target inferred graph structure matrix as post-noise reduction processing, and sharing the edge posterior existence probability matrix as the target shared graph structure information to each participant participating in the vertical federated graph learning.

[0041] Specifically, the edge existence probability matrix can be estimated by the Chung-Lu model based on the neighbor number sequence and the connection number matrix after the perturbation. The estimated edge existence probability matrix consists of two parts, namely, the updated intra-community edge existence probability matrix and the updated inter-community edge existence probability matrix. First, based on the neighbor number sequence after the perturbation Calculate the same update community The probability p that there is an edge between each node i and each node j in ij , the calculation method can be expressed as: in, and Respectively represent the update communities to which nodes i and j belong The number of neighbor nodes after the disturbance, Indicates the number of neighbor nodes after disturbance of each node s in the updated community Therefore, updating the edge existence probability matrix within the community includes updating the edge existence probability of each updated community in the community set. Then, according to the perturbed connection number matrix Calculate the probability of existence of edges between nodes in different update communities. Each node i within and update the community The probability that there is an edge between each node j in The calculation method can be expressed as: in, and They represent the number of nodes in the kth and mth update communities, respectively. Therefore, the update community edge existence probability matrix includes the existence probability of each update community edge in the update community set.

[0042] For the initial shared graph structure matrix after perturbation Each pair of elements in Since the privacy budget of the random response mechanism ∈ r is known, so the flip probability of each edge is is known. Participants can rCalculate the initial shared graph structure matrix after perturbation when the condition exists between nodes i and j and when the condition does not exist between nodes i and j Each pair of elements in The likelihood probability of whether a connection exists can be calculated as: Among them, l ij Indicates When the condition that there is an edge between nodes i and j in the table is Likelihood probability; l′ ij Indicates When there is no edge between nodes i and j in the Likelihood probability. Since the edge has probability p ij 、 and the likelihood probability l ij , l′ ij Is the target inference graph structure matrix Based on the independent observation results, the participants can estimate the probability of the posterior existence of the edge through the Bayesian rule, and the calculation formula can be: Where, 1≤i≠j≤|V|, |V| represents the number of nodes in the node set V; Indicates that evidence is observed After that, the posterior probability that the node pair (i, j) has an edge in the target inference graph structure matrix; Pr represents the probability.

[0043] Since the Bayesian estimation is only based on the initial shared graph structure matrix after perturbation Neighbor number sequence after perturbation And the perturbed connection matrix The three kinds of information that satisfy the edge differential privacy constraints are calculated, so the Bayesian estimation can be regarded as the post-processing process of the differential privacy algorithm, and the edge posterior has a probability The differential privacy constraint is still satisfied. The posterior existence probability of each edge The edge posterior existence probability matrix The ultimate goal is to share graph structure information.

[0044] In some embodiments, step S130, determining global graph structure information based on the shared graph structure information of each target and the target inferred graph structure information to reconstruct the global graph structure, includes the following steps: For the edges between each node in the target shared graph structure information of each participant participating in the vertical federated graph learning, if the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants is greater than or equal to 1, or the value of the corresponding edge in the target inferred graph structure information is 1, then the existence probability of the edge is determined to be 1. Otherwise, the existence probability of the edge is the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants; Based on the existence probability of each edge, the global edge existence probability matrix is ​​obtained and used as the global graph structure information; The global graph structure information is hard sampled according to the sampling threshold to reconstruct the global graph structure.

[0045] In step S130, participant u receives the edge posterior existence probability matrix shared by all participants except the local participant in the vertical federated graph learning and obtains After that, U represents the number of participants participating in the learning, and the edge posterior existence probability matrices from other participants are merged to calculate the global edge existence probability matrix That is, the global graph structure information is obtained, and then the global graph structure is reconstructed based on the global graph structure information. In this embodiment, since each participant holds a non-overlapping subgraph structure or local graph structure, The existence probability of each edge in is the accumulation of the posterior existence probability of the corresponding edges shared by other participants, which can be calculated as: Among them, when the cumulative sum of the edge posteriori existence probabilities between nodes i and j shared by other participants is higher than 1, or the target inference graph structure matrix of participant u locally When there is an edge between nodes i and j in is limited to 1, that is, the edge must exist. In other embodiments, each participant may also hold overlapping subgraph structures, which does not constitute a limitation of the present invention.

[0046] Afterwards, the global edge existence probability matrix representing the global graph structure information is calculated based on the sampling threshold. Perform hard sampling to reconstruct the global graph structure It can be expressed as: The sampling threshold can be 0.5. Afterwards, the participants can Perform distributed training and learning of global graph models (i.e., global graph neural network models).

[0047] In some embodiments, step S140, performing distributed training on a preset global graph model based on the global graph structure and the local partial graph data by sending intermediate results to a service provider to obtain a trained global graph model, includes the following steps: obtaining a neighborhood of each node in the global graph structure based on the global graph structure by a local feature extraction module, performing feature aggregation based on the neighborhood of each node and a set of node attributes in the local partial graph data to extract local features; Send the extracted local features to the service provider so that the service provider can receive local features from all participants participating in the vertical federated graph learning, perform feature aggregation based on the local features through the global feature aggregation module to obtain global features, and send the global features to all participants; Receive global features from the service party, perform classification based on the global features through the local classification module to output the predicted node category, and send the updated gradient in each iteration to the service party so that the service party receives the updated gradients from each participant and aggregates the updated gradients to obtain the aggregated gradients, until the loss between the predicted node category and the node category label set in the local local graph data is minimized to obtain the trained global graph model, wherein the updated gradients are obtained by back propagation update based on the aggregated gradients obtained in the previous iteration.

[0048] In the distributed iterative training process of the model in step S140, the global graph model needs to first perform global feature extraction, and then classify according to the extracted global features. The training process is a semi-supervised learning process. In the process of feature extraction, for each node in the global graph structure, the global graph neural network model needs to aggregate the complete high-order neighborhood structure information of the node in the global graph structure and the node attributes or features of all its neighboring nodes to generate a node embedding with rich feature representation capabilities, thereby obtaining the node embedding of all nodes in the global graph structure. In order to achieve the above purpose, the present method designs two processes: local feature extraction (aggregation) and global feature aggregation of the service party. Specifically, in the local feature aggregation process, for each node in the global graph structure, the participating party first locally aggregates the complete high-order neighborhood structure information of the node and some node attributes in the global graph structure held by the party (locally) (i.e., the node attribute set in the local local graph data), generates the local embedding of the node, and obtains the local embedding set composed of the local embeddings of all nodes to achieve local feature extraction. The participants then send their local embedding sets to the server. After receiving the local embedding sets from each participant, the server performs global feature aggregation to obtain a global embedding set, thereby achieving global feature aggregation on the server. In other words, the global graph model includes a local feature extraction module on the participant, a global feature aggregation module on the server, and a local classification module on the participant, all connected in sequence.

[0049] The local feature aggregation process is similar to the commonly used GNN (Graph Neural Network) feature aggregation process. Participants reconstruct the global graph structure Calculated In this embodiment, the GraphSAGE graph neural network model is used as an example for local feature extraction. This is a widely used backbone graph neural network. GraphSAGE is used to introduce an aggregation function Agg(·), which is used to sample all neighboring nodes from the neighborhood of each node and perform feature aggregation to update the embedding of each node. The calculation method can be expressed as: in, represents the neighborhood embedding of node i in the hth layer, Agg represents the aggregation function, which is used to sample all neighbor nodes j of node i from its neighborhood N(i) and aggregate the node embeddings of these neighbor nodes in the h-1th layer. To generate neighborhood embeddings; represents the node embedding of node i at layer h. When h = 0, the node embedding of node i is the locally held node attribute, i.e. S represents the nonlinear activation function, W h Denotes the learnable model parameters of the hth layer, and Concat represents the concatenation function. There are three types of aggregation functions in GraphSAGE: average, LSTM, and pooling. To address the computational efficiency bottleneck in vertical federated graph learning, this embodiment uses the average function as the aggregation function.

[0050] According to the above formula, GraphSAGE embeds the neighborhood of node i in the hth layer into and the node embedding of node i at layer h-1 Splice them together and get the node embedding of node i in layer h through the linear mapping layer in the local feature extraction module The node embedding aggregates the neighborhood structure information within the h-hop neighborhood of node i and its locally held neighbor node attribute information. Therefore, the node embedding of node i at the hth layer is That is the local embedding of the node. Finally, the local embedding of each node in the global graph structure is Constructing a local embedding set (i.e. local features).

[0051] During the distributed iterative training process, participants need to send the local embedding set obtained by the local feature extraction module in each iteration to the server for global feature aggregation. Furthermore, in order to avoid the risk of privacy information leakage during the forward propagation process of model training, before sending the generated local embedding set to the server each time, participants use a total privacy budget of ∈ g The Gaussian mechanism function adds noise perturbation to the local embedding set so that the perturbed local embedding set satisfies differential privacy protection.

[0052] Afterwards, the participants embed the local set (the perturbed local embedding set that satisfies differential privacy) Upload to the server so that the server can perform global feature aggregation based on the local embedding sets uploaded by each participant participating in the vertical federated graph learning, so that the global feature contains the complete neighborhood structure information of each node and the attribute information of all neighboring nodes. Since the local embedding sets uploaded by each participant are calculated using the different node attribute sets they hold, the importance of each local embedding set to the global feature is different. In this embodiment, the global feature aggregation module uses a regression aggregation function to learn the importance differences between different local embedding sets. Specifically, the global embedding of the i-th node It can be expressed as: Among them, W g Denotes the learnable regression weight parameter, S denotes the nonlinear activation function, and ⊙ denotes the Hadamard product. Global embedding of all nodes through the global graph structure Form a global embedding set Z g (i.e. global features).

[0053] Next, the server embeds the global set Z g Sent to each participant. In this embodiment, the participant uses the softmax function (normalized exponential function) as the local classification module and trains the classification module based on the global embedding set and the node category label set in the local graph data (partial node labels in the global graph structure), which can be expressed as: in, represents the predicted node category of node i, Represents the local node category label set Y u The category label of node i in . During the training process, the participants use the predicted node category and corresponding labels Calculate the cross entropy cost function And calculate the model gradient through the back propagation algorithm To update the global graph model parameters θ (which include the parameters used in the local feature extraction module, the global feature aggregation module, and the local classification module) to minimize and Specifically, the participants will calculate the initial gradient Upload to the server so that the server can receive the initial gradients from each participant, and perform mean aggregation on each initial gradient to obtain the aggregated gradient. Based on the aggregated gradient, the server updates the parameters and gradients in the global feature aggregation module stored by the server, and continues to backpropagate the gradient to each participant to update the parameters and initial gradients in the local feature extraction module stored by each participant, so that the participant obtains the updated gradient. Then, based on the updated gradient, multiple updates are performed according to a similar process to iteratively train the global graph model, and finally a trained global graph model is obtained. During this process, the participants need to send the updated gradients in each iteration to the server for aggregation.

[0054] Furthermore, during the back propagation process of model training, each time a participant uploads an updated gradient to the server, there is a risk of leaking private information, so differential privacy protection is required. Before sending the updated gradient to the server each time, the participant uses a total privacy budget of ∈ g The Gaussian mechanism is updated after the gradient Noise perturbation is added to the model so that the updated gradient after the perturbation satisfies differential privacy, thereby ensuring that all intermediate results of the model that leave the local computing environment of the participants satisfy the differential privacy constraints.

[0055] In this approach, privacy budget consumption is determined by standard moment accounting methods.

[0056] An embodiment of the present invention further provides a differentially private vertical federated graph prediction method, which is also performed by any of the participants in the vertical federated graph learning, and includes the following steps: Acquire local graph data, and obtain a global graph structure according to a process similar to steps S110 to S130 in the differentially private vertical federated graph learning method described in the aforementioned embodiments, wherein the local graph data includes a node set, an adjacency matrix representing an edge set, and a node attribute set; The trained global graph model obtained by the differentially private longitudinal federated graph learning method described in the aforementioned embodiments is used to predict node categories based on the global graph structure and local graph data. Specifically, the trained local feature extraction module performs feature aggregation based on the global graph structure and the node attribute set in the local graph data to extract local features; the extracted local features are sent to the service provider so that the service provider can receive the local features of each participant in the longitudinal federated graph learning; the trained global feature aggregation module performs feature aggregation based on the local features to obtain global features, and the global features are sent to each participant; the global features received from the service provider are classified based on the global features by the trained local classification module to output the predicted node category.

[0057] In addition to performing the above node classification tasks, this method can also perform edge classification tasks. Specifically, this can be achieved by replacing the node attribute set and node category label set in the local graph data with the edge attribute set and edge category label set, respectively.

[0058] Finally, the present invention verifies the model training efficiency and prediction accuracy of this method and compares it with the existing technology. The datasets used include three open source datasets: Cora, Citeseer, and PubMed, as shown in Table 1: Table 1: Dataset information Dataset Number of nodes Number of edges Node feature dimension Task Cora 2708 10556 1433 7 types of node classification Citeseer 3327 9228 3703 6 types of node classification PubMed 19717 88651 500 3 types of node classification Existing technologies include five existing methods and their trained models: Central, Local, VFGNN, DPVGL, and FDML. Central refers to the GraphSAGE model trained entirely centrally (server-side). Local refers to the GraphSAGE model trained entirely locally by participants. VFGNN involves first generating a public-private key pair by a semi-honest server. Participants then use the public-private key pair to perform multi-party secure feature aggregation under ciphertext using secret sharing techniques. The decrypted feature aggregation results are uploaded to the server for further feature aggregation, generating the final node embedding for node classification. A Gaussian mechanism is used to protect the privacy of intermediate results transmitted between participants and the server. DPVGL involves directly perturbing the subgraph structure held by each participant through a random response mechanism with noise and sharing it with all participants. Participants then merge all perturbed subgraph structures to generate a global graph structure, perform feature aggregation based on the global graph structure, and finally upload the feature aggregation results to the server to generate node embeddings for node classification. Similarly, DPVGL uses a Gaussian mechanism to protect the privacy of intermediate results transmitted between participants and the server. FDML involves participants using local subgraphs and local graph neural networks to generate local node embeddings. Each participant's local node embeddings are then uploaded to a semi-honest server for weighted summation and node classification. FDML also uses a Gaussian mechanism to protect the privacy of intermediate results transmitted between participants and the server.

[0059] Figure 4 The comparison of the model prediction accuracy of this method and the above five existing technologies under different privacy budgets ∈ is shown. Figure 4 As shown, the prediction accuracy of the model trained by our method (Ours) varies with the privacy budget. When the privacy protection strength is moderate, our method can achieve model prediction accuracy similar to that of the VFGNN method. As the privacy budget increases, the model prediction accuracy of our method gradually approaches that of the VFGNN method based on secure multi-party computation technology and the fully centralized Central method. In most cases, the model prediction accuracy of our method is significantly higher than that of the fully locally trained Local method and the FDML method that does not consider the feature aggregation computation problem in graph neural networks. This shows that our method can maintain high model prediction accuracy during the vertical federated graph learning process.

[0060] Table 2 shows the comparison of model training efficiency between our method and existing methods when the privacy budget ∈=8, and gives the average training time in seconds for 5 different random seeds.

[0061] Table 2: Comparison of training time between this method and existing technologies (unit: seconds) This shows that while VFGNNs, which incorporate secure multi-party computation technology for feature aggregation during graph neural network computations, achieve high model accuracy, they incur the highest computational overhead and training time, orders of magnitude longer than other methods. In contrast, this method significantly improves computational efficiency while maintaining high model accuracy, with training time comparable to that of the fully centralized Central method. Therefore, this method achieves a balance between high training efficiency, low computational overhead, and high prediction accuracy.

[0062] Corresponding to the above method, the present invention also provides a vertical federated graph learning device that satisfies differential privacy, which includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the device implements the steps of the above method.

[0063] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method. The computer-readable storage medium may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the art.

[0064] An embodiment of the present invention further provides a computer program product, comprising computer instructions, which implement the steps of the aforementioned method when executed by a processor.

[0065] It should be understood by those skilled in the art that the various exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is specifically performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0066] It should be understood that the present invention is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, a detailed description of known methods is omitted. In the above embodiments, several specific steps are described and illustrated as examples. However, the method of the present invention is not limited to the specific steps described and illustrated. Those skilled in the art may make various changes, modifications, and additions, or change the order of the steps after understanding the spirit of the present invention.

[0067] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or replace features of other embodiments.

[0068] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations to the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A vertical federated graph learning method that satisfies differential privacy, characterized by: The method comprises: Sending a vertical federated graph learning request to the service provider and all other participants except the local party among the participants participating in the vertical federated graph learning, obtaining graph structure encoding information based on the target inferred graph structure information and the updated community set to generate perturbed graph structure encoding information that satisfies differential privacy, wherein the target inferred graph structure information is obtained by iteratively optimizing the initialized inferred graph structure information based on the local local graph data using a local local graph model, and the updated community set is obtained by updating the initial community set obtained by community division of the local local graph data based on the target inferred graph structure information; The target shared graph structure information that satisfies differential privacy is obtained based on the perturbed graph structure encoding information and the initial shared graph structure information, and is shared with other participants. The perturbed initial shared graph structure information is the initial shared graph structure information that satisfies differential privacy and is generated based on the initial shared graph structure information obtained by merging the target inferred graph structure information and the local partial graph data. Receiving target shared graph structure information from other participants, and determining global graph structure information based on the target shared graph structure information and the target inferred graph structure information to reconstruct the global graph structure; Based on the global graph structure and local local graph data, the preset global graph model is distributedly trained by sending intermediate results to the service party to obtain a trained global graph model, wherein the global graph model includes a local feature extraction module, a global feature aggregation module located on the service party and a local classification module that are communicatively connected in sequence, and the intermediate results include the local features extracted in each iteration and the updated gradient. In the first iteration, the updated gradient is the initial gradient.

2. The method according to claim 1, characterized in that The local partial graph data includes a node set, an adjacency matrix representing an edge set, a node attribute set, and a node category label set representing a real node category.

3. The method according to claim 1, characterized in that The target shared graph structure information satisfying differential privacy is obtained based on the perturbed graph structure encoding information and the initial shared graph structure information, including: The edge existence probability matrix is ​​estimated based on the perturbed graph structure encoding information and used as the prior probability; The initial shared graph structure information after perturbation is used as evidence, and Bayesian estimation is performed on the target inferred graph structure information based on the prior probability and evidence. The edge posterior existence probability matrix that satisfies differential privacy is obtained and used as the target shared graph structure information.

4. The method according to claim 3, characterized in that The edge existence probability matrix includes an updated intra-community edge existence probability matrix obtained based on the perturbed neighbor number sequence in the perturbed graph structure encoding information and an updated inter-community edge existence probability matrix obtained based on the perturbed connection number matrix in the perturbed graph structure encoding information.

5. The method according to claim 1, 3 or 4, characterized in that The graph structure encoding information includes a neighbor number sequence and a connection number matrix. The neighbor number sequence includes the number of neighbor nodes in each update community to which each node in the local partial graph data belongs in the update community set determined based on the target inferred graph structure information. The connection number matrix includes the number of connections between each two update communities in the update community set determined based on the target inferred graph structure information.

6. The method according to claim 3, characterized in that The determining of global graph structure information based on the shared graph structure information of each target and the target inferred graph structure information to reconstruct the global graph structure includes: For the edges between each node in the target shared graph structure information of each participant participating in the vertical federated graph learning, if the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants is greater than or equal to 1, or the value of the corresponding edge in the target inferred graph structure information is 1, then the existence probability of the edge is determined to be 1. Otherwise, the existence probability of the edge is the sum of the edge posterior existence probabilities of the corresponding edges in the target shared graph structure information shared by other participants; Based on the existence probability of each edge, the global edge existence probability matrix is ​​obtained and used as the global graph structure information; The global graph structure information is hard sampled according to the sampling threshold to reconstruct the global graph structure.

7. The method according to claim 1, characterized in that The method of performing distributed training on a preset global graph model based on the global graph structure and the local local graph data by sending intermediate results to the service provider to obtain a trained global graph model includes: The local feature extraction module obtains the neighborhood of each node in the global graph structure based on the global graph structure, and performs feature aggregation based on the neighborhood of each node and the node attribute set in the local graph data to extract local features; Send the extracted local features to the service provider so that the service provider can receive local features from all participants participating in the vertical federated graph learning, perform feature aggregation based on the local features through the global feature aggregation module to obtain global features, and send the global features to all participants; Receive global features from the server, perform classification based on the global features through a local classification module to output a predicted node category, and send the updated gradients in each iteration to the server so that the server receives the updated gradients from each participant and aggregates the updated gradients to obtain an aggregated gradient until the loss between the predicted node category and the node category label set in the local local graph data is minimized to obtain a trained global graph model, wherein the updated gradients are obtained by backpropagation based on the aggregated gradients obtained in the previous iteration; The number of node categories in the initial community in the initial community set and the node set in the local partial graph data is the same; the number of neighbor nodes of each node in each update community in the update community set is the largest, and the number of neighbor nodes is calculated based on the target inferred graph structure information; the update community set is a perturbed update community set generated by adding noise perturbation that satisfies differential privacy to each update community in the update community set; the intermediate result is a perturbed intermediate result generated by adding noise perturbation that satisfies differential privacy to the intermediate result.

8. A vertical federated graph learning device that satisfies differential privacy, comprising a processor, a memory, and computer instructions stored in the memory, characterized in that: The processor is configured to execute the computer instructions. When the computer instructions are executed, the device implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images