Federated learning method, apparatus, electronic device, storage medium, and program product

Abstract

Embodiments of the present application disclose a federated learning method and device, electronic equipment, a storage medium and a program product. The method comprises: obtaining graph data; determining a to-be-hidden edge and a to-be-added edge from the graph data; performing perturbation processing on the graph data according to the to-be-hidden edge and the to-be-added edge to obtain target graph data; performing aggregation calculation on each node in the target graph data using a federated model to obtain a node representation of each node; and updating parameters of the federated model according to a back propagation gradient sent by a service party to obtain a target federated model. The embodiments of the present application can perturb the graph data based on the to-be-added edge and the to-be-hidden edge, implement privacy protection of the graph data, so that the calculated node representation does not have the risk of leaking the graph structure and features of the graph data, and thus the security of data in federated learning is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a federated learning method and device, electronic equipment, storage medium and program product. BACKGROUND

[0002] The federated learning technology trains an effective machine learning model through interaction of models or intermediate results, can avoid direct transmission of data, and provides a new solution for cross-department, cross-organization and cross-industry data cooperation. The graph neural network model is a deep learning method that can comprehensively utilize the graph structure and sample features in the data set, and can realize prediction of nodes and edges.

[0003] In federated learning based on the graph neural network, each participant usually calculates node representations on locally held features by using private sharing and homomorphic encryption technology, but the edges in the graph data are not protected in the process of calculating the node representations, so that the node representations have the risk of leaking the graph structure and features, and thus the security of data in federated learning is low. SUMMARY

[0004] The embodiments of the present application provide a federated learning method, device, electronic equipment, storage medium and program product, which can improve the security of data in federated learning.

[0005] The embodiments of the present application provide a federated learning method, which comprises:

[0006] Obtaining graph data, the graph data comprising nodes and edges, the edges being used to represent the association relationship between the nodes;

[0007] Determining hidden edges and added edges from the graph data, the added edges being at least one of the edges between a specified node pair, the specified node pair being two nodes satisfying a preset condition, and the hidden edges being at least one of the edges;

[0008] Performing perturbation processing on the graph data according to the hidden edges and the added edges to obtain target graph data;

[0009] Using a federated model to perform aggregation calculation on each node in the target graph data to obtain node representations of each node in the target graph data;

[0010] Updating parameters of the federated model according to a back propagation gradient sent by a service party to obtain a target federated model, the back propagation gradient being calculated by the service party according to held label data and a processing result of the node representations by the federated model.

[0011] The embodiments of the present application also provide a federated learning device, which comprises:

[0012] An acquisition module is used to acquire graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes.

[0013] The determination module is used to determine the edges to be hidden and the edges to be added from the graph data. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges.

[0014] The perturbation module is used to perturb the graph data according to the edge to be hidden and the edge to be added, so as to obtain the target graph data;

[0015] The aggregation module is used to perform aggregation calculations on each node in the target graph data using a federated model to obtain the node representation of each node in the target graph data.

[0016] The learning module is used to update the parameters of the federated model based on the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

[0017] In some embodiments, the determining module further includes:

[0018] A node pair determination unit is used to obtain all node pairs from the graph data, wherein each node pair includes any two nodes;

[0019] The designated node pair determination unit is used to determine, based on the graph data, node pairs among all node pairs that do not have edges as designated node pairs;

[0020] The edge-to-be-added determination unit is used to calculate the existence probability of the edge between the specified node pairs and determine the edge to be added based on the existence probability;

[0021] The edge to be hidden determination unit is used to determine the edges to be hidden from the edges of the graph data according to a preset differential privacy budget.

[0022] In some embodiments, the unit for determining the new edge is further configured to:

[0023] Perform convolution processing on the specified node pair to obtain the node convolution features corresponding to the two nodes in the specified node pair;

[0024] Based on the node convolution features, the similarity between two nodes in the specified node is calculated to obtain the similarity of each specified node pair;

[0025] The similarity of each specified node pair is mapped to a specified region to obtain the probability of the existence of an edge between the specified node pairs;

[0026] Based on the probability of the existence of the edge between the specified node pairs, the edge between the specified node pairs is determined as the edge to be added.

[0027] In some embodiments, the edge-to-be-hidden determination unit is further configured to:

[0028] The probability of edge retention is calculated based on a preset differential privacy budget.

[0029] Based on the retention probability of the edges, the edges to be hidden are determined from the edges of the graph data.

[0030] In some embodiments, the disturbance module is further configured to:

[0031] The edge to be hidden is deleted from the graph data, and the edge to be added is added to the graph data to obtain the target graph data.

[0032] In some embodiments, the disturbance module is further configured to:

[0033] In the graph data, all edges other than the edge to be hidden are identified as edges to be retained;

[0034] All nodes in the graph data, the edges to be retained, and the edges to be added are identified as the target graph data.

[0035] In some embodiments, the federation model includes a first model deployed on the participating party, the first model including at least one network layer, and the aggregation module further includes:

[0036] The adjacency node determination unit is used to determine the adjacency nodes of each node in the target graph data;

[0037] The neighborhood feature determination unit is used to determine the node features of the adjacent nodes as neighborhood features from the node features of each node output by the previous network layer.

[0038] The mean unit is used to perform mean processing on the neighborhood features of each node to obtain the target neighborhood features of each node;

[0039] The fusion unit is used to fuse the target neighborhood features and the node features of each node output by the previous network layer based on the density parameters of the current network layer to obtain the node features output by the current network layer.

[0040] The initial node representation determination unit is used to determine the output of the last network layer in the first model as the initial node representation of each node.

[0041] The perturbation unit is used to perform noise perturbation processing on the initial node representation to obtain the node representation of each node.

[0042] In some embodiments, the disturbance unit is further configured to:

[0043] Obtain the degree of each node in the graph data, where the degree of a node is the number of edges of that node;

[0044] Noise data is obtained by sampling from a preset noise distribution;

[0045] The node representation of each node is calculated based on the noise data, the degree of the node, and the initial node representation of each node.

[0046] In some embodiments, the disturbance unit is further configured to:

[0047] Based on the noise data and the degree of the node, the noise disturbance parameters corresponding to each node are calculated.

[0048] Summing the node representation and the noise perturbation parameter yields the node representation for each node.

[0049] In some embodiments, the federated model includes a first model deployed on the participating party and a second model deployed on the service provider, and the learning module further includes:

[0050] A sending unit is configured to send the node representation to the service provider, so that the service provider can calculate the loss function and backpropagation gradient of the second model based on the node representation and label data sent by all the participants, and send the backpropagation gradient to all the participants and adjust the parameters of the second model.

[0051] The parameter update unit is used to update the parameters of the first model according to the backpropagation gradient sent by the service provider until the loss function of the second model converges, thereby obtaining the target federated model.

[0052] In some embodiments, the parameter update unit is further configured to:

[0053] Calculate the backpropagation gradient of the first model based on the backpropagation gradient sent by the service provider;

[0054] Based on the backpropagation gradient of the first model, the parameters of the first model are updated until the loss function of the second model converges, thus obtaining the target federated model.

[0055] This application also provides an electronic device, including a memory storing multiple instructions; the processor loads instructions from the memory to execute steps in any of the federated learning methods provided in this application.

[0056] This application also provides a computer-readable storage medium storing a plurality of instructions adapted for loading by a processor to execute steps in any of the federated learning methods provided in this application.

[0057] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in any of the federated learning methods provided in this application.

[0058] This application embodiment can identify edges to be hidden and edges to be added from graph data, and perturb the held graph data based on these edges to obtain target graph data. Node representations are then calculated based on the target graph data. Finally, the parameters of the federated model are updated using the backpropagation gradient calculated by the service provider based on these node representations and its own label data, resulting in the target federated model. Perturbing the graph data based on the edges to be added and hidden achieves privacy protection for the graph data, ensuring that the calculated node representations do not risk revealing the graph structure and features, thereby improving data security in federated learning. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1a This is a schematic diagram of a scenario for the federated learning method provided in an embodiment of this application;

[0061] Figure 1b This is a schematic diagram of the graph data held by each participating party as provided in the embodiments of this application;

[0062] Figure 1c This is a flowchart illustrating the federated learning method provided in an embodiment of this application;

[0063] Figure 1d This is a schematic diagram of the vertical federated graph neural network learning architecture provided in an embodiment of this application;

[0064] Figure 1eThis is a schematic diagram of federated graph neural network learning based on differential privacy and negative sampling obfuscation mechanism provided in the embodiments of this application;

[0065] Figure 2a This is a flowchart illustrating a federated learning method provided in another embodiment of this application;

[0066] Figure 2b This is a schematic diagram of multi-party interactions in federated learning provided in an embodiment of this application;

[0067] Figure 3 This is a schematic diagram of the structure of the federated learning device provided in the embodiments of this application;

[0068] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0069] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0070] This application provides a federated learning method, apparatus, electronic device, storage medium, and program product.

[0071] Specifically, the federated learning device can be integrated into an electronic device, such as a terminal or server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0072] In some embodiments, the federated learning device may also be integrated into multiple electronic devices, such as multiple servers, with the federated learning method of this application being implemented by multiple servers.

[0073] In some embodiments, the server may also be implemented as a terminal.

[0074] For example, refer to Figure 1aThe diagram illustrates an application scenario of the federated learning method provided in this embodiment.

[0075] like Figure 1a As shown, both participant 101 and service provider 102 are located in a wireless network or a wired network, and participant 101 can establish a communication connection with service provider 102.

[0076] Federated learning involves multiple parties collaborating to train one or more machine learning models. Based on the distribution characteristics of the training data, federated learning can be divided into horizontal federated learning (HFL), vertical federated learning (VFL), and federated transfer learning (FTL).

[0077] Among them, HFL is suitable for sample sets that share the same feature space but have different sample spaces. VFL is suitable for sample sets that share the same sample space but have different feature spaces. FTL is suitable for sample sets that differ not only in sample space but also in feature space.

[0078] The federated learning method provided in this application is applicable to vertical federated learning, where each participant 101 has the same set of nodes (sample space), and each participant 101 holds some features and edges between some nodes. The service provider 102 holds the labels of the nodes, and may also hold some node features. Different participants 101, and participants 101 and service provider 102, do not exchange graph data or label data. By uniting multiple participants with the same sample space and different feature spaces, the number of features involved in training is increased, thereby improving the training effect of the federated model.

[0079] For example, given a training set with N nodes, meaning each participant holds N nodes in their graph data, this training set can be represented as follows: ,in It is the set of all nodes. Each participant shares the same node but holds different types of node characteristics and edges. Indicates the first The node characteristics held by each participant and the edges between nodes. It is the label of all nodes, and holding Y means that one is both a participant and a service provider.

[0080] In this embodiment of the application, there can be multiple participants 101, and each participant can hold graph data, for example, see [reference needed]. Figure 1bThis illustrates a diagram of the graph data held by each participant. Figure 1b In this example, assume there are three participants: participant A, participant B, and participant C. Each participant holds the same set of nodes in the graph data, namely v1, v2, v3, and v4. Each participant also holds some features and some edges between nodes. For example, Figure 1b In this scenario, participant A holds features f1 and f2, participant B holds features f3, f4 and f5, and participant C holds features f6 and f7.

[0081] Each participant 101 can obtain graph data; from the graph data, determine edges to be hidden and edges to be added, wherein the edges to be added are at least one of the edges between specified node pairs, the specified node pairs are two nodes that meet preset conditions, and the edges to be hidden are at least one of the edges; based on the edges to be hidden and the edges to be added, the graph data is perturbed to obtain target graph data; and a federated model is used to perform aggregation calculations on each node in the target graph data to obtain the node representation of each node in the target graph data.

[0082] Then, participant 101 sends its node representations to service provider 102. Service provider 102 fuses the node representations sent by all participants 101 and inputs them into the federated model to obtain the output of the federated model. Combining this with the label data held by service provider 102, it calculates the loss function and backpropagation gradient, and adjusts the parameters of the federated model. Then, service provider 102 sends the calculated backpropagation gradient to each participant 101, allowing each participant 101 to adjust the parameters of the federated model based on the backpropagation gradient to obtain the target federated model.

[0083] The following sections will provide detailed explanations.

[0084] Artificial intelligence (AI) is a technology that uses digital computers to simulate human perception of the environment, acquire knowledge, and use that knowledge. This technology can enable machines to possess functions similar to human perception, reasoning, and decision-making. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0085] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instruction-based learning.

[0086] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, smart customer service, vehicle networking, and intelligent transportation. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0087] Cloud technology refers to a managed technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0088] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0089] Artificial intelligence cloud services are generally also known as AIaaS (AI as a Service). This is currently a mainstream service model for artificial intelligence platforms. Specifically, AIaaS platforms break down several common AI services and provide them as independent or packaged services in the cloud. This service model is similar to opening an AI-themed marketplace: all developers can access and use one or more AI services provided by the platform through API interfaces. Some experienced developers can also use the AI ​​frameworks and AI infrastructure provided by the platform to deploy and maintain their own dedicated cloud AI services.

[0090] The method provided in this application can provide federated learning services to the public and private clouds in the form of a federated learning platform. Specifically, it can provide a model training module as a federated learning task, improve the availability of the federated learning system platform, and increase the functional modules of the federated learning system. That is, it can add the privacy protection scheme based on differential privacy and negative sampling obfuscation edges of this application, thereby improving the privacy and security characteristics of the federated graph neural network learning model.

[0091] In this embodiment, a federated learning method is provided, which can be applied to participating parties, such as... Figure 1c As shown, the specific process of this federated learning method can be as follows:

[0092] S110. Obtain graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes.

[0093] A graph is a non-linear data structure; graph data refers to data with a graph structure. A graph typically consists of a finite, non-empty set of nodes and a set of edges between nodes. In other words, graph data can include nodes and edges, where edges represent the relationships between nodes. The number of nodes and edges can be multiple.

[0094] The graph data obtained by the participants is graph data held by themselves, such as graph data obtained by the participants based on their own business. During the federated learning process, the participants do not interact with other parties participating in the federated learning process.

[0095] S120. Determine the edges to be hidden and the edges to be added from the graph data.

[0096] Graph data includes nodes and edges, with edges representing the relationships between nodes. Since the federated model learns the characteristics of the edges in the graph data when calculating the node representations, to fully protect the privacy of the graph data and prevent the subsequently calculated node representations from revealing the edge characteristics, the edges in the graph data can be perturbed. For example, some existing edges can be hidden, and some non-existent edges can be added. The edges that need to be hidden are called "hidden edges," and the edges that need to be added to the graph data are called "added edges."

[0097] In this context, the edges to be hidden are at least one of the existing edges in the graph data, and can be randomly selected from the existing edges based on a certain probability. The edges to be added are edges that did not originally exist in the graph data. These edges can be at least one of the edges between specified node pairs, which are two nodes that meet preset conditions. That is, the edges to be added can be edges introduced between specified node pairs based on a certain probability. A specified node pair refers to two nodes in the graph data that are not connected by any edge, meaning they are two nodes with no relation to each other in the graph data.

[0098] In some implementations, when determining the edges to be hidden and the edges to be added, all node pairs can be obtained from the graph data, where each node pair includes any two nodes; based on the graph data, node pairs among all node pairs that do not have edges are identified as designated node pairs; the existence probability of the edges between the designated node pairs is calculated, and the added edges are determined based on the existence probability; and the edges to be hidden are determined from the edges in the graph data based on a preset differential privacy budget.

[0099] A node pair refers to any two nodes in graph data. For example, if a graph contains three nodes v1, v2, and v3, then v1v2, v1v3, and v2v3 are all node pairs in the graph data. After obtaining all node pairs in the graph data, the node pairs without edges can be identified as specified node pairs. In other words, a specified node pair is a pair of two nodes in the graph data that are not connected by an edge. As in the previous example, in the graph data, only the node pair v1v2 has an edge connecting them; that is, nodes v1 and v2 are connected by an edge. Therefore, the specified node pairs are node pairs v1v3 and v2v3.

[0100] Then, the existence probability of an edge between a specified pair of nodes is calculated, and the edges to be added are determined based on this probability. The existence probability of an edge between a specified pair of nodes refers to the probability that an edge exists between the two nodes in the specified pair.

[0101] As one implementation method, when calculating the existence probability of an edge between a specified pair of nodes and determining the edges to be added based on the existence probability, the specified pair of nodes can be convolved to obtain the node convolution features corresponding to the two nodes in the specified pair of nodes; based on the node convolution features, the similarity between the two nodes in the specified pair of nodes can be calculated to obtain the similarity of each specified pair of nodes; the similarity of each specified pair of nodes can be mapped to a specified interval to obtain the existence probability of an edge between the specified pair of nodes; based on the existence probability of an edge between the specified pair of nodes, the edge between the specified pair of nodes can be determined as the edge to be added.

[0102] In some implementations, participants can pre-train a link predictor on locally held graph data to estimate the probability of existence of each edge in the graph data. Specifically, a specified pair of nodes can be input into the link predictor so that the link predictor outputs the probability of existence of the edge between the specified pair of nodes.

[0103] As one implementation method, the specific processing procedure of the link predictor can be as follows: perform convolution processing on the input specified node pair to obtain the node convolution features corresponding to the two nodes in the specified node pair; then calculate the similarity between the two nodes in the specified node pair based on the node convolution features to obtain the similarity of each specified node pair; then map the similarity of each specified node pair to a specified interval to obtain the probability of the existence of the edge between the specified node pairs.

[0104] The specified interval can be between 0 and 1. When mapping the similarity of a specified node to the specified interval, the sigmoid function can be used. The sigmoid function can map the input number to a number between 0 and 1. Thus, the similarity of a specified pair of nodes can be mapped to a number between 0 and 1 through the sigmoid function to obtain the probability of the existence of an edge between the specified pair of nodes.

[0105] In other words, the participants can pre-train a link predictor. The link predictor can first use a graph convolutional model to calculate the node convolutional features of each node. For any pair of nodes u and v, the following can be used: and To represent the node convolution features obtained from the graph convolution model, using Come as an edge The probability estimate of whether it actually exists, i.e., the probability of the edge's existence, where, It refers to the sigmoid function, which can map an input number to a number between 0 and 1.

[0106] After obtaining the existence probability of an edge between specified node pairs, the edges between these pairs can be identified as potential new edges based on this probability. For example, if the existence probability of an edge between specified node pairs v1 and v3 is 0.6, and the existence probability of an edge between specified node pairs v2 and v3 is 0.1, then the edge between specified node pairs v1 and v3 will be identified as a potential new edge with a probability of 0.6, and the edge between specified node pairs v2 and v3 will be identified as a potential new edge with a probability of 0.1.

[0107] It should be noted that determining the edges between specified node pairs v1 and v3 and the edges between specified node pairs v2 and v3 as edges to be added are two independent processes. That is, the edges between specified node pairs v2 and v3 and the edges between specified node pairs v1 and v3 may be determined as edges to be added at the same time.

[0108] In some implementations, edges to be hidden can be determined from the edges of the graph data based on a preset differential privacy budget. For example, the retention probability of an edge can be calculated based on a preset differential privacy budget; and the edges to be hidden can be determined from the edges of the graph data based on the retention probability of the edge.

[0109] Differential privacy is a privacy protection technique that aims to maximize the accuracy of data queries when querying statistical databases, while minimizing the chance of identifying records. A related concept is adjacent datasets. Suppose we are given two graph datasets G and G... If two graph datasets have exactly one different edge, then they can be called adjacent datasets.

[0110] If, for a random algorithm M, it is difficult to distinguish which graph dataset the outputs were obtained from when applied to two adjacent graph datasets (e.g., training two machine learning models), then this random algorithm M is considered to satisfy the edge difference privacy requirement. Mathematically, edge difference privacy can be defined as follows:

[0111] ;

[0112] Here, 'o' represents any output, and 'ε' represents the preset differential privacy budget. This formula means that for any neighboring graph datasets, the probability of training a model with a specific output is roughly the same. Therefore, observers can hardly detect subtle changes in the dataset by observing the output parameters, and thus cannot deduce specific training data from the output parameters. This is how data privacy is protected.

[0113] The preset differential privacy budget can be a value pre-set based on experience, usually a number between 0 and 10. After obtaining the preset differential privacy budget, the edge retention probability can be calculated.

[0114] The edge retention probability can be based on a predefined differential privacy budget representation, for example, it can be represented as: In some implementations, when determining the edges to be hidden based on the edge retention probability, the edges to be retained can be directly extracted from the edges of each node in the graph data according to the edge retention probability, and the other edges in the graph data other than the edges to be retained can be determined as the edges to be hidden.

[0115] In some implementations, after calculating the retention probability of an edge, the hiding probability of the edge can be calculated; that is, subtracting the retention probability of the edge from 1 yields the hiding probability of the edge. Then according to The hidden probability is obtained by directly extracting the edges to be hidden from the edges of the node.

[0116] S130. Based on the edges to be hidden and the edges to be added, the graph data is perturbed to obtain the target graph data.

[0117] Based on the identified edges to be hidden and added, the graph data can be perturbed to obtain the target graph data. In other words, the target graph data refers to the graph data obtained after perturbing the edges of the original graph data.

[0118] In some implementations, since the edge to be hidden is an edge in the graph data that needs to be hidden, and the edge to be added is an edge that needs to be added to the graph data, when perturbing the graph data, the edge to be hidden can be deleted from the graph data, and the edge to be added can be added to the graph data to obtain the target graph data.

[0119] In some implementations, the target graph data can be regenerated based on the edges to be hidden, the edges to be added, and the graph data. For example, in the graph data, all edges except the edges to be hidden can be identified as edges to be retained; all nodes in the graph data, the edges to be retained, and the edges to be added can be identified as the target graph data.

[0120] Edges to be hidden are those edges in the graph data that need to be hidden. Conversely, all other edges in the graph data besides those to be hidden are edges that need to be retained, and can be denoted as edges to be retained. Edges to be added are edges that need to be added to the graph data; that is, the generated target graph data contains edges to be added. Therefore, the target graph data can be generated based on all nodes, edges to be retained, and edges to be added in the graph data. Participants can protect the edges in the graph data by adding edges to be added and deleting edges to be hidden.

[0121] S140. Use a federated model to perform aggregation calculations on each node in the target graph data to obtain the node representation of each node in the target graph data.

[0122] After obtaining the target graph data, a federated model can be used to aggregate and compute each node in the target graph data to obtain the node representation of each node. It should be noted that the difference between the target graph data and the standard graph data lies in the change of edges; the number and characteristics of the nodes remain unchanged.

[0123] The federated model is composed of models deployed on multiple parties. It is assumed that the first model in the federated model is deployed on the participating parties. The first model may include at least one network layer.

[0124] When using a federated model to perform aggregation calculations on nodes, a first model deployed on the participating parties can be used to perform aggregation calculations on nodes in the target graph data. This first model can be a graph neural network model. When performing aggregation calculations on nodes, an initial node representation for each node can be calculated using a neighbor aggregation method. Then, noise perturbation processing is applied to the initial node representation to obtain the node representation corresponding to each node. For example, this can involve identifying the neighboring nodes of each node in the target graph data; determining the node features of the neighboring nodes as neighborhood features from the node features output by the previous network layer; averaging the neighborhood features of each node to obtain the target neighborhood features of each node; fusing the target neighborhood features and the node features output by the previous network layer based on the density parameters of the current network layer to obtain the node features output by the current network layer; determining the output of the last network layer in the first model as the initial node representation for each node; and applying noise perturbation processing to the initial node representation to obtain the node representation of each node.

[0125] Neighbor aggregation refers to each node calculating a graph based on its own neighboring nodes. The first model includes at least one network layer. Assuming the first model has L network layers, any node has corresponding node features in each network layer.

[0126] In this process, the node features output from layer 0 are the same as the input node features. When performing neighbor aggregation in each subsequent network layer, the neighboring nodes of each node in the target graph data can be determined first. These neighboring nodes are nodes directly connected to the current node. Then, the node features of the neighboring nodes are selected from the node features output from the previous network layer and used as the neighborhood features. Finally, the neighborhood features of each node are averaged to obtain the target neighborhood features for each node.

[0127] Of course, in some implementations, neighborhood features can refer to the node features output by the previous network layer for some of the neighboring nodes. For example, the neighboring nodes of node v1 are v2, v3, and v4, but when determining the neighborhood features, some nodes can be randomly selected from the neighboring nodes, for example, nodes v2 and v4. In this case, the neighborhood features are the node features of nodes v2 and v4 output by the previous network layer.

[0128] Understandably, the previous network layer outputs the node features of each node, and then fuses these node features with the node's target neighborhood features. Specifically, during fusion, the density parameters of the current network layer can be obtained. These density parameters are then used to weight the concatenated node features and target neighborhood features from the previous network layer. Finally, an activation function is used to calculate the output of the current network layer.

[0129] The node features are calculated in the manner described above for each network layer, and then the output of the last network layer, i.e. the output of the Lth network layer, is used as the initial node representation for each node.

[0130] Specifically, the above aggregation calculation process can be expressed as the following formula:

[0131]

[0132]

[0133] .

[0134] Where L refers to the total number of network layers in the first model. Indicates the first The nodes held by each participant The features, that is, the features of node u input to the first model; The features of node u in the 0th layer of the network; For the first Nodes in a layered network Features; For the first Dense parameters of layered graph neural networks, These are common activation functions in neural network learning. Mean represents the operation of taking the average of the selected node features, and Concat represents the operation of concatenating the features of two nodes. Represents nodes in the target graph data The adjacent nodes.

[0135] To further improve the privacy and availability of local graph data, after obtaining the initial node representation, noise perturbation processing can be applied to the initial node representation to obtain the node representation of each node.

[0136] In some implementations, in order to balance the privacy and availability of graph data, noise perturbation processing is applied to the initial node representation. This can be done by injecting noise into the initial node representation of each node based on the importance of the node. That is, the amount of noise injected into the initial node representation of a node varies depending on its importance, so as to achieve adaptive noise adjustment.

[0137] One implementation method is to use the degree of a node to characterize its importance, and to calibrate the magnitude of the injected noise based on the node's degree. For example, this could involve obtaining the degree of each node in the graph data, where the degree is the number of edges of that node; sampling the data in a preset noise distribution to obtain noise data; and calculating the node representation of each node based on the noise data, the degree of the nodes, and the initial node representation of each node.

[0138] The degree of a node refers to the number of edges connecting the node in graph data. For example, if node A has edges connecting to both nodes B and C, while node B only has an edge connecting to node A, then the degree of node A is 2, and the degree of node B is 1. It should be noted that the higher the degree of a node, the more important it is, indicating that the node has participated in the calculation of the initial node representations of many other nodes in the aforementioned aggregation calculation.

[0139] The preset noise distribution can be a Gaussian distribution or a Laplace distribution, etc. Sampling within the preset noise distribution can yield the corresponding noise data.

[0140] If we denote the variance as s, and the probability density function of the Gaussian distribution is:

[0141] ;

[0142] If we denote the variance as s, the probability density function of the Laplace distribution is:

[0143] ;

[0144] When acquiring noisy data, random sampling can be performed within a Gaussian or Laplace distribution, and the sampled data can be used as noise data. Introducing noise data into the initial node features of nodes can effectively protect the privacy of graph data. To maximize data usability, the amount of noise data introduced into the initial node representation of each node can be adjusted based on the node's degree to obtain the node representation for each node.

[0145] In some implementations, since nodes with higher degrees participate in the calculation of the initial node representations of many other nodes during node aggregation computation, to maximize data availability while protecting graph data privacy, relatively small noise can be used to perturb the initial node representation of nodes with higher degrees. Specifically, when calculating the node representation of each node based on the noise data, the degree of the node, and the initial node representation of each node, the noise perturbation parameter corresponding to each node can be calculated based on the noise data and the degree of the node; the node representation and the noise perturbation parameter are then summed to obtain the node representation corresponding to each node.

[0146] The noise perturbation parameter refers to the noise data introduced into the initial node representation of each node. Since relatively small noise is used to perturb the initial node representation of nodes with larger degrees, the degree of a node and the noise perturbation parameter can be inversely proportional; that is, the larger the degree of a node, the smaller the noise perturbation parameter, and vice versa. Finally, summing the noise perturbation parameter and the initial node representation yields the node representation for each node.

[0147] In some implementations, node representations can be calculated using the following formula:

[0148] ;

[0149] in, This represents the initial node representation of node u calculated by the j-th participant; Represents a node In the The degree in the graph data of each participant; To represent noise sampled from a Gaussian or Laplace distribution. It is a smoothing parameter between 0 and 1. This represents the node representation of node u calculated by the j-th participant.

[0150] S150. Update the parameters of the federated model according to the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

[0151] After obtaining the node representation for each node, the participating parties can send these representations to the server for collaborative training of the federated model. Because the participants perturb the graph data they hold by introducing differential privacy techniques—including the addition of new edges and edge protection—when calculating the initial node representation, the calculated initial node representation does not reveal edge information. Furthermore, noise perturbation is applied to the calculated initial node representation based on the node's degree, further protecting the graph data held by the participants. Therefore, after sending the node representation to the server, the server cannot deduce the graph data held by the participants, effectively protecting privacy.

[0152] Since the node representations calculated by the participants can effectively protect the privacy of graph data and there is no risk of leaking graph data, compared with the method of protecting the privacy of graph data through secret sharing and homomorphic encryption, the method provided in this application embodiment has lower computational complexity, can handle application scenarios with latency requirements, and does not have the problem of ciphertext expansion. As a result, the communication cost is lower, the requirements for network communication bandwidth and stability are not high, and it is applicable to a variety of scenarios.

[0153] It should be noted that the federated model may include a first model deployed on the participating parties and a second model deployed on the service provider. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing results of the node representations by the federated model. Therefore, in some implementations, updating the parameters of the federated model based on the backpropagation gradient sent by the service provider to obtain the target federated model can involve sending the node representations to the service provider so that the service provider can calculate the loss function and backpropagation gradient of the second model based on the node representations and label data sent by all the participating parties, and send the backpropagation gradient to all the participating parties and adjust the parameters of the second model; updating the parameters of the first model based on the backpropagation gradient sent by the service provider until the loss function of the second model converges, thus obtaining the target federated model.

[0154] It should be noted that there can be multiple participants, and each participant calculates the node representation locally according to the steps S110 to S150 described above. Then, the calculated node representations are all sent to the service provider.

[0155] After receiving node representations from each participant, the service provider calculates the loss function and backpropagation gradient of the second model based on the node representations and its own label data. In some implementations, the service provider can fuse the node representations sent by each participant to obtain a fused node representation.

[0156] In some implementations, when the service provider fuses node representations, it may concatenate the node representations sent by each participant. For example, the node representation of node u sent by participant A is: The node representation of node u sent by participant B is as follows: By concatenating them, the node representation of the merged node u can be obtained as [ ].

[0157] In some implementations, when the service provider fuses node representations, it may average the node representations sent by each participant. As illustrated in the previous example, the node representation of the fused node u obtained after averaging is Mean( ).

[0158] The service provider then inputs the fused node representations into the second model, obtains the output of the second model, and calculates the loss function based on the output and its own label data. The loss function can be set according to the specific training task; for example, the cross-entropy loss function can be used in classification tasks.

[0159] Then, the service provider can perform backpropagation according to the chain rule to calculate the backpropagation gradient in order to update the model parameters of the federated model.

[0160] In some implementations, the federated model consists of a first model deployed on each participant and a second model on the service provider. Thus, when the model parameters of the federated model are updated, the service provider needs to update the model parameters of the second model, and each participant needs to update the model parameters of the first model deployed.

[0161] When calculating the backpropagation gradient, the service provider can perform backpropagation according to the chain rule to calculate the derivative of the loss function with respect to the fused node representations, thereby obtaining the backpropagation gradient of the second model, and using this gradient to update the model parameters of the second model. There are various optimizers available for updating the model parameters of the second model, such as stochastic gradient descent with momentum and the Adam optimizer.

[0162] The service provider can also calculate the derivative of the loss function with respect to the node representations sent by the participants, obtain the gradient of the corresponding first model's output, and send it back to the corresponding participant. For example, if the loss function is L, For the node representation sent by the j-th participant, the gradient of the output of the first model can be calculated as follows: That is, it means that the loss function L is applied to... The derivative of .

[0163] Thus, the participants can update the parameters of the first model deployed on them based on the backpropagation gradient sent by the service provider, until the loss function of the second model converges, thus obtaining the target federated model.

[0164] In some implementations, the parameters of the first model deployed on the participants are updated based on the backpropagation gradient sent by the service provider until the loss function of the second model converges. The target federated model can be obtained by calculating the backpropagation gradient of the first model based on the backpropagation gradient sent by the service provider; and updating the parameters of the first model based on the backpropagation gradient of the first model until the loss function of the second model converges.

[0165] After receiving the backpropagation gradient from the service provider, each participant can perform backpropagation based on this gradient to calculate the backpropagation gradient of the first model. The calculated backpropagation gradient of the first model is then used to update its parameters until the loss function calculated by the service provider converges. The first models deployed on all participants and the service provider's second model are then used as the target federated model.

[0166] The above steps are performed once for each graph data in each iteration round, and are repeated n times in total, until the federated model converges on all graph data.

[0167] In some implementations, in the scenario of vertical federated learning, the service provider can hold both labeled data and graph data. If the service provider holds graph data, the first model and the second model can be run simultaneously on the service provider. The service provider can perform the same processing on its graph data as other participants and execute the service provider's processing steps.

[0168] In some implementations, in segmentation learning scenarios, where the service provider only holds the labeled data, the second model is run only on the service provider.

[0169] To explain the federal learning process more clearly, please refer to [link / reference]. Figure 1d This diagram illustrates a vertical federated graph neural network learning architecture. The first models deployed on each participant are denoted as F1, F2, ..., F... M The second model deployed by the service provider is denoted as H, and the graph data held by each participant is X. (1) X (2) ...X (M) Each participant inputs its held graph data into its respective deployed first model, and obtains the output Z of each first model. (1) Z (2) ...Z (M) Then, each participant sends the output of the first model to the service provider, which then sends Z...(1) Z (2) ...Z (M) The input is fed into the first model H, and the loss function is calculated based on the label data it holds. Then, the gradient is calculated and fed back to each participant so that each participant can use the gradient (▽Z) to calculate the loss function. (1) 、▽Z (2 ……▽Z (M) The parameters of each first model are adjusted.

[0170] In this embodiment of the application, to enhance the security of the graph data held by each participating party, federated graph neural network learning is performed based on differential privacy and negative sampling obfuscation mechanisms, as detailed in [reference needed]. Figure 1e This diagram illustrates federated graph neural network learning based on differential privacy and negative sampling obfuscation mechanisms.

[0171] in, Figure 1e The document primarily describes the execution process of the participating parties.

[0172] 1. Perturb the edges and then perform neighbor aggregation to obtain the initial node representation.

[0173] The participants determine the edges to be hidden based on edge differential privacy and the edges to be added based on a pre-trained link predictor, and then perturb the edges in the graph data accordingly. This involves adding the edges to be added to the graph data and removing the edges to be hidden from the graph data. Then, the first model deployed on the participants is used to perform neighbor aggregation on the perturbed graph data to obtain the initial node representation for each node.

[0174] 2. Apply adaptive noise perturbation to the initial node representation.

[0175] After the participants obtain the initial node representations output by the first model, to avoid the initial node representations leaking graph data and to improve data usability, the amount of noise injected into the initial node representations can be adaptively adjusted based on the importance of the nodes. The importance of a node can be determined by its degree in the graph data. Since a higher degree indicates that the node has participated in the initial node representation calculations of many other nodes during neighbor aggregation, a higher degree indicates a higher importance, and therefore, less noise is injected into the initial node representation. After adaptively perturbing the initial node representations with noise, the final node representations can be obtained.

[0176] 3. All participants jointly train the model.

[0177] Each participant obtains node representations as described above. Each participant then sends its node representation (forward output) to the server. The server calculates the loss function and backpropagation gradient of the second model based on the node representations sent by all participants and its own label data, and adjusts the parameters of the second model accordingly. The server then sends the calculated gradients back to each participant, who adjusts the parameters of the first model based on the gradients until the loss function calculated by the server converges. The first models from all participants and the second model from the server are then used as the jointly trained model, i.e., the target federated model.

[0178] The federated learning scheme provided in this application can be applied to various vertical federated learning scenarios. For example, taking vertical federated graph learning as an example, the participants use the graph data they hold to train the first model in the federated model. During the calculation of node representations, based on the privacy of the graph data including the edges to be added and the edges to be hidden, the node representations do not have the risk of leaking the graph structure and features of the graph data, making the graph data held by each party in vertical federated graph learning more secure.

[0179] The method provided in this application can identify edges to be hidden and edges to be added from graph data, perturb the held graph data based on these edges to obtain target graph data, calculate node representations based on the target graph data, and update the parameters of the federated model according to the backpropagation gradient calculated by the service provider based on the node representations and its own label data, thus obtaining the target federated model. Perturbing the graph data based on the edges to be added and hidden achieves privacy protection for the graph data, ensuring that the calculated node representations do not risk leaking the graph structure and features of the graph data. This effectively counters attacks attempting to steal local features and edges, improving data security in federated learning. Furthermore, adaptive noise perturbation of the initial node representations calculated by the first model not only further protects graph data privacy but also improves data availability, thereby enhancing the training effect of the federated model.

[0180] The method described in the above embodiments will be further described in detail below.

[0181] In this embodiment, the method of this application embodiment will be described in detail using vertical federated graph neural network learning as an example. In vertical federated graph neural network learning, there may be a service provider and multiple participants. Each participant is a party holding graph data. The graph data held by multiple participants contains the same nodes, but the edges and features of the nodes differ. The service provider is the party holding the label data for training the model. The participants and the server can refer to the same party. Each participant has a first model deployed on it, and the service provider has a second model deployed on it. All the first and second models constitute a federated model.

[0182] like Figure 2a As shown, the specific process of a federated learning method is as follows:

[0183] S210. The participants determine the edges to be hidden and the edges to be added from the graph data.

[0184] The participants determine which edges to be hidden and which to be added from the graph data they possess. When determining edges to be hidden, they can do so based on a pre-defined differential privacy budget, identifying edges from the graph data. When determining edges to be added, they can pre-train a link predictor based on the graph data they possess. This link predictor can estimate the probability of each edge's existence. By inputting any two nodes in the graph data that do not have an edge into the link predictor, the probability of the edge between the two nodes can be obtained. Then, based on this probability, the edges to be added are determined.

[0185] S220. The participants perturb the graph data based on the edges to be hidden and the edges to be added, and obtain the target graph data.

[0186] S230. The participants use the first model to calculate the initial node representation of each node in the target graph data.

[0187] By deleting edges to be hidden from the graph data and adding edges to be added to the graph data, the participants can perform perturbation processing on the graph data and obtain the target graph data.

[0188] Participants utilize a first model to calculate the initial node representation of each node in the target graph data. This first model can be a graph neural network model with at least one network layer. Neighbor aggregation is performed at each network layer to calculate the initial node representation for each node. Since the target graph data, compared to the original graph data, includes edges to be added and edges to be hidden, the calculated initial node representation does not reveal the edges in the graph data, effectively protecting the privacy of the graph data.

[0189] S240. Based on the degree of each node in the graph data, the participants perform noise perturbation on the initial node representation of each node to obtain the node representation of each node, and send the node representation to the service provider.

[0190] To further protect the privacy of graph data and improve training performance, the magnitude of the introduced noise can be adaptively calibrated based on the degree of each node in the graph data, and the initial node representation can be perturbed based on the noise to obtain the final node representation. The participants then send the node representation to the service provider.

[0191] S250. The service provider performs fusion processing on the node representations sent by each participant to obtain the fused node representation.

[0192] S260. The service provider inputs the fused node representation into the second model, calculates the loss function and backpropagation gradient by combining the label data, adjusts the model parameters of the second model, and sends the backpropagation gradient to the participants.

[0193] Since there are multiple participants, the service provider can fuse all the received node representations, for example, by splicing them together or calculating the mean, to obtain the fused node representations. The service provider then inputs the fused node representations into the second model, calculates the loss function and backpropagation gradient using the label data and the output of the second model, and then adjusts the model parameters of the second model. To ensure that the participants can also adjust the corresponding model parameters of the first model, the service provider sends the backpropagation gradient to each participant. The specific calculation method for the backpropagation degree can be found in the corresponding section of the aforementioned embodiment, and will not be repeated here.

[0194] S270. The participants update the model parameters of the first model according to the backpropagation gradient until the loss function converges, and then use all the first and second models as the target federated model.

[0195] The participating parties calculate the backpropagation gradient of the first model based on the received backpropagation gradient, and update the model parameters of the first model based on the calculated backpropagation gradient. Then, the process of calculating node representations and sending them to the service provider is repeated until the loss function calculated by the service provider converges, at which point the model training can be considered complete. At this point, the first models of all participating parties and the second model of the service provider are taken as the target federated model that has been trained.

[0196] For a clearer understanding of the aforementioned federal learning process, please refer to [link / reference]. Figure 2b This illustrates a diagram of multi-party interactions in federated learning. Figure 2b The following example illustrates the concept of two participating parties and one service provider.

[0197] Participant A holds graph data X (A) Participant B holds graph data X (B) Participant A and Participant B perturb their respective graph data based on edge differential privacy and negative sampling obfuscation edges, which is the aforementioned steps S210 to S220; and learn based on their own deployed first model to obtain initial node representations, which is the aforementioned step S230; then perform adaptive noise perturbation on the initial node representations, and send the final node representations to the service provider, which is the aforementioned step S240.

[0198] The service provider fuses the node representations from all participants, calculates the loss function and backpropagation gradient of the second model based on the label data and the fused node representations, and updates the model parameters of the second model. The backpropagation gradient is then sent to the participants so they can adjust the model parameters of the first model until the loss function calculated by the service provider converges.

[0199] The above method is applicable to various scenarios. For example, in a risk control and anti-fraud scenario, participant A holds basic user data for a user group, as well as data on abnormal and non-abnormal users within that user group. Participant B holds consumption data for the same user group. The tag data consists of data on abnormal and non-abnormal users within the user group; that is, the service provider is also participant A. Based on the federated learning method described above, an abnormal user prediction model can be trained jointly by multiple parties.

[0200] For example, in an intelligent recommendation scenario, participant A holds the income and expenditure data of a user group, while participant B holds the data on the items purchased by the same user group and the ad viewing data of each user in the same user group. The tag data is the ad viewing data of each user in the same user group, meaning that the service provider is also participant B. Based on the federated learning method mentioned above, multiple parties can jointly train an intelligent ad recommendation model.

[0201] It is understood that in the specific implementation of this application, user-related data is involved, such as basic user data and consumption data. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0202] As can be seen from the above, the federated learning method provided in this application embodiment does not transmit graph data between the participants or between the participants and the service provider. Each participant uses edge differential privacy and negative sampling to obfuscate edges to protect its own graph data, which can effectively protect the edges in the graph data and avoid leaking the features of the graph data. No encryption is required in the data interaction during joint training, which can reduce computational complexity and reduce network requirements. Furthermore, the initial node representation is subjected to adaptive noise perturbation based on the degree of the node, which can not only protect the privacy of the graph data, but also improve the availability of the data, increase the speed of model training, and effectively complete the learning task.

[0203] To better implement the above methods, this application also provides a federated learning device, which can be integrated into an electronic device, such as a terminal or server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein.

[0204] For example, in this embodiment, the method of this application embodiment will be described in detail by taking the federated learning device specifically integrated into a distributed system as an example.

[0205] For example, such as Figure 3 As shown, the federated learning device 300 may include an acquisition module 310, a determination module 320, a perturbation module 330, an aggregation module 340, and a learning module 350.

[0206] The acquisition module 310 is used to acquire graph data, which includes nodes and edges, and the edges are used to represent the association relationships between the nodes.

[0207] The determining module 320 is used to determine the edges to be hidden and the edges to be added from the graph data. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges.

[0208] The perturbation module 330 is used to perturb the graph data according to the edge to be hidden and the edge to be added, so as to obtain the target graph data;

[0209] The aggregation module 340 is used to perform aggregation calculations on each node in the target graph data using a federated model to obtain the node representation of each node in the target graph data.

[0210] The learning module 350 is used to update the parameters of the federated model according to the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

[0211] In some embodiments, the determining module 320 further includes:

[0212] A node pair determination unit is used to obtain all node pairs from the graph data, wherein each node pair includes any two nodes;

[0213] The designated node pair determination unit is used to determine, based on the graph data, node pairs among all node pairs that do not have edges as designated node pairs;

[0214] The edge-to-be-added determination unit is used to calculate the existence probability of the edge between the specified node pairs and determine the edge to be added based on the existence probability;

[0215] The edge to be hidden determination unit is used to determine the edges to be hidden from the edges of the graph data according to a preset differential privacy budget.

[0216] In some embodiments, the unit for determining the new edge is further configured to:

[0217] Perform convolution processing on the specified node pair to obtain the node convolution features corresponding to the two nodes in the specified node pair;

[0218] Based on the node convolution features, the similarity between two nodes in the specified node is calculated to obtain the similarity of each specified node pair;

[0219] The similarity of each specified node pair is mapped to a specified region to obtain the probability of the existence of an edge between the specified node pairs;

[0220] Based on the existence probability of the edge between the specified node pairs, the edge between the specified node pairs is determined as the edge to be added.

[0221] In some embodiments, the edge-to-be-hidden determination unit is further configured to:

[0222] The probability of edge retention is calculated based on a preset differential privacy budget.

[0223] Based on the retention probability of the edges, the edges to be hidden are determined from the edges of the graph data.

[0224] In some embodiments, the disturbance module 330 is further configured to:

[0225] The edge to be hidden is deleted from the graph data, and the edge to be added is added to the graph data to obtain the target graph data.

[0226] In some embodiments, the disturbance module 330 is further configured to:

[0227] In the graph data, all edges other than the edge to be hidden are identified as edges to be retained;

[0228] All nodes in the graph data, the edges to be retained, and the edges to be added are identified as the target graph data.

[0229] In some embodiments, the federation model includes a first model deployed on the participating party, the first model including at least one network layer, and the aggregation module 340 further includes:

[0230] The adjacency node determination unit is used to determine the adjacency nodes of each node in the target graph data;

[0231] The neighborhood feature determination unit is used to determine the node features of the adjacent nodes as neighborhood features from the node features of each node output by the previous network layer.

[0232] The mean unit is used to perform mean processing on the neighborhood features of each node to obtain the target neighborhood features of each node;

[0233] The fusion unit is used to fuse the target neighborhood features and the node features of each node output by the previous network layer based on the density parameters of the current network layer, so as to obtain the node features output by the current network layer.

[0234] The initial node representation determination unit is used to determine the output of the last network layer in the first model as the initial node representation of each node.

[0235] The perturbation unit is used to perform noise perturbation processing on the initial node representation to obtain the node representation of each node.

[0236] In some embodiments, the disturbance unit is further configured to:

[0237] Obtain the degree of each node in the graph data, where the degree of a node is the number of edges of that node;

[0238] Noise data is obtained by sampling from a preset noise distribution;

[0239] The node representation of each node is calculated based on the noise data, the degree of the node, and the initial node representation of each node.

[0240] In some embodiments, the disturbance unit is further configured to:

[0241] Based on the noise data and the degree of the node, the noise disturbance parameters corresponding to each node are calculated.

[0242] Summing the node representation and the noise perturbation parameter yields the node representation for each node.

[0243] In some embodiments, the federated model includes a first model deployed on the participating party and a second model deployed on the service provider, and the learning module 350 further includes:

[0244] A sending unit is configured to send the node representation to the service provider, so that the service provider can calculate the loss function and backpropagation gradient of the second model based on the node representation and label data sent by all the participants, and send the backpropagation gradient to all the participants and adjust the parameters of the second model.

[0245] The parameter update unit is used to update the parameters of the first model according to the backpropagation gradient sent by the service provider until the loss function of the second model converges, thereby obtaining the target federated model.

[0246] In some embodiments, the parameter update unit is further configured to:

[0247] Calculate the backpropagation gradient of the first model based on the backpropagation gradient sent by the service provider;

[0248] Based on the backpropagation gradient of the first model, the parameters of the first model are updated until the loss function of the second model converges, thus obtaining the target federated model.

[0249] In practice, each of the above modules or units can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above modules or units, please refer to the previous method embodiments, which will not be repeated here.

[0250] As shown above, the federated learning device in this embodiment can determine the edges to be hidden and the edges to be added from the graph data, and perturb the held graph data based on these edges to obtain target graph data. It then calculates node representations based on the target graph data, and updates the parameters of the federated model according to the backpropagation gradients calculated by the service provider using these node representations and its own label data, thus obtaining the target federated model. Perturbing the graph data based on the edges to be added and hidden achieves privacy protection for the graph data, ensuring that the calculated node representations do not risk leaking the graph structure and features of the graph data, thereby improving data security in federated learning.

[0251] This application also provides an electronic device, which can be a terminal, a server, or other similar device. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers, etc.

[0252] In some embodiments, the federated learning device may also be integrated into multiple electronic devices, such as multiple servers, with multiple servers implementing the federated learning method of this application.

[0253] In this embodiment, a server will be used as an example for detailed description. For example, ... Figure 4 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically:

[0254] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, and a communication module 405. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0255] The processor 401 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions of the SSS and processes data, thereby performing overall detection of the electronic device. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor 401.

[0256] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0257] The electronic device also includes a power supply 403 that supplies power to the various components. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0258] The electronic device may also include an input module 404, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0259] The electronic device may also include a communication module 405. In some embodiments, the communication module 405 may include a wireless module, through which the electronic device can perform short-range wireless transmission, thereby providing users with wireless broadband internet access. For example, the communication module 405 can be used to help users send and receive emails, browse web pages, and access streaming media.

[0260] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0261] Obtain graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes;

[0262] From the graph data, determine the edges to be hidden and the edges to be added. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges.

[0263] Based on the edges to be hidden and the edges to be added, the graph data is perturbed to obtain the target graph data;

[0264] The federated model is used to perform aggregation calculations on each node in the target graph data to obtain the node representation of each node in the target graph data;

[0265] The parameters of the federated model are updated based on the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

[0266] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0267] As can be seen from the above, the electronic device provided in this application can determine the edges to be hidden and the edges to be added from graph data, and perturb the held graph data based on the edges to be hidden and the edges to be added to obtain target graph data. It then calculates node representations based on the target graph data, and updates the parameters of the federated model according to the backpropagation gradient calculated by the service provider based on the node representations and its own held label data, thus obtaining the target federated model. Perturbing the graph data based on the edges to be added and the edges to be hidden achieves privacy protection for the graph data, so the calculated node representations do not risk leaking the graph structure and features of the graph data, thereby improving the security of data in federated learning.

[0268] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0269] To this end, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the federated learning methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0270] Obtain graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes;

[0271] From the graph data, determine the edges to be hidden and the edges to be added. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges.

[0272] Based on the edges to be hidden and the edges to be added, the graph data is perturbed to obtain the target graph data;

[0273] The federated model is used to perform aggregation calculations on each node in the target graph data to obtain the node representation of each node in the target graph data;

[0274] The parameters of the federated model are updated based on the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

[0275] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0276] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various alternative implementations of the federated learning aspect provided in the above embodiments.

[0277] Since the instructions stored in the storage medium can execute the steps of any of the federated learning methods provided in the embodiments of this application, the beneficial effects that any of the federated learning methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0278] The foregoing has provided a detailed description of a federated learning method, apparatus, electronic device, storage medium, and program product provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A federated learning method, characterized in that, Applied to participating parties, the method includes: Obtain graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes; From the graph data, determine the edges to be hidden and the edges to be added. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges. Based on the edges to be hidden and the edges to be added, the graph data is perturbed to obtain the target graph data; The federated model is used to perform aggregation calculations on each node in the target graph data to obtain the node representation of each node in the target graph data; The parameters of the federated model are updated based on the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

2. The method according to claim 1, characterized in that, The step of determining the edges to be hidden and the edges to be added from the graph data includes: From the graph data, obtain all node pairs, where each node pair includes any two nodes; Based on the graph data, the node pairs that do not have edges among all the node pairs are identified as specified node pairs; Calculate the probability of the existence of an edge between the specified pair of nodes, and determine the edges to be added based on the probability of existence; Based on a preset differential privacy budget, edges to be hidden are determined from the edges of the graph data.

3. The method according to claim 2, characterized in that, The step of calculating the existence probability of an edge between the specified pair of nodes and determining the edge to be added based on the existence probability includes: Perform convolution processing on the specified node pair to obtain the node convolution features corresponding to the two nodes in the specified node pair; Based on the node convolution features, the similarity between the two nodes in the specified node pair is calculated to obtain the similarity of each specified node pair. The similarity of each specified node pair is mapped to a specified region to obtain the probability of the existence of an edge between the specified node pairs; Based on the existence probability of the edge between the specified node pairs, the edge between the specified node pairs is determined as the edge to be added.

4. The method according to claim 2, characterized in that, The step of determining the edges to be hidden from the edges of the graph data according to a preset differential privacy budget includes: The probability of edge retention is calculated based on a preset differential privacy budget. Based on the retention probability of the edges, the edges to be hidden are determined from the edges of the graph data.

5. The method according to claim 1, characterized in that, The step of perturbing the graph data based on the edges to be hidden and the edges to be added to obtain the target graph data includes: The edge to be hidden is deleted from the graph data, and the edge to be added is added to the graph data to obtain the target graph data.

6. The method according to claim 1, characterized in that, The step of perturbing the graph data based on the edges to be hidden and the edges to be added to obtain the target graph data includes: In the graph data, all edges other than the edge to be hidden are identified as edges to be retained; All nodes in the graph data, the edges to be retained, and the edges to be added are identified as the target graph data.

7. The method according to claim 1, characterized in that, The federated model includes a first model deployed by the participants, the first model including at least one network layer, wherein the aggregation calculation of each node in the target graph data using the federated model to obtain a node representation of each node in the target graph data includes: Determine the adjacent nodes of each node in the target graph data; In the node features of each node output from the previous network layer, the node features of the adjacent nodes are determined as neighborhood features. The neighborhood features of each node are averaged to obtain the target neighborhood features of each node. Based on the density parameters of the current network layer, the target neighborhood features and the node features of each node output by the previous network layer are fused to obtain the node features output by the current network layer. The output of the last network layer in the first model is used to determine the initial node representation for each node. The initial node representation is subjected to noise perturbation processing to obtain the node representation of each node.

8. The method according to claim 7, characterized in that, The step of performing noise perturbation processing on the initial node representation to obtain the node representation of each node includes: Obtain the degree of each node in the graph data, where the degree of a node is the number of edges of that node; Noise data is obtained by sampling from a preset noise distribution; The node representation of each node is calculated based on the noise data, the degree of the node, and the initial node representation of each node.

9. The method according to claim 8, characterized in that, The step of calculating the node representation of each node based on the noise data, the degree of the node, and the initial node representation of each node includes: Based on the noise data and the degree of the node, the noise disturbance parameters corresponding to each node are calculated. Summing the initial node representation and the noise perturbation parameters yields the node representation for each node.

10. The method according to any one of claims 1-9, characterized in that, The federated model includes a first model deployed on the participating party and a second model deployed on the service provider. The step of updating the parameters of the federated model based on the backpropagation gradient sent by the service provider to obtain the target federated model includes: The node representation is sent to the service provider so that the service provider can calculate the loss function and backpropagation gradient of the second model based on the node representation and label data sent by all the participants, and send the backpropagation gradient to all the participants and adjust the parameters of the second model. Based on the backpropagation gradient sent by the service provider, the parameters of the first model are updated until the loss function of the second model converges, thus obtaining the target federated model.

11. The method according to claim 10, characterized in that, The step of updating the parameters of the first model based on the backpropagation gradient sent by the service provider until the loss function of the second model converges to obtain the target federated model includes: Calculate the backpropagation gradient of the first model based on the backpropagation gradient sent by the service provider; Based on the backpropagation gradient of the first model, the parameters of the first model are updated until the loss function of the second model converges, thus obtaining the target federated model.

12. A federated learning device, characterized in that, Applied to participating parties, the device includes: An acquisition module is used to acquire graph data, which includes nodes and edges, and the edges are used to represent the relationships between the nodes. The determination module is used to determine the edges to be hidden and the edges to be added from the graph data. The edges to be added are at least one of the edges between a specified pair of nodes. The specified pair of nodes are two nodes that meet a preset condition. The edges to be hidden are at least one of the edges. The perturbation module is used to perturb the graph data according to the edge to be hidden and the edge to be added, so as to obtain the target graph data; The aggregation module is used to perform aggregation calculations on each node in the target graph data using a federated model to obtain the node representation of each node in the target graph data. The learning module is used to update the parameters of the federated model based on the backpropagation gradient sent by the service provider to obtain the target federated model. The backpropagation gradient is calculated by the service provider based on the label data it holds and the processing result of the federated model on the node representation.

13. The apparatus according to claim 12, characterized in that, The determining module further includes: A node pair determination unit is used to obtain all node pairs from the graph data, wherein each node pair includes any two nodes; The designated node pair determination unit is used to determine, based on the graph data, node pairs among all node pairs that do not have edges as designated node pairs; The edge-to-be-added determination unit is used to calculate the existence probability of the edge between the specified node pairs and determine the edge to be added based on the existence probability; The edge to be hidden determination unit is used to determine the edges to be hidden from the edges of the graph data according to a preset differential privacy budget.

14. The apparatus according to claim 13, characterized in that, The unit for determining the new edge to be added is also used for: Perform convolution processing on the specified node pair to obtain the node convolution features corresponding to the two nodes in the specified node pair; Based on the node convolution features, the similarity between the two nodes in the specified node pair is calculated to obtain the similarity of each specified node pair. The similarity of each specified node pair is mapped to a specified region to obtain the probability of the existence of an edge between the specified node pairs; Based on the existence probability of the edge between the specified node pairs, the edge between the specified node pairs is determined as the edge to be added.

15. The apparatus according to claim 13, characterized in that, The unit for determining the edge to be hidden is also used for: The probability of edge retention is calculated based on a preset differential privacy budget. Based on the retention probability of the edges, the edges to be hidden are determined from the edges of the graph data.

16. The apparatus according to claim 12, characterized in that, The disturbance module is also used for: The edge to be hidden is deleted from the graph data, and the edge to be added is added to the graph data to obtain the target graph data.

17. The apparatus according to claim 12, characterized in that, The disturbance module is also used for: In the graph data, all edges other than the edge to be hidden are identified as edges to be retained; All nodes in the graph data, the edges to be retained, and the edges to be added are identified as the target graph data.

18. The apparatus according to claim 12, characterized in that, The federated model includes a first model deployed by the participants, the first model including at least one network layer, and the aggregation module further includes: The adjacency node determination unit is used to determine the adjacency nodes of each node in the target graph data; The neighborhood feature determination unit is used to determine the node features of the adjacent nodes as neighborhood features from the node features of each node output by the previous network layer. The mean unit is used to perform mean processing on the neighborhood features of each node to obtain the target neighborhood features of each node; The fusion unit is used to fuse the target neighborhood features and the node features of each node output by the previous network layer based on the density parameters of the current network layer, so as to obtain the node features output by the current network layer. The initial node representation determination unit is used to determine the output of the last network layer in the first model as the initial node representation of each node. The perturbation unit is used to perform noise perturbation processing on the initial node representation to obtain the node representation of each node.

19. The apparatus according to claim 18, characterized in that, The disturbance unit is also used for: Obtain the degree of each node in the graph data, where the degree of a node is the number of edges of that node; Noise data is obtained by sampling from a preset noise distribution; The node representation of each node is calculated based on the noise data, the degree of the node, and the initial node representation of each node.

20. The apparatus according to claim 19, characterized in that, The disturbance unit is also used for: Based on the noise data and the degree of the node, the noise disturbance parameters corresponding to each node are calculated. Summing the initial node representation and the noise perturbation parameters yields the node representation for each node.

21. The apparatus according to any one of claims 12-20, characterized in that, The federated model includes a first model deployed on the participating party and a second model deployed on the service provider. The learning module also includes: A sending unit is configured to send the node representation to the service provider, so that the service provider can calculate the loss function and backpropagation gradient of the second model based on the node representation and label data sent by all the participants, and send the backpropagation gradient to all the participants and adjust the parameters of the second model. The parameter update unit is used to update the parameters of the first model according to the backpropagation gradient sent by the service provider until the loss function of the second model converges, thereby obtaining the target federated model.

22. The apparatus according to claim 21, characterized in that, The parameter update unit is also used for: Calculate the backpropagation gradient of the first model based on the backpropagation gradient sent by the service provider; Based on the backpropagation gradient of the first model, the parameters of the first model are updated until the loss function of the second model converges, thus obtaining the target federated model.

23. An electronic device, characterized in that, It includes a processor and a memory, the memory storing multiple instructions; the processor loads instructions from the memory to perform steps in the federated learning method as described in any one of claims 1 to 11.

24. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute the steps of the federated learning method according to any one of claims 1 to 11.

25. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps of the federated learning method according to any one of claims 1 to 11.

Images