Distributed training method and apparatus, terminal device, and computer readable medium

Abstract

The present application relates to the technical field of artificial intelligence, and in particular to a distributed training method and device, terminal equipment and computer readable medium. The present application encrypts local original graph data, sends the encrypted graph data to other clients and obtains encrypted graph data of other clients, performs private intersection on the local encrypted graph data and the obtained encrypted graph data of other clients, obtains target graph data, trains the local model according to the target graph data, sends the trained model parameters to the central server to update the global model parameters, and sends the updated global model parameters to each local client to update the local model. The present application encrypts the local client graph data by the private intersection technology, interacts with the encrypted graph data of other clients, obtains data for model training, can ensure the security in the data interaction process, and improves the accuracy of the distributed training parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a distributed training method and device, a terminal device and a computer readable medium. BACKGROUND

[0002] At present, under the support of big data, intelligent guidance using artificial intelligence can provide recommended services for users more accurately, and the accuracy of the recommended services depends on the effect of model training. In order to improve the training effect of the model, a large amount of data needs to be obtained based on the model training of artificial intelligence, and many data are obtained through the network. For distributed training, each client holds its own data, and in the process of training, the data of other clients needs to be collected. In the process of data interaction, there is a risk of privacy data leakage. If there is no interaction, it may lead to poor training effect, which is not conducive to subsequent use. Therefore, how to ensure that the privacy data is not leaked and the accuracy of the training in the process of distributed training has become a problem to be solved. SUMMARY

[0003] Therefore, it is necessary to provide a distributed training method, device, terminal device and medium to solve the problem that each client holds its own data in the process of distributed training in the prior art, and the data of other clients needs to be interacted in the process of model training. In the process of data interaction, there is a risk of privacy data leakage.

[0004] In a first aspect, an embodiment of the present application provides a distributed training method, which is applied to a local client of a distributed training system, the distributed training system includes N local clients and at least one central server, N is an integer greater than 1, and the method includes:

[0005] Obtaining all local original graph data, and performing differential privacy encryption on the local original graph data to obtain first encrypted graph data corresponding to each local original graph data;

[0006] Sending all first encrypted graph data to other clients, and collecting encrypted graph data sent by the other clients to obtain second encrypted graph data corresponding to each other client, the other clients being local clients other than the sending client in the distributed training system;

[0007] Performing private intersection on all first encrypted graph data and all second encrypted graph data to determine common graph data, the common graph data being first encrypted graph data having the same statistical characteristics as the second encrypted graph data of all other clients;

[0008] Analyzing the common graph data to determine that the corresponding original graph data is target graph data;

[0009] training parameters, wherein the local training parameters are used to send to the central server to obtain global training parameters sent by the central server to update the local model.

[0010] In a second aspect, an embodiment of the present application provides a distributed training device, which is applied to a local client of a distributed training system, the distributed training system comprising N local clients and at least one central server, N being an integer greater than 1, and the distributed training device comprising:

[0011] a data encryption module configured to obtain all local original graph data, and perform differential privacy encryption on the local original graph data to obtain first encrypted graph data corresponding to each local original graph data;

[0012] a data transmission module configured to send all the first encrypted graph data to other clients, and collect encrypted graph data sent by the other clients to obtain second encrypted graph data corresponding to each other client, the other clients being local clients other than the sending client in the distributed training system;

[0013] a private intersection module configured to perform private intersection on all the first encrypted graph data and all the second encrypted graph data to determine common graph data, the common graph data being the first encrypted graph data having the same statistical characteristics as the second encrypted graph data of all the other clients;

[0014] a data analysis module configured to analyze the common graph data to determine that the corresponding original graph data is target graph data;

[0015] a model training module configured to train a local model according to the target graph data to obtain local training parameters, wherein the local training parameters are used to send to the central server to obtain global training parameters sent by the central server to update the local model.

[0016] In a third aspect, an embodiment of the present application provides a terminal device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the distributed training method of the first aspect when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the distributed training method of the first aspect.

[0018] Compared with the prior art, the embodiment of the present application has the beneficial effects that: the local original graph data is encrypted, the encrypted graph data is sent to other clients and the encrypted graph data of other clients is obtained, the local encrypted graph data and the obtained encrypted graph data of other clients are intersected in privacy to obtain target graph data, the local model is trained according to the target graph data, the trained model parameters are sent to the central server to update the global model parameters, and the updated global model parameters are sent to each local client to update the local model. The present application encrypts the local client graph data in privacy by the privacy intersection technology, and then interacts with the encrypted graph data of other clients to obtain the data for model training, which can ensure the security in the data interaction process and improve the accuracy of the distributed training parameters. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 is an application environment schematic diagram of a distributed training method provided by the first embodiment of the present application;

[0021] Figure 2 is a flow schematic diagram of a distributed training method provided by the first embodiment of the present application;

[0022] Figure 3 is a graph data schematic diagram of a user scoring a commodity provided by the first embodiment of the present application;

[0023] Figure 4 is a structure schematic diagram of a distributed training device provided by the second embodiment of the present application;

[0024] Figure 5 is a structure schematic diagram of a terminal device provided by the third embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures, techniques, etc. are presented in order to thoroughly understand the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits and methods are omitted to avoid unnecessary details that hinder the description of the present application.

[0026] It will be understood that, when used in the specification and the appended claims, the terms "comprise", "comprising", "include", "including", "contain", "containing" or variations thereof, mean "including but not limited to", unless expressly specified otherwise.

[0027] It will be further understood that the terms "and / or", "including", "comprising", "consisting" and "having", when used in the specification and the appended claims, shall not be construed as excluding the presence of one or more other features, integers, steps, operations, elements, and / or groups thereof, unless expressly stated otherwise.

[0028] As used in the specification and the appended claims, the term "if' can be construed to mean "when" or "once" or "in response to a determination" or "in response to a detection" of, depending on the context. Similarly, the phrase "if it is determined" or "if [a described condition or event] is detected" can be construed to mean "once it is determined" or "in response to a determination" or "once [a described condition or event] is detected" or "in response to a detection" of [a described condition or event], depending on the context.

[0029] In addition, the terms "first", "second", "third", etc. as used in the description of the specification and the appended claims are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0030] Reference in the specification to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrases "in one embodiment", "in some embodiments", "in other embodiments", "in additional embodiments", and so on, in various places in the specification are not necessarily all referring to the same embodiment, unless otherwise specifically stated. The terms "comprise", "comprising", "have", "having", and other variations thereof are meant to cover "including but not limited to", unless otherwise specifically stated.

[0031] Embodiments of the application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, to perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0032] The basic technology of artificial intelligence generally includes technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. The software technology of artificial intelligence mainly includes computer vision technology, robot technology, biometric identification technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0033] It should be understood that the size of the serial number of each step in the following embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0034] In order to illustrate the technical solutions of the present application, the following will be illustrated by specific embodiments.

[0035] The distributed training method based on local differential privacy provided by the embodiment of the present application can be applied in the application environment such as Figure 1 , the application environment is a distributed training system, which includes N local clients and at least one central server, N is an integer greater than 1, wherein the client and the server communicate. The client includes but is not limited to palmtop computer, desktop computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, cloud client, personal digital assistant (PDA) and the like. The server can be realized by an independent server or a server cluster composed of multiple servers.

[0036] Referring to Figure 2 , it is a flowchart of the distributed training method provided by the embodiment of the present application, and the distributed training method can be applied to the client in Figure 1 , the corresponding client connects the server through a preset application programming interface (API). Each local client model training parameter is sent to the central server, the central server trains the global model according to the model parameters sent by each local client, and then sends the global model parameters to each local client, and each local client trains the local model according to the received global model parameters. As shown in Figure 2 , the above method can include the following steps:

[0037] Step S201, all local original graph data is obtained, and the local original graph data is differentially private encrypted to obtain the first encrypted graph data corresponding to each local original graph data.

[0038] The graph data can be data corresponding to a data structure including nodes, edges, and weights. The node is a data subject of the graph data, the edge is a connection relationship between data subjects, and the weight is a weight between data subjects. For example, taking a user, a commodity graph, and a user scoring a commodity as an example, the graph data includes user data, commodity graph information, and a user scoring a commodity. The user data records the age stage, gender, and the like of the user, and is a node of the graph data. The commodity information records a graph formed by a commodity browsed or purchased by the user, and is also a node of the graph data. The user scoring the commodity records a scoring evaluation of the user on the commodity after purchasing the commodity, and is a weight between the user node and the commodity graph node. As shown in Figure 3 FIG. 1 is a schematic diagram of graph data of a user scoring a commodity. The left side represents a user, the right side represents a commodity graph corresponding to the user, the user and the commodity graph associated with the user are connected by an edge, and the number on the edge represents a score of the user on the commodity. The above information can constitute graph data.

[0039] In the present application, each local client obtains local original graph data, and performs privacy encryption on the original graph data of the local client by using a differential privacy encryption technology to obtain first encrypted graph data corresponding to each local original graph data. The use of the differential privacy technology for encryption can prevent other users from performing differential attacks on the local data.

[0040] For example, assuming that there is a marriage database, and it is known that there are 2 single persons and 8 married persons, then it can only be queried that how many persons are single and how many persons are married, and it is not known that who is single and who is married. Now, Zhang San goes to register his marital status, and it is found that there are 3 single persons after re-querying, so it can be known that Zhang San is single. Here, the appearance of Zhang San as a sample makes an attacker obtain a knowledge, and the knowledge can be used to accurately infer the required result.

[0041] Differential privacy is to make the knowledge of the attacker not change due to the appearance of a new sample. In order to achieve the above purpose, random noise is added, for example, in the above example, it is known that the query results are 2 and 3. After adding random noise, it becomes two random variables. If Zhang San is not in the database, the result obtained may be 2.5. If Zhang San is in the database, the result obtained may also be 2.5. The probability of obtaining a certain result from the two data sets is very close, so that the attacker cannot distinguish from which data set the result comes, so that the knowledge of the attacker does not change due to the appearance or non-appearance of the sample Zhang San.

[0042] The mathematical formula is explained as follows: a random mechanism is said to satisfy (ε, δ)-differential privacy if and only if for any Hamming distance difference of 1 data sets X and X', for any possible output t E Range (M), the following formula is established:

[0043] P(M(X) = t) < e ε P(M(X') = t) + d

[0044] where e is called privacy budget. In particular, it is called e-differential privacy when d = 0.

[0045] The local original graph data is encrypted by using the above process, so as to obtain the first encrypted graph data corresponding to each local original graph data.

[0046] In step S202, all the first encrypted graph data is sent to other clients, and the encrypted graph data sent by other clients is collected to obtain the second encrypted graph data corresponding to each other client.

[0047] In the present application, the other clients are local clients except the sender in the distributed training system. For any local client, the first encrypted graph data of the local client is sent to other clients, and the encrypted graph data sent by other clients, i.e. the second encrypted graph data, is received. The first encrypted graph data of the local client itself and the second encrypted graph data corresponding to other N-1 clients are stored in any local client.

[0048] For example, there are local client A, local client B and local client C, which respectively perform privacy encryption on the original graph data locally to generate the first encrypted graph data of each client. The local client A sends the first encrypted graph data of itself to the local client B and the local client C. The local client B sends the first encrypted graph data of itself to the local client A and the local client C. The local client C sends the first encrypted graph data of itself to the local client A and the local client B. Then, the local client A, the local client B and the local client C respectively generate the first encrypted graph data of itself and the second encrypted graph data corresponding to the other two local clients.

[0049] Each local client performs privacy encryption on the local original graph data, and sends the encrypted graph data to other local clients, so as to realize sharing of encrypted data while ensuring privacy and security of the data.

[0050] In step S203, all the first encrypted graph data and all the second encrypted graph data are subjected to privacy intersection to determine the common graph data.

[0051] The privacy intersection refers to a private set intersection (PSI) protocol that allows multiple parties holding respective data sets to perform intersection operation of multiple set. After the PSI protocol ends, the participants can obtain the intersection result, but the participants cannot obtain any information of the set data of other participants except the intersection.

[0052] For any local client, its first encrypted graph data includes user data, commodity graph information and user's score on commodities. Since the same user data and commodity graph information exist in different local client graph data, the statistical characteristics of the composed encrypted graph data have the same features. Therefore, the first encrypted graph data of any local client and the second encrypted graph data of other clients can obtain common graph data with the same statistical characteristics through private intersection.

[0053] For example, there are two local clients, local client N1 and local client N2. The graph data of local client N1 and local client N2 both include user A, user B, user C, commodity Figure 1 , commodity Figure 2 and commodity Figure 3 . According to the connection relationship between the user and commodity graphs, the common graph data of client N1 and client N2 can be obtained.

[0054] For example, there are three local clients, local client N1, local client N2 and local client N3. The graph data of local client N1, local client N2 and local client N3 both include user A, user B, user C, commodity Figure 1 , commodity Figure 2 and commodity Figure 3 . According to the connection relationship between the user and commodity graphs, the common graph data of client N1 and client N2 can be obtained. According to the connection relationship between the user and commodity graphs of the common graph data of client N1 and client N2 and the graph data of client N3, the common graph data of local client N1, local client N2 and local client N3 can be obtained.

[0055] Optionally, all first encrypted graph data and all second encrypted graph data are subjected to private intersection to determine the common graph data, which includes:

[0056] The first encrypted graph data is subjected to private intersection with any second encrypted graph data to obtain third encrypted graph data. The third encrypted graph data is subjected to private intersection with any remaining second encrypted graph data to obtain fourth encrypted graph data. This process is repeated until the last second encrypted graph data is subjected to private intersection to obtain the common graph data.

[0057] The first encrypted graph data of any local client is privately intersected with the second encrypted graph data corresponding to any other client to obtain third encrypted graph data, the third encrypted graph data is again privately intersected with the second encrypted graph data corresponding to any other remaining client to obtain fourth encrypted graph data, the fourth encrypted graph data is again privately intersected with the second encrypted graph data corresponding to any other remaining client, and the second encrypted graph data of the remaining clients is sequentially privately intersected according to the above method until all the second encrypted graph data is privately intersected, and finally the common graph data of all local clients is obtained.

[0058] For example, there are two local clients, local client N1 and local client N2, the local client N1 graph data includes user A, user B and user C, user A is associated with commodity 2 and commodity 3, user B is associated with commodity 1 and commodity 3, and user C is associated with commodity 2 and commodity 3. Figure 1 and commodity Figure 2 The local client N2 graph data includes user A and user B, user A is associated with commodity 2 and commodity 3, and user B is associated with commodity 1 and commodity 2. Figure 2 and commodity Figure 3 Through the private intersection technology, the third common graph data obtained includes user A and user B, user A is associated with commodity 2, and user B is associated with commodity 1.

[0059] For example, there are three local clients, local client N1, local client N2 and local client N3, the local client N1 graph data includes user A, user B and user C, user A is associated with commodity 2 and commodity 3, user B is associated with commodity 1 and commodity 3, and user C is associated with commodity 2 and commodity 3. Figure 1 and commodity Figure 2 The local client N2 graph data includes user A and user B, user A is associated with commodity 2 and commodity 3, and user B is associated with commodity 1 and commodity 2. Figure 2 and commodity Figure 3 Through the private intersection technology, the third common graph data obtained includes user A and user B, user A is associated with commodity 2, and user B is associated with commodity 1.

[0060] Step S204, the common graph data is parsed to determine the corresponding original graph data as the target graph data.

[0061] Optionally, according to the statistical characteristics of the first encrypted graph data, the mapping relationship of the first encrypted graph data is derived, and the target graph data is obtained by parsing the common graph data according to the above mapping relationship.

[0062] The mapping relationship of the encrypted graph data is an association relationship between graph nodes of the graph data, and the target graph data can be obtained by analyzing the common graph data according to the mapping relationship of the first encrypted graph data.

[0063] Optionally, according to the differential privacy encryption algorithm, an encryption key of the differential privacy encryption algorithm is analyzed, and the target graph data is obtained by analyzing the common graph data according to the encryption key.

[0064] The analysis of the common graph data can also mean that an encryption key of the differential privacy encryption algorithm is analyzed according to the differential privacy encryption algorithm, and the target graph data is obtained by analyzing the common graph data according to the encryption key. For example, a known noise with a mean value of 0 is added in the differential privacy encryption process of the original graph data, and the known noise with a mean value of 0 is the encryption key. The target graph data can be obtained by denoising and analyzing the common graph data according to the encryption key.

[0065] Optionally, after determining that the corresponding local original graph data is the target graph data, the target graph data is expanded to obtain expanded graph data, and the local model is trained using the expanded graph data to obtain local training parameters.

[0066] After obtaining the target graph data according to the above method, the target graph data is expanded, and the target graph data is augmented to achieve the purpose of expanding the target graph data. The local model is trained using the expanded graph data, which increases the range of training data and improves the training accuracy.

[0067] Optionally, the target graph data is augmented to obtain augmented graph data, and noise is added to the augmented graph data to determine that the graph data after adding the noise is the expanded graph data.

[0068] The augmentation of the target graph data is implemented by a K-hop algorithm, wherein the K-hop algorithm is a method of finding all nodes associated with a starting point through a breadth first search (BFS) from the starting point, and the found subgraph is called an ego-net of the starting point. Through the K-hop algorithm, the common graph data can be effectively expanded to obtain an expanded subgraph.

[0069] In the present application, each local client adds noise to the extended subgraph expanded by the K-hop algorithm, and the noise is added using the graph algorithm (LapGraph). For example, the total graph data is a graph composed of a user and a commodity graph, and the unweighted undirected graph generated thereby is represented by a user and a commodity graph association matrix, where 0 represents no rating relationship between the user and the commodity, and 1 represents a rating relationship between the user and the commodity. For such an association matrix, the sparsity T = n1 / n can be calculated first, where n1 is the number of 1s, and n is the number of matrix elements. Then, a Laplace noise with a mean of 0 and a strength of lambda is added to each matrix element, and a portion of the privacy budget (with a value of 0.01) is used to add noise protection to the sparsity T. The sparsity after the noise is added is T'. Finally, the top T' elements of the sparse matrix after the noise is added are retained and set to 1, and the rest are set to 0. Thus, the protection of the edge information of the unweighted undirected graph has been achieved. To protect the weight of the new graph, a Gaussian noise with a mean of 0 and a variance of sigma 2 is added to each edge, i.e. the weight of the edge is protected. Since the noise added to the matrix elements and the weight has a mean of 0, the graph algorithm (LapGraph) conforms to the epsilon-differential privacy.

[0070] In the present application, the noise with a mean of 0 is added to the data, so that the unique properties of individual data are concealed while the statistical properties of the overall data are guaranteed under the law of large numbers.

[0071] Whether the unique properties of individual data are "concealed" can be judged using the relative entropy index. Relative entropy is an index that measures the matching degree of two probability distributions. The greater the difference between the two distributions, the greater the relative entropy divergence. Based on relative entropy, it is defined whether the random mechanism of a certain data set changes significantly after a small perturbation. If the change is significant, it means that the privacy protection capability of the mechanism is weak, otherwise, the privacy is protected by the mechanism.

[0072] In step S205, the local model is trained according to the target graph data to obtain local training parameters, which are sent to the center server to obtain global training parameters from the center server to update the local model.

[0073] In the present application, according to the above steps, each local client legally utilizes the data of each party to establish a local graph neural network (GNN) model without leaking private data, and then trains the local GNN model according to the target graph data.

[0074] GNN is a machine learning algorithm that can extract important information from graph data and make useful predictions. The training of the model can include the following steps:

[0075] 1. Selecting a model, i.e., selecting an algorithm. Under the same data features, different algorithms will produce different results and have different application costs. In this invention, the GNN model is selected for training.

[0076] 2. After constructing the data and determining the algorithm model, the next step is to define the training data required by the model based on the business characteristics. In this invention, the training data used is the target graph data obtained by performing privacy intersection, augmentation, and noise addition on the graph data of each local client and other local clients.

[0077] 3. Model training: After determining the training data, model training is performed. The target image data is used for local model training to obtain trained model parameters, including local model gradients.

[0078] 4. Model optimization: The process of model optimization is an iterative upgrade of the model and data. In this invention, the local model is continuously learned and optimized based on the continuous changes in the graph data of each local client to achieve the best prediction effect.

[0079] The local GNN model is trained according to the above model algorithm. After training, the local graph neural network model parameters are obtained. The trained model parameters are sent to the central server. The central server updates the global GNN model parameters according to the local GNN model parameters sent by each local client, and obtains the updated global GNN model parameters. The updated global GNN model parameters are then sent to each local client. Each local client updates its local GNN model parameters according to the received global GNN model parameters.

[0080] In this invention, the central server of the distributed training system deploys a global GNN model. Based on the local GNN model parameters received from each local client, it updates the global GNN model to obtain updated global GNN model parameters. The updated global GNN model parameters are then sent to each local client. Each local client receives the global GNN model parameters sent by the central server and updates its own GNN model parameters to obtain an updated local GNN model. The updated local GNN model is then used for data recommendation.

[0081] The application encrypts local original graph data with differential privacy, sends the encrypted graph data to other clients and acquires encrypted graph data of other clients, performs private set intersection on the local encrypted graph data and the acquired encrypted graph data of other clients, obtains target graph data, performs extension and noise addition on the target graph data to obtain extended graph data, trains a local model according to the extended graph data, sends the trained model parameters to a central server to update global model parameters, and sends the updated global model parameters to each local client to update the local model. The application encrypts local client graph data with privacy through the private set intersection technology, interacts with encrypted graph data of other clients, obtains data for model training, can guarantee the security in the data interaction process, and improves the accuracy of distributed training parameters.

[0082] The application discloses a local differential privacy-based distributed training method, Figure 4 The application discloses a local differential privacy-based distributed training method,

[0083] The application discloses a local differential privacy-based distributed training method, Figure 4 The distributed training device comprises a data acquisition module 40, a data encryption module 41, a data transmission module 42 and a private set intersection module 43.

[0084] The data encryption module 41 is used for acquiring all local original graph data and encrypting the local original graph data with differential privacy to obtain first encrypted graph data corresponding to each local original graph data.

[0085] The data transmission module 42 is used for sending all the first encrypted graph data to other clients and acquiring encrypted graph data sent by the other clients to obtain second encrypted graph data corresponding to each other client, wherein the other clients are local clients except the sending client in the distributed training system.

[0086] The private set intersection module 43 is used for performing private set intersection on all the first encrypted graph data and all the second encrypted graph data to determine common graph data, wherein the common graph data is the first encrypted graph data that has the same statistical characteristics as the second encrypted graph data of all the other clients.

[0087] The data analysis module 44 is configured to analyze the common graph data and determine the corresponding original graph data as target graph data.

[0088] The model training module 45 is configured to train the local model according to the target graph data, obtain local training parameters, and send the local training parameters to the center server to update the local model with global training parameters sent by the center server.

[0089] Optionally, the privacy intersection module 43 comprises:

[0090] The privacy intersection unit is configured to perform privacy intersection between the first encrypted graph data and any second encrypted graph data to obtain third encrypted graph data, perform privacy intersection between the third encrypted graph data and any remaining second encrypted graph data to obtain fourth encrypted graph data, and perform privacy intersection between the fourth encrypted graph data and the last second encrypted graph data to obtain common graph data.

[0091] Optionally, the data analysis module 44 comprises:

[0092] The first analysis unit is configured to derive a mapping relationship of the first encrypted graph data according to a statistical feature of the first encrypted graph data, analyze the common graph data according to the mapping relationship, and obtain target graph data.

[0093] The second analysis unit is configured to analyze an encryption key of the differential privacy encryption algorithm according to the differential privacy encryption algorithm, analyze the common graph data according to the encryption key, and obtain target graph data.

[0094] The data augmentation unit is configured to augment the target graph data to obtain augmented graph data.

[0095] The data noise adding unit is configured to add noise to the augmented graph data to obtain extended graph data.

[0096] Optionally, the model training module 45 comprises:

[0097] The data expansion unit is configured to expand the target graph data to obtain extended graph data after determining that the corresponding local original graph data is target graph data.

[0098] The local model training unit is configured to train the local model using the extended graph data to obtain local training parameters.

[0099] It should be noted that the information interaction, execution process, and the like between the modules are based on the same concept as the method embodiments, and the specific functions and the technical effects brought by the modules can be referred to the method embodiments part, which will not be described here.

[0100] Figure 5 Figure 3 shows a schematic diagram of a terminal device according to an embodiment of the application. The terminal device comprises at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, the processor implementing the steps of any of the above-described embodiments of the method for distributed training when executing the computer program. Figure 5 Figure 5 The terminal device can comprise, but is not limited to, a processor, a memory. It is to be understood by those skilled in the art that the terminal device is only an example and does not constitute a limitation on the terminal device, the terminal device can comprise more or less components than shown, or combine certain components, or different components, for example, it can also comprise a network interface, a display screen, and an input device, etc.

[0101] The terminal device can comprise, but is not limited to, a processor, a memory. It is to be understood by those skilled in the art that the terminal device is only an example and does not constitute a limitation on the terminal device, the terminal device can comprise more or less components than shown, or combine certain components, or different components, for example, it can also comprise a network interface, a display screen, and an input device, etc. Figure 5 The processor can be a CPU, and can also be other general-purpose processors, Digital Signal Processors (DSP), Application Specific Integrated Circuits (ASIC), Field-Programmable Gate Arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0102] The memory comprises a readable storage medium, an internal memory, etc., wherein the internal memory can be a memory of the client, and the internal memory provides an environment for the operation of the operating system and the computer readable instructions in the readable storage medium. The readable storage medium can be a hard disk of the client, and in other embodiments, it can also be an external storage device of the client, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory can comprise both an internal storage unit of the client and an external storage device. The memory is used to store an operating system, an application program, a BootLoader, data, and other programs, such as program codes of computer programs, etc. The memory can also be used to temporarily store data that has been output or will be output.

[0103]

[0104] ​​Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0105] Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit, and the integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application.

[0106] The specific working process of the units and modules in the above device can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here. If the integrated unit is realized in the form of software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program to instruct related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps of the above-mentioned method embodiments when executed by a processor.

[0107] The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form.

[0108] The computer readable medium can at least include any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), electrical carrier signal, telecommunication signal and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, computer readable medium cannot be electrical carrier signal and telecommunication signal.

[0109] The present application realizes all or part of the processes in the above-mentioned embodiment methods, which can also be completed by a computer program product. When the computer program product runs on the client, it makes the client execute the steps that can realize the above-mentioned method embodiments.

[0110] In the above-mentioned embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0111] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0112] In the embodiments provided by the present application, it should be understood that the disclosed apparatuses / clients and methods can be implemented in other ways. For example, the apparatus / client embodiments described above are merely schematic. The division of the modules or units is merely a logical function division, and there can be another division in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0113] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0114] The above embodiments are only used to illustrate the technical solutions of the present application, but not limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A distributed training method, characterized in that, The method is applied to local clients of a distributed training system, the distributed training system comprising N local clients and at least one central server, where N is an integer greater than 1, and the method includes: All local raw graph data are acquired, and differential privacy encryption is performed on the local raw graph data to obtain the first encrypted graph data corresponding to each local raw graph data. All first encrypted graph data are sent to other clients, and the encrypted graph data sent by the other clients are collected to obtain the second encrypted graph data corresponding to each other client. The other clients are local clients in the distributed training system other than the sender. Perform privacy intersection calculations on all first encrypted graph data and all second encrypted graph data to determine common graph data. The common graph data is the first encrypted graph data that has the same statistical characteristics as the second encrypted graph data of all other clients. The shared graph data is parsed to determine the corresponding original graph data as the target graph data; The local model is trained based on the target graph data to obtain local training parameters. The local training parameters are sent to the central server to obtain the global training parameters sent by the central server to update the local model. The step of performing privacy intersection calculations on all first encrypted graph data and all second encrypted graph data to determine the shared graph data includes: The first encrypted graph data is subjected to privacy intersection with any of the second encrypted graph data to obtain the third encrypted graph data. The third encrypted graph data is then subjected to privacy intersection with any of the remaining second encrypted graph data to obtain the fourth encrypted graph data. This process continues until privacy intersection is performed with the last second encrypted graph data to obtain the shared graph data.

2. The distributed training method according to claim 1, characterized in that, The step of parsing the shared graph data to determine the corresponding original graph data as the target graph data includes: Based on the statistical characteristics of the first encrypted graph data, the mapping relationship of the first encrypted graph data is derived, and the shared graph data is parsed according to the mapping relationship to obtain the target graph data.

3. The distributed training method according to claim 1, characterized in that, The step of parsing the shared graph data to determine the corresponding original graph data as the target graph data includes: Based on the differential privacy encryption algorithm, the encryption key of the differential privacy encryption algorithm is obtained by parsing, and the shared graph data is parsed based on the encryption key to obtain the target graph data.

4. The distributed training method according to claim 1, characterized in that, After determining that the corresponding local original map data is the target map data, the process also includes: The target graph data is expanded to obtain expanded graph data; The step of training the local model based on the target map data to obtain local training parameters includes: The local model is trained using the extended graph data to obtain local training parameters.

5. The distributed training method according to any one of claims 1 to 4, characterized in that, The process of expanding the target map data to obtain expanded map data includes: The target graph data is augmented to obtain augmented graph data; Noise is added to the augmented graph data, and the graph data with added noise is determined to be the extended graph data.

6. A distributed training device, characterized in that, The device is applied to a local client of a distributed training system, which includes N local clients and at least one central server, where N is an integer greater than 1. The distributed training device includes: The data encryption module is used to acquire all local raw graph data and perform differential privacy encryption on the local raw graph data to obtain the first encrypted graph data corresponding to each local raw graph data. The data transmission module is used to send all the first encrypted graph data to other clients, and to collect the encrypted graph data sent by the other clients to obtain the second encrypted graph data corresponding to each other client. The other clients are local clients in the distributed training system other than the sender. The privacy intersection module is used to perform privacy intersection on all first encrypted graph data and all second encrypted graph data to determine common graph data. The common graph data is the first encrypted graph data that has the same statistical characteristics as the second encrypted graph data of all other clients. The data parsing module is used to parse the shared graph data and determine the corresponding original graph data as the target graph data; The model training module is used to train the local model based on the target graph data to obtain local training parameters. The local training parameters are sent to the central server to obtain the global training parameters sent by the central server to update the local model. The privacy-preserving intersection module includes: The privacy intersection unit is used to perform privacy intersection with the first encrypted graph data and any second encrypted graph data to obtain the third encrypted graph data, and then perform privacy intersection with any remaining second encrypted graph data to obtain the fourth encrypted graph data, until privacy intersection is performed with the last second encrypted graph data to obtain the shared graph data.

7. The distributed training device according to claim 6, characterized in that, The distributed training device also includes: The data expansion unit is used to expand the target graph data after determining that the corresponding local original graph data is the target graph data, so as to obtain expanded graph data. The model training module includes: The local model training unit is used to train the local model using the extended graph data to obtain local training parameters.

8. A terminal device, characterized in that, The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the distributed training method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the distributed training method as described in any one of claims 1 to 5.

Images