A graph federation training method, apparatus, device, storage medium, and product.

Abstract

This application discloses a graph federation training method, apparatus, device, storage medium, and product. Participants obtain public keys generated by K participants based on homomorphic encryption algorithms. Using the public key of the a-th participant, they encrypt the gradient value of the current model parameters for the a-th participant, thereby obtaining the encrypted global gradient value of the current model parameters for that participant. After updating the current model parameters using this encrypted global gradient value, the updated model parameters are encrypted again, thus obtaining the updated encrypted model parameters for all K participants. Finally, decryption is performed to obtain the updated model parameters. This embodiment avoids the possibility of attackers using model parameters to infer sample data when there are many model parameters, thereby preventing data leakage risks and achieving data usability without visibility, thus improving data security.

Description

Technical Field

[0001] This application relates to the field of graph federation technology, and in particular to a graph federation training method, apparatus, device, storage medium and product. Background Technology

[0002] Federated Graph Learning (FGL) is an application of federated learning to graph data. It addresses the privacy protection issue of graph data by training graph neural networks in a federated manner. However, current federated graph training processes involve the leakage of model parameters from the database. This poses a risk of data leakage, especially when the number of model parameters is large, as attackers can use these parameters to infer sample data. Summary of the Invention

[0003] This application provides a graph federation training method, apparatus, device, storage medium, and product to address the risk of data leakage in existing technologies where attackers can infer sample data based on model parameters when the number of model parameters is large.

[0004] To achieve the above objectives, embodiments of this application provide a graph federation training method, applied to any one of K participants in graph federation training, wherein the participant possesses at least the public keys generated by each of the K participants based on a homomorphic encryption algorithm. The graph federation training method includes:

[0005] In this iteration, the current model parameters of each of the K participants are obtained, and the local data is used to train the model to obtain the gradient values ​​of each current model parameter; the current global model is determined by the current model parameters of each of the K participants.

[0006] The gradient values ​​of the current model parameters under the responsibility of the a-th participant are encrypted using the public key of the a-th participant to obtain the encrypted gradient values ​​of each current model parameter; the encrypted global gradient value of the current model parameters under the responsibility of the participant is obtained; wherein the encrypted global gradient value is composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant obtained by training and encrypting with K participants, a = 1, 2, ..., K;

[0007] The current model parameters of the participant are updated based on the encrypted global gradient value to obtain the updated model parameters; the updated model parameters are then encrypted using the participant's public key to obtain the updated encrypted model parameters.

[0008] Obtain the updated encrypted model parameters of K participants, and decrypt them to obtain the updated model parameters.

[0009] As an improvement to the above scheme, the K participants are connected end-to-end to form a ring network architecture.

[0010] As an improvement to the above scheme, the participant is the kth participant. The step of encrypting the gradient values ​​of the current model parameters under the responsibility of the i-th participant using the public key of the i-th participant to obtain the encrypted gradient values ​​of each current model parameter; obtaining the encrypted global gradient values ​​of the current model parameters under the responsibility of the participant includes:

[0011] In the first encrypted gradient transmission process, the k-th participant uses the public key of the (k-1)-th participant to encrypt the gradient value of the current model parameter that the (k-1)-th participant is responsible for, and obtains the encrypted gradient value, which is used as the transmission value of the first encrypted gradient transmission process and transmitted to the (k+1)-th participant.

[0012] In the b-th encrypted gradient transmission process, the k-th participant uses the public key of the kb-th participant to encrypt the gradient value of the current model parameter that the kb-th participant is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)-th participant in the (b-1)-th encrypted gradient transmission process to obtain the transmission value of the b-th encrypted gradient transmission process, and transmits it to the (k+1)-th participant; where b = 2, 3, ..., K-1;

[0013] During the Kth encrypted gradient transmission process, the kth participant uses its public key to encrypt the gradient value of the current model parameter that it is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)th participant in the K-1th encrypted gradient transmission process to obtain the encrypted global gradient value of the current model parameter that the kth participant is responsible for.

[0014] As an improvement to the above scheme, after obtaining the updated encrypted model parameters of the K participants and decrypting them to obtain the updated model parameters, the graph federation training method further includes:

[0015] When the preset iteration stopping condition is met, the iteration stops, and the global model is updated according to the latest parameters of each model.

[0016] The iteration stopping condition includes any of the following:

[0017] The current iteration count is greater than the preset iteration count threshold;

[0018] The total change between the objective function of each local model in this iteration and the objective function of each local model in the previous iteration is less than the preset change value; wherein, each local model is trained by each participant using their own local data.

[0019] As an improvement to the above scheme, the step of updating the current model parameters of the participating party based on the encrypted global gradient value to obtain the updated model parameters includes:

[0020] Based on the learning rate, the total number of samples, and the encrypted global gradient value, calculate the first encrypted change value of the current model parameters for which the participant is responsible;

[0021] Decrypt the first encrypted change value to obtain the first change value;

[0022] Based on the first change value, the current model parameters of the participating party are updated to obtain the updated model parameters.

[0023] As an improvement to the above scheme, if it is the first iteration, the initial model parameters that the participating party is responsible for are used as the current model parameters that the participating party is responsible for.

[0024] If it is not the first iteration, the model parameters that the participant is responsible for, obtained after the previous iteration update, will be used as the current model parameters that the participant is responsible for.

[0025] To achieve the above objectives, embodiments of this application also provide a graph federation training device, applied to any one of the K participants in graph federation training, wherein the participant possesses at least the public keys generated by each of the K participants based on a homomorphic encryption algorithm, and the graph federation training device includes:

[0026] The training module is used in this iteration to obtain the current model parameters of each of the K participants, train the model on the local data, and obtain the gradient values ​​of each current model parameter; the current global model is determined by the current model parameters of each of the K participants.

[0027] An encrypted transmission module is used to encrypt the gradient values ​​of the current model parameters under the responsibility of the a-th participant using the public key of the a-th participant, thereby obtaining encrypted gradient values ​​of each current model parameter; and to obtain the encrypted global gradient value of the current model parameters under the responsibility of the participant; wherein the encrypted global gradient value is composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant, which are obtained by training and encrypting with K participants, where a = 1, 2, ..., K;

[0028] The update module is used to update the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters; and to encrypt the updated model parameters using the participant's public key to obtain the updated encrypted model parameters.

[0029] The decryption module is used to obtain the updated encrypted model parameters of K participants and decrypt them to obtain the updated model parameters.

[0030] To achieve the above objectives, this application also provides a graph federation training device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the graph federation training method as described above.

[0031] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium, the computer-readable storage medium including a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the graph federation training method as described above.

[0032] To achieve the above objectives, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implement the graph federation training method as described above.

[0033] Compared with existing technologies, the graph federation training method, apparatus, device, storage medium, and product provided in this application involve participants obtaining public keys generated by K participants based on homomorphic encryption algorithms. Using the public key of the a-th participant, they encrypt the gradient value of the current model parameters for that participant, thus obtaining the encrypted global gradient value of the current model parameters. After updating their own model parameters using this encrypted global gradient value, they encrypt the updated model parameters, obtaining the updated encrypted model parameters for all K participants. Decryption then yields the updated model parameters. This ensures that model parameters are never released in plaintext during the entire graph federation training process, preventing attackers from using model parameters to infer sample data when there are many model parameters, thus avoiding the risk of data leakage. This achieves data usability without visibility, improving data security. Furthermore, since participants only update the model parameters for their own responsibility, the computational load is low. Attached Figure Description

[0034] Figure 1 This is a flowchart of a graph federation training method provided in an embodiment of this application;

[0035] Figure 2 This is a network structure diagram of various participating parties provided in an embodiment of this application;

[0036] Figure 3 This is a structural block diagram of a graph federation training device provided in an embodiment of this application;

[0037] Figure 4 This is a structural block diagram of a graph federation training device provided in an embodiment of this application. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0039] It is worth noting that the embodiments of this application mainly focus on the lateral modeling scenario in node-level federated graph neural networks. Each participant stores a subgraph that is a horizontally segmented portion of the potential complete graph. These subgraphs constitute the potential complete graph, and the connections between them are lost due to data isolation during storage. Here, it is assumed that there are a total of K participants in the scenario, and each participant has its own dataset D. (k) =(G (k) ,Y (k) ), where G (k) =(V (k) E (k) V is the graph dataset of the k-th participant; (k) The node set of the k-th participant contains node feature data. m represents a node; E (k) Let k be the edge set of the k-th participant, containing edge feature data. m and n are two nodes of the edge; Y (k) It is a subgraph G (k) The tag set, N k Subgraph G (k) The number of nodes. Each participant has a localized Graph Neural Network (GNN) model (hereinafter referred to as the local model, which is trained by the participant using its own local data) to learn graph representations and make predictions. Each participant trains its local model by training and updating the model parameters, thus completing the graph federation training.

[0040] See Figure 1 , Figure 1 This is a flowchart of a graph federation training method provided in an embodiment of this application. The graph federation training method is applied to any one of the K participants in the graph federation training, and the participant possesses at least the public keys generated by the K participants based on homomorphic encryption algorithms. The graph federation training method includes:

[0041] S1. In this iteration, obtain the current model parameters of each of the K participants, train the local data, and obtain the gradient values ​​of each current model parameter; among them, the current global model is determined by the current model parameters of each of the K participants.

[0042] It is understandable that each participant transmits / encrypts the model parameters they are responsible for, so that each participant can directly obtain or decrypt the model parameters that each of the K participants is responsible for.

[0043] Optionally, if it is the first iteration, the initial model parameters that the participant is responsible for are used as the current model parameters that the participant is responsible for.

[0044] If it is not the first iteration, the model parameters that the participant is responsible for, obtained after the previous iteration update, will be used as the current model parameters that the participant is responsible for.

[0045] Understandably, during the first iteration, each participant is responsible for initializing the model parameters they are responsible for, thus obtaining the initial model parameters, i.e., the k-th participant initializes the model parameters W. (k) This includes, but is not limited to, message functions. The parameter θ and the update function The parameter φ is then used. The initial model parameters that the participant is responsible for are then sent to the other participants, so that each participant has the initial model parameters that all other participants are responsible for. Each participant, upon receiving the initial model parameters from all other participants, completes the initialization of its local model. Since the initial model parameters do not contain data information, they do not require encrypted transmission; only the updated model parameters received subsequently require encrypted transmission.

[0046] In the t-th iteration, for the k-th participant, the updated model parameters obtained in the (t-1)-th iteration are used as the current model parameters for each of the K participants in the t-th iteration. That is, the model parameters obtained in step S4 of the (t-1)-th iteration are used as the current model parameters for each of the K participants in step S1 of the t-th iteration.

[0047] It should be noted that in one iteration, for any participant, the process of training its own local data based on the current model parameters includes: a forward computation phase and a backpropagation phase.

[0048] Here, the Message Passing Neural Network (MPNN) framework is used to unify the local model into this framework. The forward computation stage includes the message passing stage and the readout stage.

[0049] The message passing phase includes two steps: (1) the model collects and transforms messages from neighbors; (2) the model uses aggregated gating messages to update the hidden state of nodes. Mathematically, for the k-th participant and layer indices l = 0, ..., L-1, the L-layer MPNN is formalized as follows:

[0050]

[0051] in, Let l be the latent vector of the i-th node of the k-th participant. Here, is the node feature of the k-th participant, l is the layer ID of the local model, AGG is the aggregation function (e.g., the sum function in the GCN model), and N is the node feature of the k-th participant. i It is the set of adjacent nodes of node i. It is a message function that updates the hidden layer state and the current node based on the message function parameter θ to obtain the message passing result. It is the update function, and φ is the parameter of the update function.

[0052] Readout stage: After propagation through the L-layer MPNN, the readout stage calculates the feature vector based on the hidden state of the last MPNN layer using formula (2), which is then used for downstream prediction tasks.

[0053]

[0054] In the node classification task, S represents a node. Let R represent the set of nodes associated with node S in the k-th participant. δ This represents the readout function, where δ is the parameter of the readout function. This represents the probability that a node in the k-th participant belongs to the positive class.

[0055] Backpropagation phase: For ease of representation, define W = {W θ U φ ,R δ Let} represent all learnable model parameters in the k-th participant. Therefore, the modeling process of the local model can be formulated as a distributed optimization problem:

[0056]

[0057] f (k) (W) represents the objective function value of the k-th participant on the local data, and N k Let N be the sample size of the k-th participant, and N be the total sample size of all participants. (k) Let be the sample size of the k-th participant. The objective function is the weighted average of the objective functions of all participants, i.e., the global objective function. The goal is to minimize this global objective function by optimizing the parameter W. L is the loss function, which is based on... calculate, For the node feature data of the i-th sample of the k-th participant, Let i be the edge feature data of the i-th sample of the k-th participant. This refers to the label data for the i-th sample of the k-th participant.

[0058] The objective function f is calculated using the backpropagation algorithm. (k) The gradient (ω) relative to the model parameter ω will vary depending on the sample data of different participants.

[0059]

[0060] In other words, for the k-th participant, in this iteration, the current model parameters for each of the K participants are obtained, the local data is used for training, and the gradient values ​​of each current model parameter are obtained, including:

[0061] During the message passing phase, the last hidden layer vector of each node is calculated according to formula (1).

[0062] During the readout phase, the prediction probability for each phase is calculated according to formula (2);

[0063] The result f of the objective function on the local data is calculated according to formula (3). (k) (W);

[0064] Using the backpropagation algorithm and based on formula (4), the gradient values ​​of the objective function with respect to each current model parameter are calculated. in, This represents the gradient value of the current model parameter ω obtained by the k-th participant based on local data (i.e., its own data).

[0065] S2. Encrypt the gradient values ​​of the current model parameters under the responsibility of the a-th participant using the public key of the a-th participant to obtain the encrypted gradient values ​​of each current model parameter; obtain the encrypted global gradient value of the current model parameters under the responsibility of the participant; wherein, the encrypted global gradient value is composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant obtained by training and encrypting with K participants, a = 1, 2, ..., K;

[0066] In one example, in the k-th participant, the gradient value of the current model parameter that the first participant is responsible for is encrypted using the public key of the first participant, the gradient value of the current model parameter that the second participant is responsible for is encrypted using the public key of the second participant, and so on, until the gradient value of the current model parameter that the K-th participant is responsible for is encrypted using the public key of the K-th participant. This gives the k-th participant the gradient values ​​of the current model parameters that the K participants are responsible for, thus obtaining the encrypted gradient values ​​of each current model parameter.

[0067] It is understandable that the encrypted global gradient value of the current model parameters obtained by the k-th participant is represented as follows: in, This represents the encrypted gradient value of the current model parameter ω, which is trained by the a-th participant based on local data and is the responsibility of the k-th participant. This encrypted gradient value is obtained by encrypting the public key of the k-th participant.

[0068] In other words, after obtaining the encrypted gradient values ​​of each current model parameter, each participant transmits them directly or indirectly to the participant responsible for that current model parameter, based on the network structure formed by the K participants (i.e., the encrypted gradient values ​​of the current model parameter are ultimately aggregated at the responsible participant). This ensures that each participant receives the encrypted gradient values ​​of the model parameters they are responsible for, calculated by the other participants. For example, the encrypted gradient values ​​of the model parameters they are responsible for, received by the k-th participant, calculated by the other participants, are respectively represented as follows: in, These values ​​are not equal to k, and are transmitted directly or indirectly to the kth participant through the other participants.

[0069] S3. Update the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters; encrypt the updated model parameters using the participant's public key to obtain the updated encrypted model parameters.

[0070] Understandably, in the current model parameter update process, each participant is only responsible for a portion. The update of the current model parameters, that is, only updating the parameters of the current model that I am responsible for, is represented by W. (k) This indicates that the k-th participant is responsible for updating the current model parameters, and all W... (k) It forms a complete

[0071] S4. Obtain the updated encrypted model parameters of K participants, and decrypt them to obtain the updated model parameters.

[0072] In this embodiment, each participant updates the current model parameters they are responsible for, obtaining their updated model parameters. These updated parameters are then transmitted to the other participants via encrypted transmission, allowing each participant to obtain their own updated encrypted model parameters from all other participants. Each participant then decrypts the parameters using their private key. For example, after obtaining the updated encrypted model parameters from the K participants, the k-th participant uses its own Prvkey, generated based on a homomorphic encryption algorithm. k These encrypted model parameters are then decrypted to obtain the updated model parameters.

[0073] It is understood that each of the K participants in the graph federation training can implement the graph federation training method provided in the embodiments of this application. Steps S1-S4 describe one of the iteration processes of any participant. Each iteration can obtain the updated model parameters. Then, the updated model parameters are aggregated to complete the update of the global model. At this time, the local model is reset with the updated global model to confirm the latest local model until the iteration ends and the latest local model is obtained.

[0074] Each participant possesses K public keys generated by itself based on a homomorphic encryption algorithm; specifically, for any participant k, participant k has its own public key Pubkey generated based on the homomorphic encryption algorithm. k and your private key Prvkey k Then, use your public key Pubkey k Send it to the other participants; each participant does the same, so that each participant not only has its own generated private key PrvKey, but also... k It also possesses the public keys generated by all participants; here, the homomorphic encryption algorithm satisfies the requirements of additive homomorphism and scalar multiplication homomorphism, such as the Paillier algorithm (additive homomorphic encryption algorithm).

[0075] Therefore, it can be seen that the embodiments of this application adopt homomorphic encryption method in the process of model parameter training and updating, adhere to the principle of plaintext not leaving the database, realize the entire process of ciphertext calculation and transmission, and finally achieve the calculation result in ciphertext state is consistent with the calculation result in plaintext, and is more secure and reliable.

[0076] Optionally, the K participants are connected end-to-end to form a ring network architecture.

[0077] It is understood that this application embodiment does not have a central server, but transmits data between the various participating parties, which can avoid the risk of data leakage.

[0078] In an optional embodiment, the participant is the kth participant, and the step of encrypting the gradient values ​​of the current model parameters under the responsibility of the i-th participant using the public key of the i-th participant to obtain the encrypted gradient values ​​of each current model parameter; obtaining the encrypted global gradient values ​​of the current model parameters under the responsibility of the participant includes:

[0079] In the first encrypted gradient transmission process, the k-th participant uses the public key of the (k-1)-th participant to encrypt the gradient value of the current model parameter that the (k-1)-th participant is responsible for, and obtains the encrypted gradient value, which is used as the transmission value of the first encrypted gradient transmission process and transmitted to the (k+1)-th participant.

[0080] In the b-th encrypted gradient transmission process, the k-th participant uses the public key of the kb-th participant to encrypt the gradient value of the current model parameter that the kb-th participant is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)-th participant in the (b-1)-th encrypted gradient transmission process to obtain the transmission value of the b-th encrypted gradient transmission process, and transmits it to the (k+1)-th participant; where b = 2, 3, ..., K-1;

[0081] During the Kth encrypted gradient transmission process, the kth participant uses its public key to encrypt the gradient value of the current model parameter that it is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)th participant in the K-1th encrypted gradient transmission process to obtain the encrypted global gradient value of the current model parameter that the kth participant is responsible for.

[0082] In this embodiment of the application, in the ring network architecture composed of K participants, each participant only sends data to the next participant. The kth participant sends the transmission value to the (k+1)th participant. After one loop, each participant obtains the encrypted global gradient value of the current model parameter it is responsible for, and completes the update of the current model parameter it is responsible for. Then, after another loop, the updated model parameter is distributed to all participants, so that each participant obtains the updated model parameter and completes the update of the global model.

[0083] Suppose that the k-th participant is responsible for the model parameter set W. (k) , which contains multiple parameters ω∈W (k) Take any ω∈W (k) To better understand the embodiments of this application, the process of obtaining the encrypted global gradient value is explained using the model parameter ω, which is the responsibility of the kth participant, as an example:

[0084] First, starting with the next(k)th participant, the calculation is performed using the public key of the kth participant. Encryption is performed to obtain Here, next(k) = mod(k+1, K) represents the next participant after the k-th participant, represented by [[ ]]. k Indicates using the public key PubKey k The result of homomorphic encryption. Then the next(k)th participant will... Send it to the mod(k+2,K)th participant.

[0085] The mod(k+2,K)th participant first similarly... Encryption is performed to obtain Then combine it with the received Summation yields Then it is sent to the mod(k+3,K)th participant.

[0086] This process is repeated K-1 times until the kth participant is reached. The kth participant then receives the message sent by the previous participant. The encrypted gradient values ​​calculated by the participants for the model parameters they are responsible for. Summing is performed to obtain the model parameters ω.

[0087] For ease of understanding, let's assume there are four participants. For example, if we want to update the model parameter ω1 that the first participant is responsible for, then the second participant first uses the first participant's public key to encrypt... get And send it to the third participant, the third participant 3, to calculate. And send it to the fourth participant, who then calculates... It is then sent to the first participant, who ultimately calculates... in, This represents the encrypted gradient value of the model parameters trained by the second participant using its own data, which is the parameter value of the model managed by the first participant. The remaining parameters... and The meaning is similar and will not be elaborated here.

[0088] The above describes only the update process of a parameter matrix. In actual calculations, the update calculation of all model parameters is performed as follows: First, any participant k encrypts the gradient values ​​of all model parameters for which its predecessor participant k-1 is responsible using the (k-1)th public key and sends it to the (k+1)th participant; then, each participant k encrypts the gradient values ​​of all model parameters for which its predecessor participant k-2 is responsible using the (k-2)th key, sums them with the encrypted gradient values ​​sent by its predecessor participant k-1, and sends the sum to the (k+1)th participant. This process is repeated K-1 times until participant k encrypts the gradient values ​​of all model parameters for which it is responsible using the kth key, sums them with the encrypted gradient values ​​sent by its predecessor participant k-1, and finally obtains the result. Finally, all model parameters are updated according to formulas (5) and (6).

[0089] For ease of understanding, let's assume there are four participants. The fourth participant first calculates the model parameter ω3, which is the responsibility of the third participant. This is then sent to the first participant, and then the fourth participant combines the model parameters ω2, which are the responsibility of the second participant, with the parameters sent by the third participant. Calculated It is sent to the first participant; then, the model parameters ω1, which are the responsibility of the first participant, are combined with those sent by the third participant. Calculated And send it to the first participant; finally, the fourth participant, for the model parameters ω4 they are responsible for, combines them with those sent by the third participant. Calculated

[0090] In other words, for any participant, K encryption gradient transmission processes are required:

[0091] In the first encrypted gradient transmission process, the k-th participant uses the public key of the (k-1)-th participant to encrypt the gradient value of the current model parameter that the (k-1)-th participant is responsible for, and obtains the encrypted gradient value, which is used as the transmission value of the first encrypted gradient transmission process and transmitted to the (k+1)-th participant.

[0092] In the b-th encrypted gradient transmission process, the k-th participant uses the public key of the kb-th participant to encrypt the gradient value of the current model parameter that the kb-th participant is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)-th participant in the (b-1)-th encrypted gradient transmission process to obtain the transmission value of the b-th encrypted gradient transmission process, and transmits it to the (k+1)-th participant; where b = 2, 3, ..., K-1;

[0093] During the Kth encrypted gradient transmission process, the kth participant uses its public key to encrypt the gradient value of the current model parameter that it is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)th participant in the K-1th encrypted gradient transmission process to obtain the encrypted global gradient value of the current model parameter that the kth participant is responsible for.

[0094] In other words, each participant transmits the transmission value of the a-th (a = 1, 2, ..., K) transmission to the next participant. This transmission value can be used by the next participant. During the (a+1)-th encryption gradient transmission, the next participant adds the received transmission value to the corresponding encryption gradient value to obtain the transmission value of the next participant's (a+1)-th process.

[0095] In the above, each participant only updates the current model parameters they are responsible for based on their own encrypted global gradient value. Subsequently, it is necessary to obtain the updated model parameters of all participants in order to complete the global synchronization update of all model parameters, i.e., step S4.

[0096] This is based on a ring network architecture consisting of K participants. First, for any participant k, the updated model parameters responsible for participant k are encrypted using the public key of the next participant k+1, resulting in the updated encrypted model parameters [[W]. (k) ]] k+1 Then it is sent to participant k+1, who decrypts it to obtain the updated model parameters W that participant k is responsible for. (k) ;

[0097] Following this logic, participant k encrypts again to obtain [[W] (k) ]] k+2 The updated model parameters W are sent to participant k+2, who then decrypts them to obtain the updated model parameters W that participant k is responsible for. (k) Repeat this process K-1 times to finally complete the global update of all model parameters.

[0098] In an optional embodiment, after obtaining the updated encrypted model parameters of the K participants and decrypting them to obtain the updated model parameters, the graph federation training method further includes:

[0099] When the preset iteration stopping condition is met, the iteration stops, and the global model is updated according to the latest parameters of each model.

[0100] The iteration stopping condition includes any of the following:

[0101] The current iteration count is greater than the preset iteration count threshold;

[0102] The total change between the objective function of each local model in this iteration and the objective function of each local model in the previous iteration is less than the preset change value; wherein, each local model is trained by each participant using their own local data.

[0103] It is understandable that steps S1-S4 are repeated until the preset iteration stopping condition is met, at which point the iteration stops and the global model is updated based on the latest model parameters obtained after this iteration.

[0104] Optionally, the total change value is obtained by adding the second change values ​​between the objective function of each local model in the current iteration and the objective function of the corresponding local model in the previous iteration; wherein, the second change value is characterized by the difference, for example, the difference, the sum of squared errors, the absolute error and other values ​​related to the difference can all characterize the change between the objective function of each local model in the current iteration and the objective function of each local model in the previous iteration.

[0105] Specifically, the total change is calculated as follows:

[0106] Each participant calculates the objective function f of each local model in this iteration according to formula (3). (k,t) (W) and the objective function f of each local model in the previous iteration (k,t-1) (W); where k represents the participants and t represents the current iteration number;

[0107] Each participant calculates the objective function f (k,t) (W) and objective function f (k,t-1) The second change value between (W) is encrypted to obtain the second encrypted change value; for example, the sum of squared errors is used as the second change value, and the second change value calculated by the kth participant is Dist. k =||f (k,t) (W)-f (k,t-1) (W)|| 2 ;

[0108] Each participant summarizes and sums the values ​​of the second change to obtain the summation result. The summation result is decrypted to obtain the total change value Dist; alternatively, one of the participants summarizes all the second change values ​​and sums them to obtain the summation result, which is then decrypted to obtain the total change value. This total change value is then transmitted to the other participants. For example, each participant sends its calculated second change values ​​to the first participant, who sums them to obtain the total change value. The Dist is decrypted and sent back to all participating parties.

[0109] In an optional embodiment, updating the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters includes:

[0110] Based on the learning rate, the total number of samples, and the encrypted global gradient value, calculate the first encrypted change value of the current model parameters for which the participant is responsible;

[0111] Decrypt the first encrypted change value to obtain the first change value;

[0112] Based on the first change value, the current model parameters of the participating party are updated to obtain the updated model parameters.

[0113] For example, each participant calculates the first encrypted change value of the current model parameter they are responsible for according to formula (5):

[0114]

[0115] In the formula, This represents the first encryption change value, λ is the learning rate, and N is the total number of samples. This represents the encrypted global gradient value of the current model parameter ω calculated by the k-th participant.

[0116] Each participant decrypts its calculated first encrypted change value to obtain the first change value.

[0117] Each participant updates the parameters of the current model they are responsible for according to formula (6):

[0118]

[0119] Therefore, the current model parameter ω has been updated.

[0120] This application's embodiments can be applied to modeling and training scenarios based on complex relational network data where data cannot leave the database, such as anti-telecom fraud modeling scenarios involving multiple provincial branches of telecom operators. The data used mainly comprises three categories:

[0121] (1) User characteristic data

[0122] User characteristic data mainly consists of operational data and DPI analysis data from mobile services, including user technical profiles, communication information, roaming information, device information, and APP usage information.

[0123] (2) User Relationship Data

[0124] The relationship data mainly includes user call relationships, user SMS relationships, user device relationships, user location relationships, and so on.

[0125] (3) User tag data

[0126] User tag data is essentially user performance data, and its meaning varies depending on the business scenario. For example, in an anti-fraud scenario, 0 represents a normal user and 1 represents a fraudulent user; similarly, in credit scoring and "coupon hunter" scenarios, 0-1 can represent normal users and abnormal users, respectively.

[0127] In summary, the above data forms a network graph with three types of nodes: users, locations, and devices. Each type of node has its own characteristics, with users having the most features. Location features include latitude and longitude, while device features include device price, etc. Nodes are connected by relationships. Users are connected by call and SMS relationships, the relationship between a user and a location indicates whether a user is at a certain location at a certain time, and the relationship between a user and a device indicates whether a user has used a certain device.

[0128] Each institution, such as a bank, a telecom operator, or a provincial branch of a telecom operator, can train its own model to obtain a predictive model (i.e., a local model) targeting fraudulent users, provided it has the aforementioned data. However, due to insufficient data from a single provincial branch, it is often difficult to build a good model. To improve model accuracy, it is necessary to combine data from various provincial branches of the telecom operator for modeling. Specifically, each provincial branch of the telecom operator deploys participating parties, and the feature, relationship, and tag data of users within their respective provincial branches are stored in the storage layer of each participating party. Through the embodiments of this application, each participating party can obtain the final local model and deploy it independently.

[0129] See Figure 3 , Figure 3 This is a structural block diagram of a graph federation training device 10 provided in an embodiment of this application. The graph federation training device 10 is applied to any one of the K participants in the graph federation training, and the participant possesses at least the public keys generated by each of the K participants based on a homomorphic encryption algorithm. The graph federation training device 10 includes:

[0130] Training module 11 is used to obtain the current model parameters of each of the K participants in this iteration, train the local data, and obtain the gradient values ​​of each current model parameter; wherein, the current global model is determined by the current model parameters of each of the K participants.

[0131] The encrypted transmission module 12 is used to encrypt the gradient values ​​of the current model parameters under the responsibility of the a-th participant using the public key of the a-th participant, to obtain the encrypted gradient values ​​of each current model parameter; and to obtain the encrypted global gradient values ​​of the current model parameters under the responsibility of the participant; wherein the encrypted global gradient values ​​are composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant, which are obtained by training and encrypting with K participants, and a = 1, 2, ..., K;

[0132] Update module 13 is used to update the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters; and to encrypt the updated model parameters using the participant's public key to obtain the updated encrypted model parameters.

[0133] The decryption module 14 is used to obtain the updated encrypted model parameters of K participants and decrypt them to obtain the updated model parameters.

[0134] Optionally, the K participants are connected end-to-end to form a ring network architecture.

[0135] Optionally, the participant is the kth participant, and the encrypted transmission module 12 is specifically used for:

[0136] In the first encrypted gradient transmission process, the k-th participant uses the public key of the (k-1)-th participant to encrypt the gradient value of the current model parameter that the (k-1)-th participant is responsible for, and obtains the encrypted gradient value, which is used as the transmission value of the first encrypted gradient transmission process and transmitted to the (k+1)-th participant.

[0137] In the b-th encrypted gradient transmission process, the k-th participant uses the public key of the kb-th participant to encrypt the gradient value of the current model parameter that the kb-th participant is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)-th participant in the (b-1)-th encrypted gradient transmission process to obtain the transmission value of the b-th encrypted gradient transmission process, and transmits it to the (k+1)-th participant; where b = 2, 3, ..., K-1;

[0138] During the Kth encrypted gradient transmission process, the kth participant uses its public key to encrypt the gradient value of the current model parameter that it is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)th participant in the K-1th encrypted gradient transmission process to obtain the encrypted global gradient value of the current model parameter that the kth participant is responsible for.

[0139] Optionally, the graph federation training device 10 further includes:

[0140] The iteration stopping module is used to stop the iteration when the preset iteration stopping condition is met, and update the global model according to the latest parameters of each model.

[0141] The iteration stopping condition includes any of the following:

[0142] The current iteration count is greater than the preset iteration count threshold;

[0143] The total change between the objective function of each local model in this iteration and the objective function of each local model in the previous iteration is less than the preset change value; wherein, each local model is trained by each participant using their own local data.

[0144] Optionally, the update module 13 is specifically used for:

[0145] Based on the learning rate, the total number of samples, and the encrypted global gradient value, calculate the first encrypted change value of the current model parameters for which the participant is responsible;

[0146] Decrypt the first encrypted change value to obtain the first change value;

[0147] Based on the first change value, the current model parameters of the participating party are updated to obtain the updated model parameters.

[0148] Optionally, if it is the first iteration, the initial model parameters that the participant is responsible for are used as the current model parameters that the participant is responsible for.

[0149] If it is not the first iteration, the model parameters that the participant is responsible for, obtained after the previous iteration update, will be used as the current model parameters that the participant is responsible for.

[0150] It is worth noting that the working process of each module in the graph federation training device 10 described in this application embodiment can refer to the working process of the graph federation training method described in the above embodiment, and will not be repeated here.

[0151] The graph federation training device 10 provided in this application embodiment allows participating parties to obtain public keys generated by K participating parties based on homomorphic encryption algorithms. Using the public key of the a-th participating party, they encrypt the gradient value of the current model parameters under the responsibility of that participating party, thereby obtaining the encrypted global gradient value of the current model parameters under that participating party's responsibility. After updating the current model parameters under their responsibility using this encrypted global gradient value, the updated model parameters are encrypted again, thus obtaining the updated encrypted model parameters of the K participating parties. These are then decrypted to obtain the updated model parameters. This ensures that model parameters are not released in plaintext during the entire graph federation training process, preventing attackers from using model parameters to infer sample data when there are many model parameters, thereby avoiding the risk of data leakage. This achieves data usability without visibility, improving data security. Furthermore, each participating party only updates the model parameters under its responsibility, resulting in low computational load.

[0152] Furthermore, this application also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the graph federation training method as described in any of the above embodiments.

[0153] Furthermore, this application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements the graph federation training method as described in any of the above embodiments.

[0154] See Figure 4 , Figure 4 This is a structural block diagram of a graph federation training device 20 provided in an embodiment of this application. The graph federation training device 20 includes: a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above-described graph federation training method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above-described device embodiments.

[0155] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the graph federation training device 20.

[0156] The graph federation training device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the graph federation training device 20 and does not constitute a limitation on the graph federation training device 20. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the graph federation training device 20 may also include input / output devices, network access devices, buses, etc.

[0157] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the graph federation training device 20, connecting all parts of the graph federation training device 20 via various interfaces and lines.

[0158] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the graph federation training device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0159] The modules / units integrated into the federation training device 20, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0160] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0161] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A graph federation training method, characterized in that, The graph federation training method is applied to any of the K participants in the graph federation training, where the participant possesses at least the public keys generated by the K participants based on homomorphic encryption algorithms. The graph federation training method includes: In this iteration, the current model parameters of each of the K participants are obtained, and the local data is used to train the model to obtain the gradient values ​​of each current model parameter; the current global model is determined by the current model parameters of each of the K participants. The gradient values ​​of the current model parameters under the responsibility of the a-th participant are encrypted using the public key of the a-th participant to obtain the encrypted gradient values ​​of each current model parameter; the encrypted global gradient value of the current model parameters under the responsibility of the participant is obtained; wherein the encrypted global gradient value is composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant obtained by training and encrypting with K participants, a = 1, 2, ..., K; The current model parameters of the participant are updated based on the encrypted global gradient value to obtain the updated model parameters; the updated model parameters are then encrypted using the participant's public key to obtain the updated encrypted model parameters. Obtain the updated encrypted model parameters of K participants, and decrypt them to obtain the updated model parameters.

2. The graph federation training method as described in claim 1, characterized in that, The K participants are connected end-to-end to form a ring network architecture.

3. The graph federation training method as described in claim 2, characterized in that, The participant is the kth participant. The gradient values ​​of the current model parameters under the responsibility of the i-th participant are encrypted using the public key of the i-th participant to obtain the encrypted gradient values ​​of each current model parameter. Obtaining the encrypted global gradient value of the current model parameters for which the participating party is responsible includes: In the first encrypted gradient transmission process, the k-th participant uses the public key of the (k-1)-th participant to encrypt the gradient value of the current model parameter that the (k-1)-th participant is responsible for, and obtains the encrypted gradient value, which is used as the transmission value of the first encrypted gradient transmission process and transmitted to the (k+1)-th participant. In the b-th encrypted gradient transmission process, the k-th participant uses the public key of the kb-th participant to encrypt the gradient value of the current model parameter that the kb-th participant is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)-th participant in the (b-1)-th encrypted gradient transmission process to obtain the transmission value of the b-th encrypted gradient transmission process, and transmits it to the (k+1)-th participant; where b = 2, 3, ..., K-1; During the Kth encrypted gradient transmission process, the kth participant uses its public key to encrypt the gradient value of the current model parameter that it is responsible for, and adds the encrypted gradient value to the transmission value obtained by the (k-1)th participant in the K-1th encrypted gradient transmission process to obtain the encrypted global gradient value of the current model parameter that the kth participant is responsible for.

4. The graph federation training method as described in claim 1, characterized in that, After obtaining the updated encrypted model parameters of the K participants and decrypting them to obtain the updated model parameters, the graph federation training method further includes: When the preset iteration stopping condition is met, the iteration stops, and the global model is updated according to the latest parameters of each model. The iteration stopping condition includes any of the following: The current iteration count is greater than the preset iteration count threshold; The total change between the objective function of each local model in this iteration and the objective function of each local model in the previous iteration is less than the preset change value; wherein, each local model is trained by each participant using their own local data.

5. The graph federation training method as described in claim 1, characterized in that, The step of updating the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters includes: Based on the learning rate, the total number of samples, and the encrypted global gradient value, calculate the first encrypted change value of the current model parameters for which the participant is responsible; Decrypt the first encrypted change value to obtain the first change value; Based on the first change value, the current model parameters of the participating party are updated to obtain the updated model parameters.

6. The graph federation training method as described in claim 1, characterized in that, If it is the first iteration, the initial model parameters that the participant is responsible for will be used as the current model parameters that the participant is responsible for. If it is not the first iteration, the model parameters that the participant is responsible for, obtained after the previous iteration update, will be used as the current model parameters that the participant is responsible for.

7. A graph federation training device, characterized in that, The graph federation training apparatus is applied to any one of the K participants in graph federation training, wherein the participant possesses at least the public keys generated by each of the K participants based on a homomorphic encryption algorithm. The graph federation training apparatus includes: The training module is used in this iteration to obtain the current model parameters of each of the K participants, train the model on the local data, and obtain the gradient values ​​of each current model parameter; the current global model is determined by the current model parameters of each of the K participants. An encrypted transmission module is used to encrypt the gradient values ​​of the current model parameters under the responsibility of the a-th participant using the public key of the a-th participant, thereby obtaining encrypted gradient values ​​of each current model parameter; and to obtain the encrypted global gradient value of the current model parameters under the responsibility of the participant; wherein the encrypted global gradient value is composed of the encrypted gradient values ​​of the current model parameters under the responsibility of the participant, obtained by training and encrypting with K participants, where a = 1, 2, ..., K; The update module is used to update the current model parameters of the participant based on the encrypted global gradient value to obtain the updated model parameters; and to encrypt the updated model parameters using the participant's public key to obtain the updated encrypted model parameters. The decryption module is used to obtain the updated encrypted model parameters of K participants and decrypt them to obtain the updated model parameters.

8. A graph federation training device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the graph federation training method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the graph federation training method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, It includes a computer program / instruction that, when executed by a processor, implements the graph federation training method as described in any one of claims 1 to 6.

Images