A federated learning method, device, storage medium and electronic equipment

Abstract

The specification provides a federated learning method, device, storage medium and electronic equipment. In the federated learning method provided in the specification, a target node shared with a second participant is determined, and a corresponding label is determined; the data corresponding to the target node is input into a to-be-trained graph network model, and a first feature of the target node is extracted through a feature extraction layer; the first feature is encrypted using a public key of the second participant to obtain a first encrypted feature; a second encrypted feature sent by the second participant is received; the first encrypted feature and the second encrypted feature are fused to obtain a fused encrypted feature; the fused encrypted feature is input into a subsequent processing layer of the to-be-trained graph network model to obtain an encrypted output result output by the model; the encrypted output result is sent to the second participant, so that the second participant decrypts the encrypted output result using a private key to obtain a decrypted result; the decrypted result sent by the second participant is received, and the model is trained using the decrypted result.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method, apparatus, storage medium, and electronic device for federated learning. Background Technology

[0002] Currently, federated learning is often used to train models by expanding the training samples available for them. The main purpose of federated learning is to jointly train the training samples of both parties to improve the performance of the training model. To protect privacy and avoid disclosing the original data of both parties, a vertical federated learning approach is usually used for training.

[0003] However, the current vertical federated learning method still has the risk of data leakage. Therefore, how to ensure the security of the original data during the vertical federated learning process is an urgent problem to be solved. Summary of the Invention

[0004] This specification provides a federated learning method and a federated learning apparatus to partially solve the aforementioned problems existing in the prior art.

[0005] The following technical solution is adopted in this specification:

[0006] This specification provides a method for federated learning, applied to a first participant, including:

[0007] The first participant determines the user nodes shared with the second participant from its own stored graph data as target nodes, and determines the label of the target nodes based on the historical business execution results corresponding to the target nodes;

[0008] The data corresponding to the target node is input into the feature extraction layer of the graph network model to be trained, and the first feature output by the feature extraction layer is obtained.

[0009] Using a homomorphic encryption algorithm, the first feature is encrypted using the public key of the second participant to determine the first encrypted feature, and the second encrypted feature sent by the second participant is received. The second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data.

[0010] Based on the first encryption feature and the second encryption feature, a fusion feature is determined;

[0011] The fused features are input into the subsequent processing layer of the graph network model to be trained to obtain the encrypted output result of the graph network model to be trained.

[0012] The encrypted output result is sent to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key;

[0013] The decryption result returned by the second participant is received, and the graph network model to be trained is trained based on the decryption result and the annotation.

[0014] Optionally, before sending the encrypted output result to the second participant, the method further includes:

[0015] The preset perturbation feature is encrypted using the public key to obtain the encrypted perturbation feature;

[0016] The encryption output result is scrambled using the aforementioned encryption perturbation feature;

[0017] The training of the graph network model to be trained is performed based on the decryption result and the annotation, specifically including:

[0018] Based on the perturbation characteristics, the interference added to the decryption result is eliminated;

[0019] The graph network model to be trained is trained using the decryption results after interference removal and the annotations.

[0020] Optionally, the second feature is obtained by the second participant adjusting the dimension of the second intermediate feature to a specified dimension using the second parameter, and the second intermediate feature is determined by the second participant based on the target node in its stored graph data;

[0021] Before encrypting the first feature using the public key of the second participant, the method further includes:

[0022] The dimension of the first feature is adjusted to the specified dimension using the first parameter.

[0023] Optionally, the graph network model to be trained is trained based on the decryption result and the annotation, specifically including:

[0024] The parameters to be trained and the first parameter are adjusted with the goal of minimizing the difference between the decryption result and the annotation.

[0025] This specification provides a method for federated learning applied to a second participant, including:

[0026] The second participant determines the user nodes shared with the first participant from its own stored graph data, and uses them as target nodes.

[0027] Extract the second feature of the data corresponding to the target node;

[0028] The second encrypted feature is determined by encrypting the second feature using the public key of the second participant through a homomorphic encryption algorithm.

[0029] The second encryption feature is sent to the first participant, so that a fusion feature is determined based on the first encryption feature and the second encryption feature. The first encryption feature is obtained by the first participant encrypting the first feature using the public key. The first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0030] Receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant by inputting the fused features into the subsequent processing layer of the graph network model to be trained;

[0031] The encrypted output is decrypted using the private key of the second participant;

[0032] The decryption result is returned to the first participant, enabling the first participant to train the graph network model to be trained based on the decryption result and the annotations. The annotations are determined by the first participant based on its own historical business execution results corresponding to the target node.

[0033] Optionally, the encrypted output result is obtained by the first participant scrambling the intermediate result output by the first participant's graph network model to be trained using encrypted perturbation features. The intermediate result is determined by the first participant by inputting the fusion feature into the subsequent processing layer of the graph network model to be trained.

[0034] Optionally, the first feature is obtained by the first participant adjusting the dimension of the first intermediate feature to a specified dimension using the first parameter. The first intermediate feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0035] Before encrypting the second feature using the public key of the second participant, the method further includes:

[0036] The second parameter is used to adjust the dimension of the second feature to the specified dimension.

[0037] Optionally, the decryption result is returned to the first participant, enabling the first participant to train the graph network model to be trained using the decryption result and annotations, specifically including:

[0038] The decryption result is returned to the first participant, so that the first participant can adjust the parameters of the graph network model to be trained and the first parameter with the optimization goal of minimizing the difference between the decryption result and the annotation.

[0039] This specification provides a federated learning apparatus, comprising:

[0040] The first determining module is used for the first participant to determine, from its own stored graph data, a user node shared with the second participant as a target node, and to determine the label of the target node based on the historical business execution results corresponding to the target node;

[0041] The first extraction module is used to input the data corresponding to the target node into the feature extraction layer of the graph network model to be trained, and obtain the first feature output by the feature extraction layer.

[0042] The first encryption module is used to encrypt the first feature using a homomorphic encryption algorithm based on the public key of the second participant, determine the first encrypted feature, and receive the second encrypted feature sent by the second participant, wherein the second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data.

[0043] The fusion module is used to determine the fusion feature based on the first encryption feature and the second encryption feature;

[0044] The output module is used to input the fused features into the subsequent processing layer of the graph network model to be trained, and obtain the encrypted output result of the graph network model to be trained.

[0045] The sending module is used to send the encrypted output result to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key;

[0046] The training module is used to receive the decryption result returned by the second participant, and to train the graph network model to be trained based on the decryption result and the annotation.

[0047] This specification provides a federated learning apparatus, comprising:

[0048] The second determining module is used by the second participant to determine, from its own stored graph data, the user nodes shared with the first participant as target nodes;

[0049] The second extraction module is used to extract the second feature of the data corresponding to the target node;

[0050] The second encryption module is used to encrypt the second feature using a homomorphic encryption algorithm based on the public key of the second participant, and to determine the second encryption feature;

[0051] The feature sending module is used to send the second encrypted feature to the first participant, so that a fusion feature is determined based on the first encrypted feature and the second encrypted feature. The first encrypted feature is obtained by the first participant encrypting the first feature using the public key. The first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0052] The second receiving module is used to receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant by inputting the fused features into the subsequent processing layer of the graph network model to be trained;

[0053] The decryption module is used to decrypt the encrypted output using the private key of the second participant;

[0054] The return module is used to return the decryption result to the first participant, so that the first participant can train the graph network model to be trained based on the decryption result and the annotation. The annotation is determined by the first participant based on its own historical business execution results corresponding to the target node.

[0055] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described federated learning method.

[0056] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described federated learning method.

[0057] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0058] In the federated learning method provided in this specification, a target node shared with the second participant is identified, and its corresponding label is determined. The data corresponding to the target node is input into the graph network model to be trained, and the first feature of the target node is extracted through the feature extraction layer. The first feature is encrypted using the public key of the second participant to obtain the first encrypted feature. The second encrypted feature sent by the second participant is received. The first encrypted feature and the second encrypted feature are fused to obtain the fused encrypted feature. The fused encrypted feature is input into the subsequent processing layer of the graph network model to be trained to obtain the encrypted output result of the model. The encrypted output result is sent to the second participant, so that the second participant can decrypt the encrypted output result using its private key to obtain the decrypted result. The decrypted result sent by the second participant is received, and the model is trained using the decrypted result. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0060] Figure 1 This is a flowchart illustrating one of the federated learning methods provided in this specification;

[0061] Figure 2 This specification provides a detailed process diagram of a federated learning method.

[0062] Figure 3 This is a flowchart illustrating one of the federated learning methods provided in this specification;

[0063] Figure 4 A schematic diagram of a federated learning device provided in this specification;

[0064] Figure 5 A schematic diagram of a federated learning device provided in this specification;

[0065] Figure 6 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0066] Typically, vertical federated learning is a process requiring collaboration between two parties. These parties can be categorized as the trainer and the feature provider. The traditional process involves the feature provider supplying features from their raw data to the trainer, who then trains their model using these features, achieving joint training without the data leaving the domain. This only describes the process of one party training the model. It's conceivable that by simply reversing the roles of the two parties, the trainer can assist the other party in training their model.

[0067] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0068] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0069] Figure 1 This document provides a flowchart illustrating a method for federated learning applied to a first participant, comprising the following steps:

[0070] S100: The first participant determines the user node shared with the second participant from its own stored graph data as the target node, and determines the label of the target node based on the historical business execution results corresponding to the target node.

[0071] In this specification, the executing entity used to implement the federated learning method can refer to a designated device such as a server set up on the business platform. For ease of description, this specification will only use the server as the executing entity as an example to illustrate one type of federated learning method provided in this specification.

[0072] In this specification, both the first participant and the second participant are servers or other designated devices. The first participant is the one performing the training, and the second participant is the one providing feature data. That is, in the embodiments of the federated learning method provided in this specification, the second participant provides features to the first participant, and the first participant uses the features provided by the second participant to train the model that needs to be trained.

[0073] In practical applications, the specific forms of samples and sample data can be varied. In the federated learning method provided in this specification, sample data is stored in the form of graph data between the first and second participants. In this case, each sample can be represented by each node on the graph, and the information of the sample can be represented by the edges on the graph. It is worth mentioning that, in addition to graph data, training samples can also be stored in the two parties participating in federated learning in other data forms; this specification does not impose specific restrictions on this.

[0074] The flowchart of the federated learning method provided in this manual is as follows: Figure 2 As shown. When performing federated learning, the first step is to identify the user nodes shared by the first and second participants—that is, the training samples that can participate in the training—as the target nodes. The purpose of federated learning is to further train the model using feature data provided by the other party, in addition to the training party using its own data. Therefore, the training samples used for training should be the target nodes shared by both parties.

[0075] It's important to note that having some shared samples does not mean both parties possess identical sample data. For example, suppose the first participant provides users with services A and B, while the second participant provides users with services C and D. When users are considered nodes in the graph data, users who have used both the services provided by the first and second participants can be considered shared user nodes—the target nodes. However, for the target nodes, the data possessed by the first participant consists of data from when users performed services A and B, while the data possessed by the second participant consists of data from when users performed services C and D. Clearly, the sample data possessed by the two parties are different.

[0076] Furthermore, since the first participant in the embodiments of this specification is the one training the model, after determining the training samples, the first participant also needs to determine the labels corresponding to the training samples. Typically, the labels can be determined based on the historical business execution results corresponding to the target node. The corresponding historical business can also be different when training different models. There are various methods for determining the labels, such as manual label determination, and this specification does not impose specific limitations on this.

[0077] S102: Input the data corresponding to the target node into the feature extraction layer of the graph network model to be trained, and obtain the first feature output by the feature extraction layer.

[0078] After the target node is determined in step S100, the data corresponding to the target node can be input into the graph network model to be trained in this step. After the data corresponding to the target node is input, the data first enters the feature extraction layer in the graph network model, where the first feature of the data is extracted. The graph network model to be trained is a model that needs to be trained and pre-stored in the first participant.

[0079] S104: Using a homomorphic encryption algorithm, encrypt the first feature according to the public key of the second participant to determine the first encrypted feature, and receive the second encrypted feature sent by the second participant, wherein the second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data.

[0080] In the federated learning method provided in this specification, all encryption methods used are homomorphic. In homomorphic encryption, processing homomorphically encrypted data yields an output; decrypting this output results in the same output as processing the unencrypted original data using the same method. For example, suppose processing data 'a' yields data 'a'; in the case of homomorphic encryption, first encrypting data 'a' yields [[a]], then performing the same specific processing on the encrypted data [[a]] yields data [[a]]'. Decrypting data [[a]]' yields the same data as directly processing data 'a', both being 'a'. The specific processing can consist of operations consisting of addition and / or multiplication.

[0081] Those skilled in the art will understand that asymmetric encryption methods exist in data encryption technology. In asymmetric encryption, a public key is used for encryption, and a private key is used for decryption. In the federated learning method provided in this specification, both the first and second participants possess the second participant's public key, which can be used to encrypt data; while the second participant's private key is possessed only by the second participant and can be used to decrypt data.

[0082] In this step, the first feature extracted in step S102 can be encrypted using the public key of the second participant to obtain the first encrypted feature, which can be used in subsequent steps.

[0083] Simultaneously, since federated learning requires the combined feature data of both parties for training, this step also allows the receipt of a second encrypted feature sent by the second parameter party. Similar to the first encrypted feature, the second encrypted feature is obtained by the second participant extracting the second feature of the target node and encrypting it using a public key. Both the first and second participants use the same public key when encrypting data; this public key corresponds to the private key possessed by the second participant. That is, only the private key possessed by the second participant can decrypt data encrypted by the public keys of both the first and second participants.

[0084] In this scenario, because the data is encrypted using a public key, only the second participant can decrypt it, ensuring that the first participant cannot reverse engineer the second encryption feature sent by the second participant and thus deduce the second participant's original data.

[0085] S106: Determine the fusion feature based on the first encryption feature and the second encryption feature.

[0086] Typically, to improve model performance, the second and second encrypted features are fused together before being input into subsequent processing layers of the graphical network model to obtain a fused encrypted feature. There are various ways to fuse the first and second encrypted features, such as concatenation, and this specification does not impose specific limitations on this method.

[0087] S108: Input the fused features into the subsequent processing layer of the graph network model to be trained to obtain the encrypted output result of the graph network model to be trained.

[0088] After determining the fused encryption features in step S106, these features can be input into the subsequent processing layer of the graph network model to be trained in this step to calculate the fused encryption features and output the calculated results. Since both the first and second encryption features are homomorphically encrypted encrypted data, the fused encryption features obtained after fusion are also encrypted data. Therefore, the output result after calculation by the graph network model to be trained is also an encrypted output result.

[0089] S110: Send the encrypted output result to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key.

[0090] Since the output of the graph network model to be trained in step S108 is an encrypted output in a closed state, it cannot be directly used for training. Furthermore, only the second participant has the corresponding private key and can decrypt the encrypted output. Therefore, the encrypted output needs to be sent to the second participant so that the second participant can use the private key to decrypt the encrypted output and obtain the decrypted result.

[0091] S112: Receive the decryption result returned by the second participant, and train the graph network model to be trained based on the decryption result and the annotation.

[0092] Once the second participant has decrypted the encrypted output and obtained the decrypted result, it can return the decrypted result to the first participant. Under homomorphic encryption, the decrypted result is the same as the result obtained by directly inputting the unencrypted first and second features into the graph network model to be trained. Therefore, the decrypted result can be directly used to train the graph network model.

[0093] During training, the gradient of the feature can be calculated first based on the decryption result. Then, the gradient of the feature is input into the graph network model to be trained to obtain the gradient of the graph network model parameters. The obtained gradient of the graph network model parameters is then used to update the parameters in the graph network model to be trained.

[0094] Specifically, the gradient of a feature can be determined using the following formula:

[0095]

[0096] Where A represents the first participant, S represents the total number of target nodes, and i represents the i-th target node. This represents the gradient of the feature of the i-th node. This represents the result calculated by the graph network model to be trained based on the second feature of the i-th node. Let y represent the first feature of the i-th node. i θ represents the label of the i-th node. A This indicates the first parameter.

[0097] When using the federated learning method provided in this specification, the second participant can encrypt the extracted second feature using a public key and send it to the first participant. After the first participant obtains the encrypted output result through the graph network model to be trained, it decrypts the encrypted output result using the corresponding private key and returns it to the first participant, who then trains the model. This effectively prevents the first participant from inferring the second participant's original data based on the second participant's feature data, thus protecting the second participant's original data.

[0098] Additionally, the above embodiments only consider the scenario where the first participant might steal the second participant's original data, and protection is provided for the second participant. However, in practical applications, in addition to the original data, the graph network model to be trained usually also needs protection. If the first participant sends the encrypted output of the graph network model to the second participant, and the second participant uses its private key to decrypt it, the second participant could easily attack the first participant's graph network model based on the decryption result.

[0099] Therefore, to further protect the model of the first participant, the encrypted output result can be scrambled when the first participant sends it to the second participant. Specifically, the public key can be used to encrypt a preset perturbation feature to obtain an encrypted perturbation feature; this encrypted perturbation feature is then used to scramble the encrypted output result. Thus, the encrypted output result obtained by the second participant contains an additional encrypted perturbation. When the second participant decrypts the encrypted output result, the decryption result also contains the perturbation feature. Since the second participant is unaware of how the first participant scrambled the output, it cannot identify the perturbation in the scrambled decryption result, and therefore cannot know the decryption result without the perturbation, thus preventing it from attacking the model of the first participant.

[0100] As can be imagined, in the above scenario, the decryption result returned by the second participant to the first participant will contain perturbations. Before using the decryption result to train the model, the first participant needs to eliminate the perturbations. Specifically, based on the perturbation characteristics, the interference added to the decryption result can be eliminated; the decryption result after eliminating the perturbations, along with the annotations, can then be used to train the graph network model to be trained.

[0101] In practical applications, since the first and second participants may use completely different extraction networks when extracting features from the target node, the dimensions of the extracted first and second features may also be different. In this case, to better integrate the first and second encrypted features, it's advisable to process them to the same dimension before encryption and fusion. Specifically, a first parameter can be used to adjust the dimension of the first feature to a specified dimension; and the second feature is obtained by the second participant adjusting the dimension of a second intermediate feature to a specified dimension using a second parameter. This second intermediate feature is determined by the second participant based on the target nodes in its stored graph data.

[0102] Because homomorphic encryption is used and the same public key is used during encryption, after adjusting the first and second features to the same specified dimension, and then encrypting them separately, the resulting first and second encrypted features will still be in the same dimension, which is more conducive to feature fusion.

[0103] In the federated learning method provided in this specification, although the first participant only trains the graph network model to be trained, more preferably, if the method described above for adjusting the first feature and the second feature to the same specified dimension is adopted, then the first parameter used to adjust the first feature can be additionally updated during training. Specifically, the optimization objective can be to minimize the difference between the decryption result and the annotation, and the parameters of the graph network model to be trained and the first parameter can be adjusted. The gradient used to adjust the first parameter can be determined according to the following formula:

[0104]

[0105] Where A represents the first participant, S represents the total number of target nodes, and i represents the i-th target node. S represents the gradient of the first parameter. A This represents the sum of the results calculated by the graph network model to be trained based on the second feature of each target node. This represents the first feature of the i-th node. Let y represent the first feature of the i-th node after adjustment by the first parameter. i This represents the label of the i-th node. express. Figure 3 This document provides a flowchart illustrating a method for federated learning applied to a second participant, comprising the following steps:

[0106] S200: The second participant determines the user node shared with the first participant from its own stored graph data as the target node.

[0107] It should be noted that the federated learning method for the second participant provided in this specification corresponds to the federated learning method for the first participant provided in this specification; they are essentially the same method executed by different participants. Therefore, to make the specification more concise, the steps already explained in detail in the federated learning method for the first participant will not be repeated in the following description of the federated learning method for the second participant.

[0108] Similar to step S100, in this step, the second participant, as the provider of feature data, also needs to first identify the user nodes shared with the first participant, which will serve as the target nodes for training the graph network model to be trained by the first participant.

[0109] S202: Extract the second feature of the data corresponding to the target node.

[0110] Once the target node is identified, the second participant will extract the second feature of the target node for use in subsequent steps.

[0111] S204: The second encrypted feature is determined by encrypting the second feature using a homomorphic encryption algorithm based on the public key of the second participant.

[0112] In this step, the second participant will use homomorphic encryption to encrypt the second feature using its public key, thus obtaining the second encrypted feature.

[0113] S206: Send the second encryption feature to the first participant, so that a fusion feature is determined based on the first encryption feature and the second encryption feature, wherein the first encryption feature is obtained by the first participant encrypting the first feature using the public key, and the first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0114] After the second feature is encrypted, the obtained second encrypted feature can be sent to the first participant, allowing the first participant to merge the first encrypted feature with the second encrypted feature. The first encrypted feature is obtained by the first participant extracting the first feature of the target node using the graph network to be trained, and then encrypting the first feature using the public key of the second participant. The public key used by the first participant is the same as the public key used by the second participant when encrypting the second feature.

[0115] S208: Receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant when inputting the fused features into the subsequent processing layer of the graph network model to be trained.

[0116] When the first participant fuses the first and second encryption features to obtain the fused encryption feature, it can input the fused encryption feature into the subsequent processing layer of the graph network model to be trained. The subsequent processing layer calculates the fused encryption feature to obtain the encryption output result. Since the first participant does not possess the private key used for decryption, it can send the encryption output result to the second participant.

[0117] S210: Decrypt the encrypted output using the private key of the second participant.

[0118] The second participant uses its private key to decrypt the encrypted output, obtaining the decrypted result. In the federated learning method provided in this specification, both the first and second participants possess public keys capable of encrypting data, but only the second participant possesses a private key capable of decrypting the encrypted data. Therefore, only the second participant can decrypt the encrypted output.

[0119] S212: Return the decryption result to the first participant, so that the first participant can train the graph network model to be trained based on the decryption result and the annotation, wherein the annotation is determined by the first participant based on its own historical business execution results corresponding to the target node.

[0120] In this step, the second participant sends the decryption result back to the first participant, allowing the first participant to use the received decryption result to train the graph network model to be trained.

[0121] Corresponding to the additional embodiments given in the federated learning method applied to the first participant provided in this specification, corresponding operations also need to be performed in the second participant.

[0122] More preferably, to protect the model of the first participant, the first participant can scramble the fused encryption features before sending them to the second participant. Specifically, the encrypted output result is obtained by the first participant scrambling the intermediate result output of the first participant's graph network model to be trained using encryption perturbation features. The intermediate result is determined by the first participant inputting the fused features into the subsequent processing layer of the graph network model to be trained.

[0123] Better still, before encryption, the first and second features can be adjusted to a specified dimension, so that the first and second features, after the same encryption, can be better fused in the same dimension. Specifically, the first feature is obtained by the first participant adjusting the dimension of the first intermediate feature to a specified dimension using a first parameter. The first intermediate feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained; the second feature is adjusted to a specified dimension using a second parameter.

[0124] More preferably, the first participant can not only train the graph network model to be trained, but also train the first extraction model for extracting the first feature, and train the first parameter used to adjust the dimension of the first feature. Specifically, the decryption result can be returned to the first participant, so that the first participant can adjust the parameters of the graph network model to be trained and the first parameter with the optimization objective of minimizing the difference between the decryption result and the annotation.

[0125] It is worth mentioning that the embodiments provided in this specification are only illustrated by the example of the second participant providing feature data to help the first participant train. It can be imagined that as long as the roles of the first participant and the second participant are interchanged in the method, the training of the other party can be achieved.

[0126] The above describes one or more methods for implementing federated learning as outlined in this specification. Based on the same approach, this specification also provides corresponding federated learning apparatus, such as... Figure 4 , Figure 5 As shown.

[0127] Figure 4 A schematic diagram of a federated learning device provided in this specification includes:

[0128] The first determining module 300 is used for the first participant to determine, from its own stored graph data, a user node shared with the second participant as a target node, and to determine the label of the target node based on the historical business execution results corresponding to the target node;

[0129] The first extraction module 302 is used to input the data corresponding to the target node into the feature extraction layer of the graph network model to be trained, and obtain the first feature output by the feature extraction layer.

[0130] The first encryption module 304 is used to encrypt the first feature using a homomorphic encryption algorithm based on the public key of the second participant to determine the first encrypted feature, and to receive the second encrypted feature sent by the second participant, wherein the second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data.

[0131] The fusion module 306 is used to determine the fusion feature based on the first encryption feature and the second encryption feature;

[0132] The output module 308 is used to input the fused features into the subsequent processing layer of the graph network model to be trained, and obtain the encrypted output result of the graph network model to be trained.

[0133] The sending module 310 is used to send the encrypted output result to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key;

[0134] The training module 312 is used to receive the decryption result returned by the second participant, and to train the graph network model to be trained based on the decryption result and the annotation.

[0135] Optionally, the device further includes a scrambling module 314, specifically used to encrypt a preset perturbation feature using the public key to obtain an encrypted perturbation feature; and to scramble the encrypted output result using the encrypted perturbation feature.

[0136] The training module 312 is specifically used to eliminate the interference added to the decryption result based on the perturbation features; and to train the graph network model to be trained using the decryption result after interference elimination and the annotation.

[0137] Optionally, the second feature is obtained by the second participant adjusting the dimension of the second intermediate feature to a specified dimension using the second parameter, and the second intermediate feature is determined by the second participant based on the target node in its stored graph data;

[0138] The device further includes a first adjustment module 316, specifically used to adjust the dimension of the first feature to a specified dimension using a first parameter.

[0139] Optionally, the training module 312 is specifically used to adjust the parameters to be trained and the first parameter with the goal of minimizing the difference between the decryption result and the annotation.

[0140] Figure 5 A schematic diagram of a federated learning device provided in this specification includes:

[0141] The second determining module 400 is used for the second participant to determine, from its own stored graph data, the user nodes shared with the first participant as target nodes;

[0142] The second extraction module 402 is used to extract the second feature of the data corresponding to the target node;

[0143] The second encryption module 404 is used to encrypt the second feature using a homomorphic encryption algorithm based on the public key of the second participant, and to determine the second encryption feature;

[0144] The feature sending module 406 is used to send the second encrypted feature to the first participant, so that a fusion feature is determined based on the first encrypted feature and the second encrypted feature. The first encrypted feature is obtained by the first participant encrypting the first feature using the public key. The first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0145] The second receiving module 408 is used to receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant by inputting the fusion feature into the subsequent processing layer of the graph network model to be trained;

[0146] The decryption module 410 is used to decrypt the encrypted output result using the private key of the second participant;

[0147] The return module 412 is used to return the decryption result to the first participant, so that the first participant can train the graph network model to be trained based on the decryption result and the annotation. The annotation is determined by the first participant based on its own historical business execution results corresponding to the target node.

[0148] Optionally, the encrypted output result is obtained by the first participant scrambling the intermediate result output by the first participant's graph network model to be trained using encrypted perturbation features. The intermediate result is determined by the first participant by inputting the fusion feature into the subsequent processing layer of the graph network model to be trained.

[0149] Optionally, the first feature is obtained by the first participant adjusting the dimension of the first intermediate feature to a specified dimension using the first parameter. The first intermediate feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained.

[0150] The device further includes a second adjustment module 414, specifically used to adjust the dimension of the second feature to a specified dimension using a second parameter.

[0151] Optionally, the return module 412 is specifically used to return the decryption result to the first participant, so that the first participant can adjust the parameters of the graph network model to be trained and the first parameter with the goal of minimizing the difference between the decryption result and the annotation.

[0152] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 It provides a method for federated learning.

[0153] This instruction manual also provides Figure 6 The one shown corresponds to Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1The method of federated learning described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0154] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0155] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0156] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0157] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0158] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0159] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0160] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0161] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0162] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0163] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0164] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0165] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0166] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0168] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0169] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A federated learning method, said method being applied to a first participant, comprising: The first participant determines the user nodes shared with the second participant from its own stored graph data as target nodes, and determines the label of the target nodes based on the historical business execution results corresponding to the target nodes; The data corresponding to the target node is input into the feature extraction layer of the graph network model to be trained, and the first feature output by the feature extraction layer is obtained. Using a homomorphic encryption algorithm, the first feature is encrypted using the public key of the second participant to determine the first encrypted feature, and the second encrypted feature sent by the second participant is received. The second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data. Based on the first encryption feature and the second encryption feature, a fusion feature is determined; The fused features are input into the subsequent processing layer of the graph network model to be trained to obtain the encrypted output result of the graph network model to be trained. The encrypted output result is sent to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key; The decryption result returned by the second participant is received, and the graph network model to be trained is trained based on the decryption result and the annotation.

2. The method of claim 1, further comprising, before sending the encrypted output result to the second participant: The preset perturbation feature is encrypted using the public key to obtain the encrypted perturbation feature; The encryption output result is scrambled using the aforementioned encryption perturbation feature; The training of the graph network model to be trained is performed based on the decryption result and the annotation, specifically including: Based on the perturbation characteristics, the interference added to the decryption result is eliminated; The graph network model to be trained is trained using the decryption results after interference removal and the annotations.

3. The method as described in claim 1, wherein the second feature is obtained by the second participant adjusting the dimension of the second intermediate feature to a specified dimension using the second parameter, and the second intermediate feature is determined by the second participant based on the target node in its stored graph data; Before encrypting the first feature using the public key of the second participant, the method further includes: The dimension of the first feature is adjusted to the specified dimension using the first parameter.

4. The method as described in claim 3, wherein training the graph network model to be trained is performed based on the decryption result and the annotation, specifically including: The parameters to be trained and the first parameter are adjusted with the goal of minimizing the difference between the decryption result and the annotation.

5. A federated learning method, said method being applied to a second participant, comprising: The second participant determines the user nodes shared with the first participant from its own stored graph data, and uses them as target nodes. Extract the second feature of the data corresponding to the target node; The second encrypted feature is determined by encrypting the second feature using the public key of the second participant through a homomorphic encryption algorithm. The second encryption feature is sent to the first participant, so that a fusion feature is determined based on the first encryption feature and the second encryption feature. The first encryption feature is obtained by the first participant encrypting the first feature using the public key. The first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained. Receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant by inputting the fused features into the subsequent processing layer of the graph network model to be trained; The encrypted output is decrypted using the private key of the second participant; The decryption result is returned to the first participant, enabling the first participant to train the graph network model to be trained based on the decryption result and the annotations. The annotations are determined by the first participant based on its own historical business execution results corresponding to the target node.

6. The method as described in claim 5, wherein the encrypted output result is obtained by the first participant scrambling the intermediate result output by the first participant's graph network model to be trained using encrypted perturbation features, and the intermediate result is determined by the first participant by inputting the fusion feature into the subsequent processing layer of the graph network model to be trained.

7. The method as described in claim 5, wherein the first feature is obtained by the first participant adjusting the dimension of the first intermediate feature to a specified dimension using a first parameter, and the first intermediate feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained; Before encrypting the second feature using the public key of the second participant, the method further includes: The second parameter is used to adjust the dimension of the second feature to the specified dimension.

8. The method as described in claim 7, wherein the decryption result is returned to the first participant, enabling the first participant to train the graph network model to be trained using the decryption result and annotations, specifically includes: The decryption result is returned to the first participant, so that the first participant can adjust the parameters of the graph network model to be trained and the first parameter with the optimization goal of minimizing the difference between the decryption result and the annotation.

9. A federated learning device, comprising: The first determining module is used by the first participant to determine the user node shared with the second participant from its own stored graph data as the target node, and to determine the label of the target node based on the historical business execution results corresponding to the target node. The first extraction module is used to input the data corresponding to the target node into the feature extraction layer of the graph network model to be trained, and obtain the first feature output by the feature extraction layer. The first encryption module is used to encrypt the first feature using a homomorphic encryption algorithm based on the public key of the second participant, determine the first encrypted feature, and receive the second encrypted feature sent by the second participant, wherein the second encrypted feature is obtained by the second participant encrypting the second feature using the public key, and the second feature is determined by the second participant based on the target node in its stored graph data. The fusion module is used to determine the fusion feature based on the first encryption feature and the second encryption feature; The output module is used to input the fused features into the subsequent processing layer of the graph network model to be trained, and obtain the encrypted output result of the graph network model to be trained. The sending module is used to send the encrypted output result to the second participant, so that the second participant can decrypt the encrypted output result using the private key corresponding to the public key; The training module is used to receive the decryption result returned by the second participant, and to train the graph network model to be trained based on the decryption result and the annotation.

10. The apparatus of claim 9, further comprising a scrambling module, specifically configured to encrypt a preset perturbation feature using the public key to obtain an encrypted perturbation feature; and to scramble the encrypted output result using the encrypted perturbation feature; The training module is specifically used to eliminate the interference added to the decryption result based on the perturbation features; and to train the graph network model to be trained using the decryption result after interference elimination and the annotation.

11. The apparatus of claim 9, wherein the second feature is obtained by the second participant adjusting the dimension of the second intermediate feature to a specified dimension using a second parameter, and the second intermediate feature is determined by the second participant based on the target node in its stored graph data; The device further includes a first adjustment module, specifically used to adjust the dimension of the first feature to a specified dimension using a first parameter.

12. The apparatus of claim 11, wherein the training module is specifically configured to adjust the parameters to be trained and the first parameter with the goal of minimizing the difference between the decryption result and the annotation.

13. A federated learning device, comprising: The second determining module is used by the second participant to determine the user nodes shared with the first participant from its own stored graph data, and to use them as target nodes. The second extraction module is used to extract the second feature of the data corresponding to the target node; The second encryption module is used to encrypt the second feature using a homomorphic encryption algorithm based on the public key of the second participant, and to determine the second encryption feature; The feature sending module is used to send the second encrypted feature to the first participant, so that a fusion feature is determined based on the first encrypted feature and the second encrypted feature. The first encrypted feature is obtained by the first participant encrypting the first feature using the public key. The first feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained. The second receiving module is used to receive the encrypted output result sent by the first participant, wherein the encrypted output result is determined by the first participant by inputting the fused features into the subsequent processing layer of the graph network model to be trained; The decryption module is used to decrypt the encrypted output using the private key of the second participant; The return module is used to return the decryption result to the first participant, so that the first participant can train the graph network model to be trained based on the decryption result and the annotation. The annotation is determined by the first participant based on its own historical business execution results corresponding to the target node.

14. The apparatus of claim 13, wherein the encrypted output result is obtained by the first participant scrambling the intermediate result output by the first participant's graph network model to be trained using an encrypted perturbation feature, and the intermediate result is determined by the first participant by inputting the fusion feature into the subsequent processing layer of the graph network model to be trained.

15. The apparatus of claim 13, wherein the first feature is obtained by the first participant adjusting the dimension of the first intermediate feature to a specified dimension using a first parameter, and the first intermediate feature is determined by the first participant inputting the data corresponding to the target node in its stored graph data into the feature extraction layer of the graph network model to be trained; The device further includes a second adjustment module, specifically used to adjust the dimension of the second feature to a specified dimension using a second parameter.

16. The apparatus of claim 15, wherein the return module is specifically configured to return the decryption result to the first participant, so that the first participant adjusts the parameters of the graph network model to be trained and the first parameter with the goal of minimizing the difference between the decryption result and the annotation.

17. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 8.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 8.

Images