A federated learning system

Abstract

The embodiment of the present invention discloses a federated learning system, including: a collaboration end and at least two participating ends, the participating ends include a label holding end and at least one feature holding end, the collaboration end and each participating end are connected in communication, and the training of the federated learning system includes: determining a participating end as the current model update end; the current model update end obtains model update parameters, determines an encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaboration end, wherein the model update parameters are determined by the label holding end according to the local model parameters of the previous current model update end; the collaboration end determines the model update direction according to the encrypted gradient value, and sends the model update direction to the current model update end; the current model update end updates the local model parameters according to the model update direction. By updating the local models of the participating ends one by one, it is avoided that each participating end simultaneously transmits a large amount of data to the collaboration end, thereby improving the efficiency of federated learning.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence technology, and in particular to a federated learning system. Background Art

[0002] Federated learning refers to a machine learning framework that can effectively help multiple participants (which can represent individuals or institutions) to jointly train models while meeting the requirements of data privacy protection. According to the different relationships between the samples and features of the data held by each participant in federated learning, it can be divided into horizontal federated learning, vertical federated learning, and federated transfer learning. In some scenarios, vertical federated learning has been more widely used. Vertical federated learning modeling requires encrypted alignment of samples first, and then encrypted model training. During the modeling process, each participant sends information to the third-party collaboration end, that is, each participant cannot obtain the other party's data information and gradient information, which basically guarantees the data security of each participant. In the process of realizing the present invention, the inventor found that there are at least the following technical problems in the prior art: the amount of data that needs to be transmitted in each iteration process in the modeling process in the prior art is large, and the efficiency of federated learning is low. Summary of the invention

[0003] The embodiment of the present invention provides a federated learning system to reduce the amount of data transmission during iterations in a modeling process and improve the efficiency of federated learning.

[0004] The embodiment of the present invention provides a federated learning system, including: a collaboration end and at least two participating ends, the participating end including a tag holding end and at least one feature holding end, the collaboration end and each participating end are connected in communication, and the training of the federated learning system includes:

[0005] Determine a participating end as the current model update end;

[0006] The current model update end obtains the model update parameters, determines the encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaboration end, wherein the model update parameters are determined by the tag holding end according to the local model parameters of the previous current model update end;

[0007] The collaboration end determines the model update direction based on the encrypted gradient value and sends the model update direction to the current model update end;

[0008] The current model update end updates the local model parameters according to the model update direction.

[0009] Optionally, based on the above solution, the current model update end is a label holding end, the current model update end obtains a model update parameter, and determines an encrypted gradient value according to the model update parameter, including:

[0010] The current model update end determines the encrypted gradient value based on local data and model update parameters.

[0011] Optionally, based on the above solution, the current model update end is a feature holding end, the current model update end obtains model update parameters, determines the encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaboration end, including:

[0012] The current model update end obtains the model update parameters from the tag holding end and determines the encrypted gradient value based on the local data and the model update parameters.

[0013] Optionally, based on the above solution, the collaboration end determines the model update direction according to the encrypted gradient value, including:

[0014] The collaborative end decrypts the encrypted gradient value to obtain the original gradient value, and determines the model update direction based on the original gradient value and the preset step size.

[0015] Optionally, based on the above solution, the current model update end updates the local model parameters according to the model update direction, including:

[0016] The current model update end uses the difference between the current local model parameters and the model update direction as the updated local model parameters.

[0017] Optionally, based on the above solution, it also includes:

[0018] The current model updating end determines the intermediate value according to the updated local model parameters, so that the tag holding end updates the model updating parameters according to the intermediate value.

[0019] Optionally, based on the above solution, the current model update end is the tag holding end, and the current model update end determines the intermediate value according to the updated local model parameter, so that the tag holding end updates the model update parameter according to the intermediate value, including:

[0020] The current model update end determines the intermediate value according to the updated local model parameters, and updates the model update parameters according to the intermediate value.

[0021] Optionally, based on the above solution, the current model updating end is a feature holding end, and the current model updating end determines an intermediate value according to the updated local model parameter, so that the label holding end updates the model update parameter according to the intermediate value, including:

[0022] The current model updating end determines the intermediate value according to the updated local model parameters and sends the intermediate value to the tag holding end;

[0023] The tag holder updates the model update parameters according to the intermediate value.

[0024] Optionally, based on the above solution, it also includes:

[0025] The collaborative end determines whether the convergence condition is met based on the encrypted gradient value sent by each participating end, and determines that the federated learning system converges when the convergence condition is met.

[0026] Optionally, based on the above solution, the collaboration end determines whether the convergence condition is met according to the encrypted gradient value sent by each participating end, including:

[0027] For the encrypted gradient value sent by each participating end, the collaboration end determines whether the characteristic value of the encrypted gradient value is less than a set threshold;

[0028] When the characteristic values ​​of the encrypted gradient values ​​of each participating terminal are all less than the set threshold, it is determined that the convergence condition is met.

[0029] The federated learning system provided by the embodiment of the present invention includes: a collaborative end and at least two participating ends, the participating end includes a label holding end and at least one feature holding end, the collaborative end and each participating end are communicatively connected, and the training of the federated learning system includes: determining a participating end as the current model update end; the current model update end obtains model update parameters, determines an encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaborative end, wherein the model update parameters are determined by the label holding end according to the local model parameters of the previous current model update end; the collaborative end determines the model update direction according to the encrypted gradient value, and sends the model update direction to the current model update end; the current model update end updates the local model parameters according to the model update direction, and by updating the local models of the participating ends one by one, it is avoided that each participating end simultaneously transmits a large amount of data to the collaborative end, thereby improving the efficiency of federated learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a training flow chart of a federated learning system provided in Embodiment 1 of the present invention;

[0031] Figure 2 This is a training flow chart of a federated learning system provided in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0033] Embodiment 1

[0034] Figure 1This is a training flow chart of a federated learning system provided by an embodiment of the present invention. This embodiment is applicable to the case of training a federated learning system, and is particularly applicable to the case of training a vertical federated learning system.

[0035] In this embodiment, the federated learning system includes a collaboration end and at least two participating ends. The participating ends include a label holding end and at least one feature holding end. The collaboration end and each participating end are connected in communication. Taking the risk control scenario as an example, its characteristic is that the label is only held by one of the participating ends (which can be called Guest), while the other participating ends only have some features of the data (which can be called Host). The effect of the model is improved by the cooperation between Guest and Host, so as to reduce the risk. In this process, both the Guest and Host need to ensure the security of their own data through encryption and collaboration. Assume that there are p participating ends, one of which is Guest, and the remaining p-1 participating ends are Host. There is also a third-party collaboration end Coordinator to coordinate the whole process. Let the local data of the pth participating end after privacy data alignment be x p , the model parameter is w p . Then the overall training data is equivalent to x=(x 1 , x 2 , …, x p ), and the overall logistic regression model is w=(w 1 , w 2 ,…,w p ).

[0036] This embodiment provides an exemplary description of a certain iteration process of the federated learning system. Figure 1 As shown, the training of the federated learning system includes:

[0037] S110. Determine a participating end as the current model updating end.

[0038] In order to avoid the technical problem of excessive data transmission caused by each participating end sending data to the collaboration end at the same time, in this embodiment, the local model of each participating end is updated one by one to reduce the amount of data transmission. Therefore, in a certain iteration, it is necessary to first select a participating end as the current model update end.

[0039] Optionally, a participating terminal may be randomly selected as the current model update terminal, or a model update sequence may be preset, and the current model update terminal may be determined according to the preset model update sequence. Exemplarily, assuming that there are p participating terminals in total, each participating terminal may be labeled with a number, and the sequence may be executed from 1 to p, i.e., p = ((t-1) mod p) + 1, and a participating terminal may be determined as the current model update terminal according to the sequence; a participating terminal may also be randomly selected from 1 to P as the current model update terminal.

[0040] S120: The current model update end obtains model update parameters, determines encrypted gradient values ​​according to the model update parameters, and sends the encrypted gradient values ​​to the collaboration end.

[0041] In this embodiment, the model update parameter is a parameter calculated and stored in the tag holding end, and is used to update the local model parameter of the current model update end. When the number of iterations is 1, the model update parameter can be determined by the tag holding end based on the intermediate value of each participating end, and when the number of iterations is ≥ 1, the model update parameter can be determined by the tag holding end based on the local model parameter of the last current model update end. Specifically, the model update parameter is determined based on the intermediate value of the last current model update end.

[0042] Optionally, at the beginning of training, the local model of each participating terminal generates initial model parameters. For each participating terminal, an intermediate value is calculated based on the initial model parameters of its local model. Each feature holding terminal transmits its intermediate value to the tag holding terminal through the additive homomorphic encryption algorithm. The tag holding terminal can obtain the intermediate values ​​of all feature holding terminals and calculate the model update parameters based on the principle of additive homomorphic encryption, so that the initial current model update terminal updates the local model parameters according to the model update parameters. Exemplarily, the intermediate values ​​in the participating terminals can be transmitted through u p (0) = w p (0) T x p Calculated, where u p (0) is the median value of the pth participating terminal, w p (0) is the local model parameter of the pth participant, x p is the local training data of the pth participant. The model update parameters can be Calculated, where [[d(0)]] is the model update parameter, y is the data label, and u p (0) is the median value of the pth participating terminal, and N is the total number of participating terminals.

[0043] In this embodiment, the model update parameters are calculated and stored by the tag holding end. Therefore, when the current model update end is the tag holding end and the current model update end is the feature holding end, the manner of obtaining the model update parameters is different.

[0044] In one embodiment, the current model update end is a tag holding end, the current model update end obtains model update parameters, and determines the encrypted gradient value according to the model update parameters, including: the current model update end determines the encrypted gradient value according to local data and model update parameters. When the current model update end is a tag holding end, the model update parameters stored in the local end can be directly obtained, and the encrypted gradient value can be calculated according to the model update parameters and the local data (i.e., training data) of the current model update end. Exemplarily, it can be based on Calculate the encrypted gradient value, where [[g p (t)]] is the encrypted gradient value, [[d i (t-1)]] is the model update parameter, It is the local training data of the current model update end.

[0045] In one embodiment, the current model update end is a feature holding end, the current model update end obtains model update parameters, determines the encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaboration end, including: the current model update end obtains the model update parameters from the tag holding end, determines the encrypted gradient value according to the local data and the model update parameters. When the current model update end is a feature holding end, it is necessary to obtain the model update parameters from the tag holding end, and then calculate the encrypted gradient value according to the model update parameters and the local data (i.e., training data) of the current model update end. Similarly, it can be based on Calculate the encrypted gradient value, where [[g p (t)]] is the encrypted gradient value, [[d i (t-1)]] is the model update parameter, x i p It is the local training data of the current model update end.

[0046] S130. The collaboration end determines a model update direction according to the encrypted gradient value, and sends the model update direction to the current model update end.

[0047] After receiving the encrypted gradient value sent by the current model update end, the collaboration end determines the model update direction according to the encrypted gradient value, and returns the model update direction to the current model update end. In this embodiment, during one iteration, only the current model update end needs to send the encrypted gradient value to the collaboration end, which greatly reduces data transmission and improves the efficiency of federated learning.

[0048] In one embodiment of the present invention, the collaboration end determines the model update direction according to the encrypted gradient value, including: the collaboration end decrypts the encrypted gradient value to obtain the original gradient value, and determines the model update direction according to the original gradient value and a preset step size. Specifically, the collaboration end decrypts the encrypted gradient value according to the private key to obtain the original gradient value, and then calculates the model update direction according to the original gradient value and a preset step size. Exemplarily, the product of the original gradient value and the preset step size can be used as the model update direction, that is, by Calculate the model update direction. Among them, is the model update direction, η is the preset step size, g p (t) is the original gradient value.

[0049] S140: The current model updating end updates the local model parameters according to the model updating direction.

[0050] After receiving the model update direction sent by the collaboration end, the current model update end updates the local model parameters based on the model update direction to obtain an updated local model. The updating of the local model parameters based on the model update direction can refer to the model update method in the prior art.

[0051] In one embodiment, the current model update end updates the local model parameters according to the model update direction, including: the current model update end uses the difference between the current local model parameters and the model update direction as the updated local model parameters. For the parameters w in the local model p Update, where Updates the orientation for the model.

[0052] The federated learning system provided by the embodiment of the present invention includes: a collaborative end and at least two participating ends, the participating end includes a label holding end and at least one feature holding end, the collaborative end and each participating end are communicatively connected, and the training of the federated learning system includes: determining a participating end as the current model update end; the current model update end obtains model update parameters, determines an encrypted gradient value according to the model update parameters, and sends the encrypted gradient value to the collaborative end, wherein the model update parameters are determined by the label holding end according to the local model parameters of the previous current model update end; the collaborative end determines the model update direction according to the encrypted gradient value, and sends the model update direction to the current model update end; the current model update end updates the local model parameters according to the model update direction, and by updating the local models of the participating ends one by one, it is avoided that each participating end simultaneously transmits a large amount of data to the collaborative end, thereby improving the efficiency of federated learning.

[0053] Optionally, based on the above scheme, after the current model update end updates the local model parameters, it also includes: the current model update end determines an intermediate value based on the updated local model parameters, so that the label holding end updates the model update parameters based on the intermediate value. In this embodiment, in order to effectively utilize the information of each iteration, after the local model parameters of the current model update end are updated, the intermediate value is determined based on the updated local model parameters, and the model update parameters in the label holding end are adjusted using the intermediate value, so that the intermediate value of the current model update end can have a corresponding impact on the overall gradient of the federated learning system.

[0054] Exemplarily, the current model update end determines the intermediate value according to the updated local model parameters by: p (t) = w p (t) T x p Calculate the intermediate value. Among them, up(t) is the intermediate value, w p (t) is the local model parameter of the current model update end, x p Update the local training data for the current model.

[0055] In one embodiment of the present invention, the tag holder can update the model update parameter according to the intermediate value by: The model update parameters are updated. The specific meanings of the parameters in the above formula can be referred to the above embodiment and will not be repeated here.

[0056] In this embodiment, the model update parameters are calculated and stored by the tag holding end. Therefore, when the current model update end is the tag holding end and the current model update end is the feature holding end, the model update parameters are updated in different ways after calculating the intermediate value.

[0057] In one embodiment, the current model update end is a tag holding end, and the current model update end determines an intermediate value according to the updated local model parameter, so that the tag holding end updates the model update parameter according to the intermediate value, including: the current model update end determines the intermediate value according to the updated local model parameter, and updates the model update parameter according to the intermediate value. Optionally, when the current model update end is a tag holding end, the current model update end can calculate the intermediate value and update the model update parameter at the local end. Exemplarily, when the current model update end is a tag holding end, it can be updated by u p (t) = w p (t) T x p Calculate the intermediate value, and then directly based on the calculated intermediate value, through Update the model update parameters.

[0058] In one embodiment, the current model update end is a feature holding end, and the current model update end determines an intermediate value based on the updated local model parameters, so that the tag holding end updates the model update parameters based on the intermediate value, including: the current model update end determines the intermediate value based on the updated local model parameters, and sends the intermediate value to the tag holding end; the tag holding end updates the model update parameters based on the intermediate value. Optionally, when the current model update end is a feature holding end, the current model update end can only calculate the intermediate value on this end, and after calculating the intermediate value, send the intermediate value to the tag holding end, and the tag holding end updates the model update parameters based on the received intermediate value. Exemplarily, when the current model update end is a feature holding end, it can be updated through u p (t) = w p (t) T x p Calculate the intermediate value and send it to the tag holder, which then Update the model update parameters.

[0059] In general, no matter whether the current model update end is the label holding end or the feature holding end, it is necessary to recalculate the intermediate value after the local model parameters of the current model update end are updated, and then the label holding end updates the model update parameters according to the intermediate value calculated after the current model update end is updated. The updated model update parameters are used to update the next model update end. The update logic of the next model update end can refer to the update logic of the current model update end until the local models of all participating ends converge, stop iteration, and obtain a trained federated learning system. It can be understood that the method for judging model convergence can refer to the method for judging model convergence in the prior art, and is not limited here.

[0060] According to the above embodiments, in one iteration, the encrypted information of the previous iteration is effectively utilized, avoiding the high computational complexity and communication complexity caused by calculating and transmitting the information of all participating terminals each time, and the model update is realized by updating the information of only one party. At the same time, although only the model of one party is updated, the updated intermediate value affects the overall gradient and will be propagated to the next participating terminal in the gradient calculation of the next iteration. In this process, if the label holding terminal is updated, the label holding terminal and the collaborative terminal only need to communicate back and forth once; if the feature holding terminal is updated, only one feature holding terminal and the collaborative terminal need to communicate back and forth once, and one feature holding terminal and the label holding terminal need to communicate back and forth once. Therefore, the scheme provided by the embodiment of the present invention greatly reduces the amount of data transmission in each iteration. The communication overhead of each iteration of the embodiment of the present invention can be compared with the scheme in the prior art. The specific comparison is as follows: Assuming that the total number of data samples is N, the feature dimension of the pth participating terminal (i.e., the current model update terminal) is m p, each value is F bits before encryption and F bits after encryption e bits. The communication overhead of each iteration of the existing scheme is For the embodiment of the present invention: if p is the host side: the communication overhead of each iteration mainly comes from the transmission of model update parameters from the tag holding end to the current model update end, the transmission of encrypted gradient values ​​from the current model update end to the collaboration end, the transmission of model update direction from the collaboration end to the current model update end, and the transmission of intermediate values ​​from the current model update end to the tag holding end, and the communication overhead is NF e 、m p F e NF, NF e , then the total cost is 2NF e +m p (F+F e ) bits; if p is the Guest side: the communication overhead of each iteration mainly comes from the encrypted gradient value transmission from the current model update end to the collaborative end and the model update direction transmission from the collaborative end to the current model update end, and their communication overheads are m p F e 、m p F, then the total cost is m p (F+F e ) bits. Therefore, compared with the existing solution, the embodiment of the present invention can greatly reduce the transmission amount of each iteration, thereby improving the efficiency of federated learning.

[0061] In one embodiment of the present invention, it also includes: the collaboration end determines whether the convergence condition is met according to the encrypted gradient value sent by each participating end, and determines that the federated learning system converges when the convergence condition is met. That is, the collaboration end determines whether the local model of each participating end meets the convergence condition through the encrypted gradient value sent by each participating end after iteration, and when the local model of each participating end meets the convergence condition, the federated learning system is determined to converge. By judging whether the local model of each participating end converges through the collaboration end, whether the federated learning system converges is judged, the information received by the collaboration end is fully utilized, and the efficiency of federated learning is improved.

[0062] Optionally, the collaborative end determines whether the convergence condition is met based on the encrypted gradient value sent by each participating end, including: for each encrypted gradient value sent by the participating end, the collaborative end determines whether the characteristic value of the encrypted gradient value is less than a set threshold; when the characteristic values ​​of the encrypted gradient values ​​of each participating end are all less than the set threshold, it is determined that the convergence condition is met. Exemplarily, after receiving the encrypted gradient value sent by any participating end, the collaborative end can use the private key to decrypt the encrypted gradient value to obtain the original gradient value, calculate the characteristic value of the original gradient value, and compare the characteristic value of the original gradient value with the preset threshold to determine whether the local model of the participating end has converged. Among them, the characteristic value of the original gradient value can be any characteristic value of the original gradient value, and the specific setting can be determined according to actual needs. For example, the characteristic value of the original gradient value can be the binary norm of the original gradient value, and the corresponding preset threshold can be set to 10 -3 .

[0063] Embodiment 2

[0064] This embodiment provides a preferred embodiment based on the above embodiments. In this embodiment, the tag holding end is concretized as Guest, the feature holding end is concretized as Host, and the collaboration end is concretized as a third-party Coordinator.

[0065] In this embodiment, it is assumed that there are P participating terminals, one of which is a guest, and the remaining p-1 participating terminals are hosts. In addition, there is a third-party Coordinator to coordinate the entire process. The local data of the pth participating terminal after privacy data alignment is x p , the model parameter is w p Before iterative training, each participating terminal needs to be initialized. Specifically, each participating terminal generates initial model parameters w p (0), and then calculate the intermediate values ​​locally. Each host transmits the ciphertext to the guest through additive homomorphic encryption (such as Paillier). After the guest obtains all the intermediate values, it calculates the model update parameters according to the principle of additive homomorphic encryption. After initialization is completed, it enters the iterative training phase.

[0066] Figure 2 is a flow chart of a single iterative training of a federated learning system provided in the second embodiment of the present invention. Figure 2 As shown, Figure 2 The training process of the tth iteration is schematically shown in FIG. , and the specific steps are as follows:

[0067] Step 1: Select the pth participating end as the model updater for this iteration (i.e., the current model updater);

[0068] Step 2: If the pth participant is the Host, the Guest transmits the model update parameters to p, otherwise directly executes Step 3;

[0069] Step 3: The pth participant calculates the encrypted gradient value based on local data and transmits it to the Coordinator;

[0070] Step 4: The Coordinator decrypts the gradient, multiplies it by the step size to calculate the model update direction, and sends it back to the pth participating end;

[0071] Step 5: The pth participant updates the local model parameters. The update formula can be

[0072] Step 6: The participating end p recalculates the intermediate value based on the updated local model parameters;

[0073] Specifically, if the pth participating end is the host, the intermediate value is encrypted and transmitted to the guest. If it is the guest, the intermediate value is directly calculated locally.

[0074] Step 7: The guest updates the model and parameters according to the intermediate value of p;

[0075] Specifically:

[0076] Step 8: t←t+1, enter the next iteration.

[0077] In Step 1, a participating end p needs to be selected. The specific selection method can refer to the above embodiment. During training, the above process is iterated until the algorithm converges. The convergence of the algorithm can be judged by the gradient: for example, if the Coordinator finds that the two norms of the gradient values ​​of each party are less than the set threshold (such as 10 -3 ), the algorithm is considered to have converged and the iteration stops.

[0078] The embodiment of the present invention proposes a longitudinal federated learning framework for co-building a logistic regression model with the participation of multiple parties, and provides a model update scheme with high transmission efficiency. In each iteration process, only one host party is required to participate, so that only a small number of participating terminals update the model parameters in each iteration, reducing the number of times data is transmitted to each other in each iteration, thereby improving the overall communication efficiency; an additive homomorphic encryption scheme is used to protect the security of intermediate results and prevent the leakage of data privacy to other parties.

[0079] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A federated learning system, characterized in that: include: A collaboration end and at least two participating ends, wherein the participating end includes a tag holding end and at least one feature holding end, the collaboration end and each of the participating ends are connected in communication, and the training of the federated learning system includes: Determine a participating end as the current model update end; The current model update end obtains a model update parameter, determines an encrypted gradient value according to the model update parameter, and sends the encrypted gradient value to the collaboration end, wherein the model update parameter is determined by the tag holding end according to the local model parameter of the previous current model update end; The collaboration end decrypts the encrypted gradient value to obtain the original gradient value, determines the model update direction according to the original gradient value and a preset step size, and sends the model update direction to the current model update end; The current model updating end uses the difference between the current local model parameter and the model updating direction as the updated local model parameter.

2. The system according to claim 1, characterized in that The current model update end is a tag holding end, and the current model update end obtains a model update parameter, and determines an encrypted gradient value according to the model update parameter, including: The current model updating end determines the encrypted gradient value according to local data and the model updating parameter.

3. The system according to claim 1, characterized in that The current model update end is a feature holding end, the current model update end obtains a model update parameter, determines an encrypted gradient value according to the model update parameter, and sends the encrypted gradient value to the collaboration end, including: The current model updating end obtains the model updating parameter from the tag holding end, and determines the encrypted gradient value according to local data and the model updating parameter.

4. The system according to claim 1, characterized in that Also includes: The current model updating end determines an intermediate value according to the updated local model parameters, so that the tag holding end updates the model updating parameters according to the intermediate value.

5. The system according to claim 4, characterized in that The current model updating end is a tag holding end, and the current model updating end determines an intermediate value according to the updated local model parameter, so that the tag holding end updates the model updating parameter according to the intermediate value, including: The current model updating end determines an intermediate value according to the updated local model parameters, and updates the model updating parameters according to the intermediate value.

6. The system according to claim 4, characterized in that The current model updating end is a feature holding end, and the current model updating end determines an intermediate value according to the updated local model parameter, so that the tag holding end updates the model updating parameter according to the intermediate value, including: The current model updating end determines an intermediate value according to the updated local model parameters, and sends the intermediate value to the tag holding end; The tag holding end updates the model update parameter according to the intermediate value.

7. The system according to claim 1, characterized in that Also includes: The collaboration end determines whether a convergence condition is met based on the encrypted gradient value sent by each of the participating ends, and determines that the federated learning system is converged when the convergence condition is met.

8. The system according to claim 1, characterized in that The collaboration end determines whether a convergence condition is reached according to the encrypted gradient value sent by each of the participating ends, including: For each encrypted gradient value sent by the participating end, the collaborative end determines whether a characteristic value of the encrypted gradient value is less than a set threshold; When the characteristic values ​​of the encrypted gradient values ​​of the participating terminals are all less than the set threshold, it is determined that the convergence condition is met.

Images