Federated learning modeling optimization method, device, medium, and computer program product

Abstract

The application discloses a federal learning modeling optimization method, device, medium and computer program product. The federal learning modeling optimization method is applied to a first device. The federal learning modeling optimization method comprises the following steps: obtaining a first-party local model prediction value corresponding to a first-party local model to be trained; performing encrypted value exchange with a second device based on a first-party random number encryption value generated by performing random number encryption on the first-party local model prediction value, to obtain a second-party random number encryption value; performing encrypted intermediate parameter exchange with the second device based on a second-party random number encryption intermediate parameter generated based on the second-party random number encryption value, to obtain a first-party random number encryption intermediate parameter; and optimizing the first-party local model to be trained based on the first-party random number encryption intermediate parameter, to obtain a first-party federal model. The application solves the technical problem of low security when a model is constructed based on federal learning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology in financial technology (Fintech), and more particularly to a federated learning modeling optimization method, device, medium, and computer program product. Background Technology

[0002] With the continuous development of fintech, especially internet fintech, more and more technologies (such as distributed systems and artificial intelligence) are being applied in the financial field. However, the financial industry is also placing higher demands on technology, such as on the distribution of tasks to be completed.

[0003] With the continuous development of computer technology, federated learning is being applied more and more widely. Currently, when building models using federated learning, the label owner typically needs to send homomorphically encrypted labels to the unlabeled feature owner. The feature owner then calculates the homomorphically encrypted gradient and loss based on the encrypted labels and sends them to the label owner. The label owner then decrypts the encrypted gradient and loss and sends the plaintext model gradient and loss to the feature owner, who can then update their local model. However, if the label owner... After receiving homomorphically encrypted labels, the feature owner sends the ciphertext content (homomorphically encrypted labels) that it wants to know, instead of the homomorphically encrypted gradient and loss, to the label owner. The label owner then decrypts the ciphertext and sends the decryption result back to the feature owner. The feature owner can then obtain the sample labels from the label owner. In other words, there is a decryption interface in the entire federated learning process of building the model. The feature owner can then obtain the privacy data of the label owner through the decryption interface, which will lead to the leakage of the sample labels, which are the privacy data of the label owner. Therefore, the current method of building models based on federated learning still has security risks. Summary of the Invention

[0004] The main purpose of this application is to provide a federated learning modeling optimization method, device, medium, and computer program product, which aims to solve the technical problem of low security when building models based on federated learning in the prior art.

[0005] To achieve the above objectives, this application provides a federated learning modeling optimization method, which is applied to a first device, and includes:

[0006] Obtain the first-party local model prediction value corresponding to the first-party local model to be trained;

[0007] Based on the first random number encrypted value generated by encrypting the first local model prediction value, the second random number encrypted value is exchanged with the second device to obtain the second random number encrypted value. The second random number encrypted value is obtained by the second device by encrypting the second local model prediction value and sample label corresponding to the second local model to be trained, so that the second device can obtain the first random number encrypted value.

[0008] The second random number encryption intermediate parameter generated based on the second random number encryption value is exchanged with the second device to obtain the first random number encryption intermediate parameter. The first random number encryption intermediate parameter is generated by the second device based on the first random number encryption value, so that the second device can optimize the second local model to be trained based on the second random number encryption intermediate parameter to obtain the second federated model.

[0009] Based on the intermediate parameters encrypted using the first-party random number, the first-party local model to be trained is optimized to obtain the first-party federated model.

[0010] To achieve the above objectives, this application provides a federated learning modeling optimization method, which is applied to a second device, and includes:

[0011] Obtain the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value;

[0012] Based on the second-party random number encryption value generated by encrypting the second-party local model prediction value and the sample label, the encryption value is exchanged with the first device to obtain the first-party random number encryption value. The first-party random number encryption value is obtained by the first device by encrypting the first-party local model prediction value corresponding to the first-party local model to be trained, so that the first device can obtain the second-party random number encryption value.

[0013] The first random number encryption intermediate parameter generated based on the first random number encryption value is exchanged with the first device to obtain the second random number encryption intermediate parameter. The second random number encryption intermediate parameter is generated by the first device based on the second random number encryption value, so that the first device can optimize the first local model to be trained based on the first random number encryption intermediate parameter to obtain the first federated model.

[0014] Based on the intermediate parameters encrypted using the second-party random number, the second-party local model to be trained is optimized to obtain the second-party federated model.

[0015] This application also provides a federated learning modeling optimization device, which is a virtual device and is applied to a first device. The federated learning modeling optimization device includes:

[0016] The acquisition module is used to acquire the first-party local model prediction value corresponding to the first-party local model to be trained.

[0017] The encrypted value exchange module is used to exchange encrypted values ​​with the second device based on the first random number encrypted value generated by random number encryption of the first local model prediction value, and to obtain the second random number encrypted value. The second random number encrypted value is obtained by the second device by random number encryption of the second local model prediction value and sample label corresponding to the second local model to be trained, so that the second device can obtain the first random number encrypted value.

[0018] The encrypted intermediate parameter exchange module is used to exchange encrypted intermediate parameters generated based on the encrypted value of the second random number with the second device to obtain encrypted intermediate parameters of the first random number. The encrypted intermediate parameters of the first random number are generated by the second device based on the encrypted value of the first random number, so that the second device can optimize the second local model to be trained based on the encrypted intermediate parameters of the second random number to obtain the second federated model.

[0019] The optimization module is used to encrypt intermediate parameters based on the first party's random number, optimize the first party's local model to be trained, and obtain the first party's federated model.

[0020] This application also provides a federated learning modeling optimization device, which is a virtual device and is applied to a second device. The federated learning modeling optimization device includes:

[0021] The acquisition module is used to acquire the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.

[0022] The encrypted value exchange module is used to exchange encrypted values ​​with the first device based on the second-party random number encrypted value generated by random number encryption of the second-party local model prediction value and the sample label, so as to obtain the first-party random number encrypted value. The first-party random number encrypted value is obtained by the first device by random number encryption of the first-party local model prediction value corresponding to the first-party local model to be trained, so as to allow the first device to obtain the second-party random number encrypted value.

[0023] An encrypted intermediate parameter exchange module is used to exchange encrypted intermediate parameters generated based on the encrypted value of the first random number with the first device to obtain encrypted intermediate parameters of the second random number. The encrypted intermediate parameters of the second random number are generated by the first device based on the encrypted value of the second random number, so that the first device can optimize the first local model to be trained based on the encrypted intermediate parameters of the first random number to obtain the first federated model.

[0024] The optimization module is used to encrypt intermediate parameters based on the second-party random number, optimize the second-party local model to be trained, and obtain the second-party federated model.

[0025] This application also provides a federated learning modeling optimization device, which is a physical device. The federated learning modeling optimization device includes: a memory, a processor, and a program of the federated learning modeling optimization method stored in the memory and executable on the processor. When the program of the federated learning modeling optimization method is executed by the processor, it can implement the steps of the federated learning modeling optimization method as described above.

[0026] This application also provides a medium, which is a readable storage medium, on which a program implementing the federated learning modeling optimization method is stored. When the program is executed by a processor, it implements the steps of the federated learning modeling optimization method as described above.

[0027] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the federated learning modeling optimization method described above.

[0028] This application provides a federated learning modeling optimization method, device, medium, and computer program product. Compared to existing technologies where the label owner participating in federated learning typically needs to send homomorphically encrypted labels to the unlabeled feature owner, and then the feature owner calculates the homomorphically encrypted gradient and loss based on the homomorphically encrypted labels, sends the homomorphically encrypted gradient and loss to the label owner, and then the label owner decrypts the homomorphically encrypted gradient and loss and feeds back the plaintext model gradient and model loss to the feature owner for model update, this application first obtains the first-party local model prediction value corresponding to the first-party local model to be trained, and then exchanges the first-party random number encrypted value generated by randomly encrypting the first-party local model prediction value with a second device to obtain the second-party random number encrypted value. The second-party random number encrypted value is obtained by the second device by randomly encrypting the second-party local model prediction value corresponding to the second-party local model to be trained and the sample labels, so that the second device can obtain the first-party random number encrypted value. The system then uses a second-party random number encrypted intermediate parameter generated based on the second-party random number encrypted value to exchange encrypted intermediate parameters with the second device, thereby obtaining a first-party random number encrypted intermediate parameter. This first-party random number encrypted intermediate parameter is generated by the second device based on the first-party random number encrypted value. The second device then uses this second-party random number encrypted intermediate parameter to optimize the second-party local model to be trained, obtaining a second-party federated model. Finally, based on the first-party random number encrypted intermediate parameter, the first-party local model to be trained is optimized, obtaining a first-party federated model. Throughout the entire interaction process of building the federated model, all data exchanged between the first and second devices is encrypted. Therefore, this application does not have a decryption interface during the federated model construction process. This overcomes the technical defect in the prior art where the presence of a decryption interface during the entire federated learning process allows the feature owner to extract the privacy data of the tag owner, leading to the leakage of sample tags, which are the privacy data of the tag owner. Therefore, this improves the security of building models based on federated learning. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0030] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating the first embodiment of the federated learning modeling optimization method of this application;

[0032] Figure 2 This is a schematic diagram of the process of iteratively training the first-party local model to be trained on the first device side in the federated learning modeling optimization method of this application;

[0033] Figure 3 This is a schematic diagram of the process of iteratively training the second-party local model to be trained on the second device side in the federated learning modeling optimization method of this application;

[0034] Figure 4 This is a flowchart illustrating the second embodiment of the federated learning modeling optimization method of this application;

[0035] Figure 5 This is a flowchart illustrating the third embodiment of the federated learning modeling optimization method of this application;

[0036] Figure 6 This is a flowchart illustrating the fourth embodiment of the federated learning modeling optimization method in this application;

[0037] Figure 7 A flowchart illustrating the process of the second device generating federated prediction results in the federated learning modeling optimization method of this application;

[0038] Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the federated learning modeling optimization method in the embodiments of this application;

[0039] Figure 9 This is a schematic diagram of the hardware architecture of federated learning involved in the embodiments of this application.

[0040] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0041] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0042] This application provides a federated learning modeling optimization method. In the first embodiment of the federated learning modeling optimization method of this application, refer to... Figure 1 The federated learning modeling optimization method is applied to the first device, and the federated learning modeling optimization method includes:

[0043] Step S10: Obtain the prediction value of the first-party local model corresponding to the first-party local model to be trained;

[0044] In this embodiment, it should be noted that the federated learning modeling optimization method is applied to the scenario of vertical federated learning. Both the first device and the second device are participants in the vertical federated learning. The first device has local samples without sample labels, and the second device has local samples with sample labels. The first-party local model to be trained is the untrained first-party local model in the first device. The first-party local model can be a logistic regression model, and the predicted value of the first-party local model is the logistic regression value output by the first-party local model to be trained in the first device.

[0045] Additionally, it should be noted that during the first iteration of the federated iterative training of the first-party local model, the model parameters of the first-party local model are in plaintext. Starting from the second iteration of the federated iterative training of the first-party local model, the model parameters of the first-party local model are all in homomorphically encrypted ciphertext. Therefore, the predicted value of the first-party local model can be expressed as the product of the model parameters of the first-party local model and the sample, or it can be expressed as the homomorphically encrypted product of the model parameters of the first-party local model and the sample. For example, assuming sample A is X and the model parameters of the first-party local model are W, the predicted value of the first-party local model is WX or [[WX]], where [[]] is the homomorphic encryption symbol.

[0046] To obtain the predicted value of the first-party local model corresponding to the first-party local model to be trained, specifically, to obtain the public sample ID, wherein the public sample ID is the sample ID determined by the first device and the second device through sample alignment, and then to obtain the first-party local sample corresponding to the public sample ID and the first-party local model parameters corresponding to the first-party local model parameters, and then to calculate the product of the first-party local model parameters and the first-party local sample to obtain the predicted value of the first-party local model.

[0047] Step S20: Based on the first random number encryption value generated by encrypting the first local model prediction value, the encryption value is exchanged with the second device to obtain the second random number encryption value. The second random number encryption value is obtained by the second device by encrypting the second local model prediction value and sample label corresponding to the second local model to be trained with random numbers, so that the second device can obtain the first random number encryption value.

[0048] In this embodiment, it should be noted that in the scenario of vertical federated learning, the vertical federated model is divided into two parts. One part is in the first device, which is the first-party local model to be trained, and the other part is in the second device, which is the second-party local model to be trained. The second-party local model to be trained is the second-party local model that has not been trained in the second device. The second-party local model can be a logistic regression model.

[0049] A first-party random number encrypted value, generated by encrypting the first-party local model prediction value, is exchanged with a second device to obtain a second-party random number encrypted value. This second-party random number encrypted value is obtained by the second device through random number encryption of the second-party local model prediction value corresponding to the second-party local model to be trained, and the sample labels. Specifically, based on a preset first-party random number mixing method, the homomorphically encrypted first-party local model prediction value is mixed with a preset first-party random number to obtain the first-party random number encrypted value. This preset first-party random number mixing method includes summation and product methods. Then, based on this first-party random number encrypted value, an encryption value exchange is performed with the second device to send the first-party random number encrypted value to the second device and receive the second-party random number encrypted value sent by the second device. The second device obtains a common sample ID consistent with that in the first device, and then obtains the second-party random number encrypted value corresponding to the common sample ID. The local samples and the second-party local model parameters corresponding to the second-party training local model parameters are used to calculate the product of the second-party local model parameters and the second-party local samples to obtain the second-party local model prediction value. Based on a preset second-party random number mixing method, the homomorphically encrypted second-party local model prediction value, the preset second-party random number, and the sample label corresponding to the second-party local samples are mixed to obtain the second-party random number encrypted value. The preset second-party random number mixing method includes summation and product, etc. For example, assuming the first-party local model prediction value is [[WX]], the preset first-party random number mixing method is summation, the preset first-party random number is r1, and the first-party random number encrypted value is [[WX]]+r1. Assuming the preset second-party random number mixing method is summation and difference, the second-party local model prediction value is [[WX2]], the sample label is y, the preset second-party random number is r2, and the second-party random number encrypted value is 1 / 4[[WX2]]-1 / 2y+r2.

[0050] The step of exchanging a first-party random number encrypted value, generated by encrypting the first-party local model prediction value with a second device to obtain a second-party random number encrypted value includes:

[0051] Step S21: Based on the public key in the preset first-party random number and the preset second-party key pair, the first-party local model prediction value is encrypted with a random number to obtain the first-party random number encrypted value.

[0052] In this embodiment, it should be noted that the second device generates a preset second-party key pair and shares the public key in the preset second-party key pair with the first device, while the first device generates a preset first-party key pair and shares the public key in the preset first-party key pair with the second device.

[0053] Based on the public key in the preset first-party random number and preset second-party key pair, the predicted value of the first-party local model is encrypted with a random number to obtain a first-party random number encrypted value. Specifically, if the predicted value of the first-party local model is in a homomorphic encryption state, where the predicted value is encrypted based on the public key in the preset second-party key pair, and the iteration round of federated learning is greater than 1, then the predicted value of the first-party local model is mixed with the preset first-party random number based on the preset first random number mixing method to obtain a first-party random number encrypted value. If the predicted value of the first-party local model is not in a homomorphic encryption state, and the iteration round of federated learning is equal to 1, then the predicted value of the first-party local model is homomorphically encrypted based on the public key in the preset second-party key pair to obtain a homomorphically encrypted first-party local model predicted value. Then, based on the preset first random number mixing method, the homomorphically encrypted first-party local model predicted value is mixed with the preset first-party random number to obtain a first-party random number encrypted value.

[0054] The step of encrypting the first-party local model prediction value with a random number based on the public key of a preset first-party random number and a preset second-party key pair to obtain the first-party random number encrypted value includes:

[0055] Step S211: Based on the public key in the preset second-party key pair, homomorphically encrypt the first-party local model prediction value to obtain the first-party homomorphically encrypted model prediction value.

[0056] In this embodiment, it should be noted that the iteration round of federated learning is 1, and the prediction value of the first-party homomorphic encryption model is the prediction value of the first-party local model in the homomorphic encryption state.

[0057] Step S212: Mix the predicted value of the first homomorphic encryption model with the preset first random number to generate the encrypted value of the first random number.

[0058] In this embodiment, the predicted value of the first homomorphic encryption model and the preset first random number are mixed to generate the encrypted value of the first random number. Specifically, the sum of the predicted value of the first homomorphic encryption model and the preset first random number is calculated to mix the predicted value of the first homomorphic encryption model and the preset first random number to obtain the encrypted value of the first random number.

[0059] Step S22: Based on the first party's encrypted random number value, exchange encrypted values ​​with the second device to obtain the second party's encrypted random number value.

[0060] In this embodiment, based on the first party's encrypted random number value, an encrypted value exchange is performed with the second device to obtain the second party's encrypted random number value. Specifically, the first device sends the first party's encrypted random number value to the second device and receives the second party's encrypted random number value sent by the second device. The second device sends the second party's encrypted random number value to the first device and receives the first party's encrypted random number value sent by the first device.

[0061] Step S30: The second random number encryption intermediate parameter generated based on the second random number encryption value is exchanged with the second device to obtain the first random number encryption intermediate parameter. The first random number encryption intermediate parameter is generated by the second device based on the first random number encryption value, so that the second device can optimize the second local model to be trained based on the second random number encryption intermediate parameter to obtain the second federated model.

[0062] In this embodiment, a second-party random number encrypted intermediate parameter, generated based on the second-party random number encrypted value, is exchanged with the second device to obtain a first-party random number encrypted intermediate parameter. This first-party random number encrypted intermediate parameter is generated by the second device based on the first-party random number encrypted value. The second device uses this parameter to optimize the second-party local model to be trained, obtaining a second-party federated model. Specifically, the second-party random number encrypted value is decrypted using the private key in a preset first-party key pair to obtain a second-party random number mixed value. Then, based on the second-party random number mixed value and the first-party local model prediction value, a second-party random number mixed intermediate parameter is calculated. Finally, based on the public key in the preset first-party key pair, the second-party random number mixed intermediate parameter is homomorphically encrypted to obtain the second-party random number encrypted intermediate parameter. This second-party random number encrypted intermediate parameter is then exchanged with the second device to obtain the second-party random number encrypted intermediate parameter. The second-party random number encryption intermediate parameter is sent to the second device, and the second device receives the first-party random number encryption intermediate parameter sent by the second device. The second device receives the second-party random number encryption intermediate parameter sent by the first device, performs random number elimination on the second-party random number intermediate parameter, and obtains the second-party encrypted intermediate parameter. Then, by calculating the product of the second-party encrypted intermediate parameter and the second-party local sample, a second-party encrypted model gradient is generated. Based on the second-party encrypted model gradient, the second-party local model to be trained is optimized to obtain the second-party federated model. The second device decrypts the first-party random number encryption value based on the private key in the preset second-party key pair to obtain the first-party random number mixture value. Then, based on the first-party random number mixture value and the second-party local model prediction value, the first-party random number mixture intermediate parameter is calculated. Based on the public key in the preset second-party key pair, the first-party random number mixture intermediate parameter is homomorphically encrypted to obtain the first-party random number encryption intermediate parameter.

[0063] The step of exchanging the second-party random number encryption intermediate parameter generated based on the second-party random number encryption value with the second device to obtain the first-party random number encryption intermediate parameter includes:

[0064] Step S31: Based on the private key and public key in the preset first-party key pair, the second-party random number encryption value, and the first-party local model prediction value, generate the second-party random number encryption intermediate parameters;

[0065] In this embodiment, based on the private and public keys of a preset first-party key pair, the encrypted value of the second-party random number, and the predicted value of the first-party local model, intermediate parameters for second-party random number encryption are generated. Specifically, based on the private key of the preset first-party key pair, the encrypted value of the second-party random number is decrypted to obtain a mixed value of the second-party random number. Then, based on the mixed value of the second-party random number and the first-party local sample corresponding to the predicted value of the first-party local model, intermediate parameters for second-party random number mixing are calculated. Finally, based on the public key of the preset first-party key pair, the intermediate parameters for second-party random number mixing are homomorphically encrypted to generate intermediate parameters for second-party random number encryption. The process of calculating the intermediate parameters for second-party random number mixing is as follows:

[0066]

[0067] Where d is the intermediate parameter of the second random number mixture, WX1 is the predicted value of the first local model, WX2 is the predicted value of the second local model, y is the sample label, r2 is the preset second random number, and the mixed value of the second random number is 1 / 4WX2-1 / 2y+r2.

[0068] The step of generating intermediate parameters for second-party random number encryption based on the private and public keys in the preset first-party key pair, the encrypted value of the second-party random number, and the predicted value of the first-party local model includes:

[0069] Step S311: Based on the private key in the preset first-party key pair, decrypt the encrypted value of the second-party random number to obtain the mixed value of the second-party random number;

[0070] In this embodiment, it should be noted that the second random number encryption value is a homomorphically encrypted second random number mixture value.

[0071] Step S312: Generate intermediate parameters of the second random number mixture based on the first local model prediction value and the second random number mixture value;

[0072] In this embodiment, a second random number mixture intermediate parameter is generated based on the first local model prediction value and the second random number mixture value. Specifically, the second random number mixture intermediate parameter is generated by calculating the sum of the first local model prediction value and the second random number mixture value.

[0073] Step S313: Based on the public key in the preset first-party key pair, encrypt the second-party random number mixing intermediate parameter to obtain the second-party random number encryption intermediate parameter.

[0074] In this embodiment, it should be noted that the private key in the preset first-party key pair is held only by the first device, while both the first device and the second device hold the public key in the preset first-party key pair.

[0075] Step S32: Based on the second party's random number encrypted intermediate parameters, exchange encrypted intermediate parameters with the second device to obtain the first party's random number encrypted intermediate parameters.

[0076] In this embodiment, based on the second-party random number encryption intermediate parameter, the first-party random number encryption intermediate parameter is obtained by exchanging encryption intermediate parameters with the second device. Specifically, the first device sends the second-party random number encryption intermediate parameter to the second device for the second device to receive the second-party random number encryption intermediate parameter, and the second device sends the first-party random number encryption intermediate parameter to the first device for the first device to receive the first-party random number encryption intermediate parameter. It should be noted that since the first-party random number encryption intermediate parameter is encrypted based on the public key in the preset second-party key, and the private key in the preset second-party key is only held by the second device, the first device cannot decrypt the first-party random number encryption intermediate parameter. Furthermore, since the second-party random number encryption intermediate parameter is encrypted based on the public key in the preset first-party key, and the private key in the preset first-party key is only held by the first device, the second device cannot decrypt the second-party random number encryption intermediate parameter.

[0077] Step S40: encrypt intermediate parameters based on the first party's random number, optimize the first party's local model to be trained, and obtain the first party's federated model.

[0078] In this embodiment, the first-party local model to be trained is optimized based on the first-party random number encrypted intermediate parameters to obtain a first-party federated model. Specifically, the first-party random number encrypted intermediate parameters are randomized to obtain first-party encrypted intermediate parameters. Then, the model parameters of the first-party local model to be trained are updated based on the first-party encrypted model gradient generated by the first-party encrypted intermediate parameters to obtain a first-party federated model.

[0079] The step of encrypting intermediate parameters based on the first party's random number, optimizing the first party's local model to be trained, and obtaining the first party's federated model includes:

[0080] Step S41: Perform random number elimination on the first party's random number encrypted intermediate parameters to obtain the first party's encrypted intermediate parameters;

[0081] In this embodiment, the first-party random number encryption intermediate parameter is subjected to random number elimination to obtain the first-party encrypted intermediate parameter. Specifically, the first-party random number encryption intermediate parameter is obtained by calculating the difference between the first-party random number encryption intermediate parameter and a preset first-party random number. The process of eliminating random numbers from the first-party random number encryption intermediate parameter is as follows:

[0082] [[d]]1=[[d1]]-r1

[0083]

[0084] Wherein, [[d]]1 is the first-party encrypted intermediate parameter, [[d1]] is the first-party random number encrypted intermediate parameter, WX1 is the first-party local model prediction value, WX2 is the second-party local model prediction value, y is the sample label, and r1 is the preset first-party random number.

[0085] Step S42: Generate a first-party encrypted gradient based on the first-party encrypted intermediate parameters and the first-party local sample corresponding to the first-party local model prediction value;

[0086] In this embodiment, a first-party encrypted gradient is generated based on the first-party encrypted intermediate parameters and the first-party local samples corresponding to the first-party local model prediction values. Specifically, the first-party encrypted gradient is generated by calculating the product between the first-party encrypted intermediate parameters and the first-party local samples corresponding to the first-party local model prediction values. The specific process for generating the first-party encrypted gradient is as follows:

[0087] [[g]]1=[[d]]1*X

[0088] Wherein, [[g]]1 is the first party's encryption gradient, [[d]]1 is the first party's encryption intermediate parameter, and X is the first party's local sample.

[0089] Step S43: Based on the first party's encrypted gradient, optimize the first party's local model to be trained to obtain the first party's federated model.

[0090] In this embodiment, the first-party local model to be trained is optimized based on the first-party encrypted gradient to obtain the first-party federated model. Specifically, based on the first-party encrypted gradient, the model parameters of the first-party local model to be trained are updated using a preset model parameter update method to obtain the first-party federated model. The preset model parameter update method includes gradient descent and gradient ascent, etc.

[0091] The step of optimizing the first-party local model to be trained based on the first-party encryption gradient to obtain the first-party federated model includes:

[0092] Step S431: Based on the first-party encrypted gradient, update the model parameters of the first-party local model to be trained, and determine whether the first-party local model to be trained after the model parameter update meets the preset training termination condition.

[0093] In this embodiment, it should be noted that the preset training end includes loss convergence and reaching the maximum number of iterations.

[0094] Step S432: If satisfied, the first-party local model to be trained after the model parameters are updated is used as the first-party federated model.

[0095] If the conditions are not met in step S433, return to the step of obtaining the first-party local model prediction value corresponding to the first-party local model to be trained.

[0096] In this embodiment, if the first-party local model to be trained after model parameter update meets the preset training termination condition, the first-party local model to be trained after model parameter update is directly used as the first-party federated model. If the first-party local model to be trained after model parameter update does not meet the preset training termination condition, the step of obtaining the first-party local model prediction value corresponding to the first-party local model to be trained is returned.

[0097] Throughout the federated learning process in this application embodiment, the first-party random number encrypted value, the second-party random number encrypted value, the first-party random number encrypted intermediate parameter, and the second-party random number encrypted intermediate parameter exchanged between the first device and the second device are all in a homomorphic encryption state. While the second device can decrypt the first-party random number encrypted value, the privacy of the data is still guaranteed due to the mixing of random numbers. Similarly, while the first device can decrypt the second-party random number encrypted value, the privacy of the data is also guaranteed due to the mixing of random numbers. The first-party random number encrypted intermediate parameter held by the first device is encrypted using the public key in a preset second-party key pair. Unable to decrypt, the second device holds intermediate parameters encrypted with a second-party random number, which are encrypted using the public key in the preset first-party key pair. The second device cannot decrypt these parameters. Furthermore, throughout the federated learning process, the gradients and model parameters in the models of both the first and second devices are in a homomorphic encrypted state. Neither the first nor the second device can decrypt their own data, thus ensuring data privacy during the federated learning process. Since the data transmitted between the first and second devices is encrypted, neither device needs the other's assistance to decrypt the data during the federated learning process. Consequently, there is no decryption interface during the entire federated learning process, thus enhancing the security of the federated learning process.

[0098] Furthermore, such as Figure 2 The diagram shows a flowchart of iterative training of the first-party local model on the first device side. In this flowchart, A is the first device, B is the second device, x1, x2, x3, x4, and x5 are features of the samples, y is the sample label, r1 is a preset first-party random number, wx_g is the predicted value of the first-party local model, wx_h is the predicted value of the second-party local model, the homomorphic encryption public and private keys generated by B are the preset second-party key pair, [[wx]]+r1 is the encrypted value of the first-party random number, [[d_b]] is the intermediate parameter for the first-party random number encryption, [[d]] is the intermediate parameter for the first-party encryption, and [[g]] is the first-party encryption gradient. Figure 3 The diagram shows the process of iteratively training the second-party local model to be trained on the second device side. In this diagram, A is the first device, B is the second device, x1, x2, x3, x4 and x5 are all features of the samples, y is the sample label, r2 is the preset second-party random number, wx_g is the predicted value of the first-party local model, wx_h is the predicted value of the second-party local model, the homomorphic encryption public key and private key generated by A are the preset first-party key pair, 1 / 4[[wx]]-1 / 2y+r2 is the encrypted value of the second-party random number, [[d_a]] is the intermediate parameter of the second-party random number encryption, [[d]] is the intermediate parameter of the second-party encryption, and [[g]] is the second-party encryption gradient.

[0099] Furthermore, when constructing a logistic regression model based on federated learning, the label owner participating in the federated learning process typically needs to send homomorphically encrypted labels to the unlabeled feature owner. The feature owner then calculates the homomorphically encrypted gradient and loss based on the homomorphically encrypted labels and sends these to the label owner. The label owner then decrypts the homomorphically encrypted gradient and loss and sends the plaintext model gradient and loss to the feature owner, who can then update their local logistic regression model. Therefore, in the prior art, there is a decryption interface in the process of constructing a logistic regression model based on federated learning, posing a risk of data privacy leakage for the label owner. However, in the process of constructing the model based on federated learning in this application embodiment, there is no decryption interface from beginning to end. Therefore, by constructing the logistic regression model using the federated learning modeling optimization method in this application embodiment, the risk of data privacy leakage for the label owner can be avoided, thus improving the security of constructing a logistic regression model based on federated learning.

[0100] This application provides a federated learning modeling optimization method. Compared to existing technologies where the label owner typically sends homomorphically encrypted labels to the unlabeled feature owner, and the feature owner calculates the homomorphically encrypted gradient and loss based on the labels, then sends these to the label owner, who decrypts them and feeds back the plaintext model gradient and loss to the feature owner for model updates, this application first obtains the first-party local model prediction value corresponding to the first-party local model to be trained. Then, based on the first-party local model prediction value, a first-party random number encrypted value is generated by encrypting the first-party local model prediction value with a second device to exchange encrypted values, obtaining a second-party random number encrypted value. This second-party random number encrypted value is obtained by the second device through random number encryption of the second-party local model prediction value and sample labels, allowing the second device to obtain the first-party random number encrypted value and then, based on the... The second-party random number encryption intermediate parameter, generated from the second-party random number encryption value, is exchanged with the second device to obtain the first-party random number encryption intermediate parameter. The first-party random number encryption intermediate parameter is generated by the second device based on the first-party random number encryption value. The second device uses this parameter to optimize the second-party local model to be trained, obtaining a second-party federated model. Then, based on the first-party random number encryption intermediate parameter, it optimizes the first-party local model to be trained, obtaining a first-party federated model. Throughout the entire interaction process of building the federated model, all data exchanged between the first and second devices is encrypted. Therefore, this embodiment does not have a decryption interface during the federated model construction process. This overcomes the technical defect in the prior art where a decryption interface exists during the entire federated learning process, allowing the feature owner to extract the label owner's privacy data, thus leading to the leakage of sample labels, which are the label owner's privacy data. Therefore, the security of building models based on federated learning is improved.

[0101] Furthermore, referring to Figure 4 Based on the first embodiment of this application, in another embodiment of this application, the first-party federated model includes first-party encryption model parameters encrypted with the public key of a preset second-party key pair, and the second-party federated model includes second-party encryption model parameters encrypted with the public key of a preset first-party key pair.

[0102] After the step of encrypting intermediate parameters based on the first party's random number, optimizing the first party's local model to be trained, and obtaining the first party's federated model, the federated learning modeling optimization method further includes:

[0103] Step A10: Obtain the first-party sample to be predicted, and generate the output value of the first-party encrypted model based on the first-party sample to be predicted and the parameters of the first-party encrypted model.

[0104] In this embodiment, it should be noted that the first-party encryption model parameters are model parameters encrypted based on the public key in the preset second-party key pair.

[0105] A first-party sample to be predicted is obtained, and a first-party encrypted model output value is generated based on the first-party sample to be predicted and the first-party encrypted model parameters. Specifically, the first-party sample to be predicted is obtained, and the first-party encrypted model output value is obtained by calculating the product between the first-party sample to be predicted and the first-party encrypted model parameters.

[0106] Step A20: Send the first-party encryption model output value and the target sample ID of the first-party sample to be predicted to the second device, so that the second device can generate the second-party encryption model output value based on the second-party sample to be predicted corresponding to the target sample ID and the second-party encryption model parameters, and decrypt the first-party encryption model output value based on the private key in the preset second-party key pair to obtain the first-party model output value.

[0107] In this embodiment, the output value of the first-party encryption model and the target sample ID of the first-party predictable sample are sent to the second device. The second device generates a second-party encryption model output value based on the second-party predictable sample corresponding to the target sample ID and the second-party encryption model parameters. The first-party encryption model output value is then decrypted using the private key in the preset second-party key pair to obtain the first-party model output value. Specifically, the first-party encryption model output value and the target sample ID of the first-party predictable sample are sent to the second device. The second device obtains the second-party predicted sample corresponding to the target sample ID, calculates the product between the second-party predicted sample and the second-party encryption model parameters to obtain the second-party encryption model output value, and decrypts the first-party encryption model output value using the private key in the preset second-party key pair to obtain the first-party model output value. The second device then sends the first-party model output value and the second-party encryption model output value to the first device.

[0108] Step A30: Receive the second-party encryption model output value and the first-party model output value sent by the second device, and decrypt the second-party encryption model output value based on the private key in the preset first-party key pair to obtain the second-party model output value;

[0109] In this embodiment, it should be noted that since the parameters of the second-party encryption model are encrypted based on the public key in the preset first key pair, the output value of the second-party encryption model can be decrypted based on the private key in the preset first key pair to obtain the output value of the second-party model.

[0110] Step A40: Generate federated prediction results based on the output values ​​of the first-party model and the second-party model.

[0111] In this embodiment, a federated prediction result is generated based on the output values ​​of the first-party model and the second-party model. Specifically, the output values ​​of the first-party model and the second-party model are aggregated based on a preset aggregation method to obtain the federated prediction result. The preset aggregation method includes weighted summation and weighted average, etc.

[0112] Furthermore, the model constructed based on federated learning in this application can be a logistic regression model. Therefore, when classifying based on the logistic regression model, since the logistic regression model is a federated model without a decryption interface, it is possible to achieve the purpose of classification based on the logistic regression model constructed based on federated learning with higher security, thereby improving the security of classification based on the logistic regression model constructed based on federated learning.

[0113] This application provides a federated prediction method. Since the first and second federated models are constructed based on a federated learning process without a decryption interface, the model parameters in both the first and second devices are in a homomorphic encrypted state. This allows the acquisition of a first-party sample to be predicted. Based on the first-party sample to be predicted and the first-party encrypted model parameters, a first-party encrypted model output value is generated. The first-party encrypted model output value and the target sample ID of the first-party sample to be predicted are then sent to the second device. The second device then generates a second-party encrypted model output value based on the second-party sample to be predicted corresponding to the target sample ID and the second-party encrypted model parameters. The system outputs a value and, based on the private key in the preset second-party key pair, decrypts the output value of the first-party encryption model to obtain the first-party model output value. It then receives the second-party encryption model output value and the first-party model output value sent by the second device, and, based on the private key in the preset first-party key pair, decrypts the second-party encryption model output value to obtain the second-party model output value. Finally, based on the first-party model output value and the second-party model output value, it generates a federated prediction result. This achieves the goal of federated prediction based on a federated model without a decryption interface, thereby achieving the goal of federated prediction based on a more secure federated model and improving the security of federated prediction.

[0114] Furthermore, in the first embodiment of the federated learning modeling optimization method of this application, referring to Figure 5The federated learning modeling optimization method is applied to the second device, and the federated learning modeling optimization method includes:

[0115] Step B10: Obtain the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value;

[0116] In this embodiment, it should be noted that the federated learning modeling optimization method is applied to the scenario of vertical federated learning. Both the first device and the second device are participants in the vertical federated learning. The first device has local samples without sample labels, and the second device has local samples with sample labels. The second-party local model to be trained is the untrained second-party local model in the second device. The second-party local model can be a logistic regression model, and the predicted value of the second-party local model is the logistic regression value output by the second-party local model to be trained in the second device.

[0117] Additionally, it should be noted that during the first iteration of the federated iterative training of the second-party local model, the model parameters of the second-party local model are in plaintext. Starting from the second iteration of the federated iterative training of the second-party local model, the model parameters of the second-party local model are all in homomorphically encrypted ciphertext. Therefore, the predicted value of the second-party local model can be expressed as the product of the model parameters of the second-party local model and the sample, or it can be expressed as the homomorphically encrypted product of the model parameters of the second-party local model and the sample. For example, assuming sample A is X and the model parameters of the second-party local model are W, the predicted value of the second-party local model is WX or [[WX]], where [[]] is the homomorphic encryption symbol.

[0118] The process involves obtaining the predicted value of the second-party local model corresponding to the second-party local model to be trained and the sample label corresponding to the predicted value of the second-party local model. Specifically, it involves obtaining a common sample ID, which is the sample ID determined by the first device and the second device through sample alignment. Then, it involves obtaining the second-party local sample corresponding to the common sample ID and the second-party local model parameters corresponding to the parameters of the second-party local model to be trained. The process then involves calculating the product of the second-party local model parameters and the second-party local sample to obtain the predicted value of the second-party local model. The first-party local sample and the second-party local sample correspond to the same common sample ID. Finally, it involves obtaining the sample label corresponding to the second-party local sample, which is the identifier of the second-party local sample and is used to identify the category or attribute of the second-party local sample.

[0119] Step B20: Based on the second-party random number encryption value generated by encrypting the second-party local model prediction value and the sample label, the encryption value is exchanged with the first device to obtain the first-party random number encryption value. The first-party random number encryption value is obtained by the first device by encrypting the first-party local model prediction value corresponding to the first-party local model to be trained, so that the first device can obtain the second-party random number encryption value.

[0120] In this embodiment, it should be noted that in the scenario of vertical federated learning, the vertical federated model is divided into two parts. One part is in the first device, which is the first-party local model to be trained, and the other part is in the second device, which is the second-party local model to be trained. The first-party local model to be trained is the first-party local model in the first device that has not been trained. The first-party local model can be a logistic regression model.

[0121] Based on the second-party local model prediction value and the sample labels, a second-party random number encrypted value is generated by encrypting the second-party local model prediction value and the sample labels. This second-party random number encrypted value is then exchanged with the first device to obtain the first-party random number encrypted value. Specifically, based on a preset second-party random number mixing method, a preset second-party random number, sample labels, and homomorphically encrypted second-party local model prediction values ​​are mixed to obtain the second-party random number encrypted value. This preset second-party random number mixing method includes summation and product methods. Then, based on the second-party random number encrypted value, an encryption value exchange is performed with the first device to send the second-party random number encrypted value to the first device and receive the first-party random number encrypted value sent by the first device. The first device obtains a common sample ID consistent with that of the second device, and then obtains the... The first-party local sample corresponding to the public sample ID and the first-party local model parameters corresponding to the first-party training local model parameters are used to calculate the product of the first-party local model parameters and the first-party local sample to obtain the first-party local model prediction value. Based on a preset first-party random number mixing method, the preset first-party random number and the homomorphically encrypted first-party local model prediction value are mixed to obtain the first-party random number encrypted value. The preset first-party random number mixing method includes summation and product, etc. For example, assuming the first-party local model prediction value is [[WX]], the preset first-party random number mixing method is summation, the preset first-party random number is r1, and the first-party random number encrypted value is [[WX]]+r1. Assuming the preset second-party random number mixing method is summation and difference, the second local model prediction value is [[WX2]], the sample label is y, the preset second-party random number is r2, and the second-party random number encrypted value is 1 / 4[[WX2]]-1 / 2y+r2.

[0122] The step of exchanging encrypted values ​​of a second-party random number generated by encrypting the second-party local model prediction value and the sample label with the first device to obtain a first-party random number encrypted value includes:

[0123] Step B21: Based on the public key in the preset second-party random number and the preset first-party key pair, the second-party local model prediction value and the sample label are encrypted with random numbers to obtain the encrypted value of the second-party random number.

[0124] In this embodiment, it should be noted that the second device generates a preset second-party key pair and shares the public key in the preset second-party key pair with the first device, while the first device generates a preset first-party key pair and shares the public key in the preset first-party key pair with the second device.

[0125] Based on a preset second-party random number and the public key in a preset first-party key pair, the predicted value of the second-party local model and the sample label are encrypted with random numbers to obtain a second-party random number encrypted value. Specifically, if the predicted value of the second-party local model is in a homomorphic encryption state, where the predicted value is encrypted based on the public key in the preset first-party key pair, and the iteration round of federated learning is greater than 1, then based on a preset second random number mixing method, the predicted value of the second-party local model, the preset second-party random number, and the sample label are mixed to obtain a second-party random number encrypted value. If the predicted value of the second-party local model is not in a homomorphic encryption state, and the iteration round of federated learning is equal to 1, then based on the public key in the preset first-party key pair, the predicted value of the second-party local model is homomorphically encrypted to obtain a homomorphically encrypted second-party local model predicted value. Then, based on a preset second random number mixing method, the preset first-party random number, the sample label, and the homomorphically encrypted first-party local model predicted value are mixed to obtain a second-party random number encrypted value.

[0126] The step of encrypting the second-party local model prediction value and the sample label with a random number based on a preset second-party random number and the public key of a preset first-party key pair to obtain the encrypted value of the second-party random number includes:

[0127] Step B211: Based on the public key in the preset first-party key pair, homomorphically encrypt the second-party local model prediction value to obtain the second-party homomorphically encrypted model prediction value.

[0128] In this embodiment, it should be noted that the iteration round of federated learning is 1, and the prediction value of the second-party homomorphic encryption model is the prediction value of the second-party local model in the homomorphic encryption state.

[0129] Step B212: Mix the predicted value of the second homomorphic encryption model, the sample label, and the preset second random number to generate the encrypted value of the second random number.

[0130] In this embodiment, the predicted value of the second-party homomorphic encryption model, the sample label, and the preset second-party random number are mixed to generate a second-party random number encrypted value. Specifically, the difference between the predicted value of the second-party homomorphic encryption model at a first preset multiple and the sample label at a preset second multiple is calculated to obtain the difference calculation result. Then, the second-party random number encrypted value is generated by calculating the sum of the difference calculation result and the preset second-party random number. The specific process for generating the second-party random number encrypted value is as follows:

[0131]

[0132] Where N is the encrypted value of the second-party random number, [[WX2]] is the predicted value of the second-party homomorphic encryption model, y is the sample label, and r2 is the preset second-party random number. The first preset multiple, It is the second preset multiple.

[0133] Step B22: Based on the encrypted value of the second random number, exchange encrypted values ​​with the first device to obtain the encrypted value of the first random number.

[0134] In this embodiment, based on the encrypted value of the second random number, the first device exchanges encrypted values ​​to obtain the encrypted value of the first random number. Specifically, the second device sends the encrypted value of the second random number to the first device and receives the encrypted value of the first random number sent by the first device. The first device sends the encrypted value of the first random number to the second device and receives the encrypted value of the second random number sent by the second device.

[0135] Step B30: The first random number encryption intermediate parameter generated based on the first random number encryption value is exchanged with the first device to obtain the second random number encryption intermediate parameter. The second random number encryption intermediate parameter is generated by the first device based on the second random number encryption value, so that the first device can optimize the first local model to be trained based on the first random number encryption intermediate parameter to obtain the first federated model.

[0136] In this embodiment, a first-party random number encryption intermediate parameter, generated based on the first-party random number encryption value, is exchanged with the first device to obtain a second-party random number encryption intermediate parameter. This second-party random number encryption intermediate parameter is generated by the first device based on the second-party random number encryption value. The first device uses this parameter to optimize the first-party local model to be trained, obtaining a first-party federated model. Specifically, the first-party random number encryption value is decrypted using the private key in a preset second-party key pair to obtain a first-party random number mixture value. Then, based on the first-party random number mixture value and the second-party local model prediction value, a first-party random number mixture intermediate parameter is calculated. Finally, based on the public key in the preset second-party key pair, the first-party random number mixture intermediate parameter is homomorphically encrypted to obtain the first-party random number encryption intermediate parameter. This first-party random number encryption intermediate parameter is then exchanged with the first device to obtain the second-party random number encryption intermediate parameter. The first random number encryption intermediate parameter is sent to the first device, and the first device receives the second random number encryption intermediate parameter sent by the first device. The first device receives the first random number encryption intermediate parameter sent by the second device, performs random number elimination on the first random number intermediate parameter, obtains the first encrypted intermediate parameter, and then generates the gradient of the first encrypted model by calculating the product of the first encrypted intermediate parameter and the first local sample. Based on the gradient of the first encrypted model, the first local model to be trained is optimized to obtain the first federated model. The first device decrypts the second random number encryption value based on the private key in the preset first key pair to obtain the second random number mixture value. Based on the second random number mixture value and the prediction value of the first local model, the first device calculates the second random number mixture intermediate parameter, and performs homomorphic encryption on the second random number mixture intermediate parameter based on the public key in the preset first key pair to obtain the second random number encryption intermediate parameter.

[0137] The step of exchanging the first-party random number encryption intermediate parameter generated based on the first-party random number encryption value with the first device to obtain the second-party random number encryption intermediate parameter includes:

[0138] Step B31: Based on the private key and public key in the preset second-party key pair, the first-party random number encryption value, the second-party local model prediction value, and the sample label, generate the first-party random number encryption intermediate parameters;

[0139] In this embodiment, based on the private and public keys in the preset second-party key pair, the encrypted value of the first-party random number, the predicted value of the second-party local model, and the sample label, a first-party random number encryption intermediate parameter is generated. Specifically, based on the private key in the preset second-party key pair, the encrypted value of the first-party random number is decrypted to obtain a first-party random number mixture value. Then, based on the first-party random number mixture value, the second-party local sample corresponding to the predicted value of the second-party local model, and the sample label corresponding to the second-party local sample, a first-party random number mixture intermediate parameter is calculated. Finally, based on the public key in the preset second-party key pair, the first-party random number mixture intermediate parameter is homomorphically encrypted to generate the first-party random number encryption intermediate parameter.

[0140] The step of generating intermediate parameters for first-party random number encryption based on the private and public keys in the preset second-party key pair, the encrypted value of the first-party random number, the predicted value of the second-party local model, and the sample label includes:

[0141] Step B311: Based on the private key in the preset second-party key pair, decrypt the encrypted value of the first-party random number to obtain the mixed value of the first-party random number.

[0142] In this embodiment, it should be noted that the first random number encryption value is a homomorphically encrypted first random number mixture value.

[0143] Step B312: Based on the second-party local model prediction value, the first-party random number mixture value, and the sample label, generate the first-party random number mixture intermediate parameters;

[0144] In this embodiment, based on the second-party local model prediction value, the first-party random number mixture value, and the sample labels, a first-party random number mixture intermediate parameter is generated. Specifically, the sum of the second-party local model prediction value and the first-party random number mixture value is calculated to obtain a summation result. Then, the difference between the summation result of a first preset multiple and the sample labels of a preset second multiple is calculated to generate the first-party random number mixture intermediate parameter. The specific process for generating the first-party random number mixture intermediate parameter is as follows:

[0145]

[0146] Wherein, d is the intermediate parameter of the first random number mixture, WX1 is the predicted value of the first local model, WX2 is the predicted value of the second local model, y is the sample label, r1 is the preset first random number, and the first random number mixture value is WX1+r1.

[0147] Step B313: Based on the public key in the preset first-party key pair, encrypt the first-party random number mixing intermediate parameter to obtain the first-party random number encryption intermediate parameter.

[0148] In this embodiment, it should be noted that the private key in the preset second-party key pair is held only by the second device, while both the first device and the second device hold the public key in the preset second-party key pair.

[0149] Step B32: Based on the first party's random number encrypted intermediate parameters, exchange encrypted intermediate parameters with the first device to obtain the second party's random number encrypted intermediate parameters.

[0150] In this embodiment, based on the first party's random number encryption intermediate parameters, the first device exchanges encryption intermediate parameters to obtain the second party's random number encryption intermediate parameters. Specifically, the second device sends the first party's random number encryption intermediate parameters to the first device for the first device to receive them, and the first device sends the second party's random number encryption intermediate parameters to the second device for the second device to receive them. It should be noted that since the second party's random number encryption intermediate parameters are encrypted based on the public key in the preset first party key, and the private key in the preset first party key is held only by the first device, the second device cannot decrypt the second party's random number encryption intermediate parameters. Similarly, since the first party's random number encryption intermediate parameters are encrypted based on the public key in the preset second party key, and the private key in the preset second party key is held only by the second device, the first device cannot decrypt the first party's random number encryption intermediate parameters.

[0151] Step B40: encrypt intermediate parameters based on the second party's random number, optimize the second party's local model to be trained, and obtain the second party's federated model.

[0152] In this embodiment, the second-party local model to be trained is optimized based on the second-party random number encrypted intermediate parameters to obtain a second-party federated model. Specifically, the second-party random number encrypted intermediate parameters are randomized to obtain second-party encrypted intermediate parameters. Then, the model parameters of the second-party local model to be trained are updated based on the second-party encrypted model gradient generated by the second-party encrypted intermediate parameters to obtain a second-party federated model.

[0153] The step of encrypting intermediate parameters based on the second-party random number, optimizing the second-party local model to be trained, and obtaining the second-party federated model includes:

[0154] Step B41: Perform random number elimination on the second-party random number encrypted intermediate parameter to obtain the second-party encrypted intermediate parameter;

[0155] In this embodiment, the second-party random number encryption intermediate parameter is subjected to random number elimination to obtain the second-party encrypted intermediate parameter. Specifically, the difference between the second-party random number encryption intermediate parameter and a preset second-party random number is calculated to eliminate the random number in the second-party random number encryption intermediate parameter, thereby obtaining the second-party encrypted intermediate parameter. The process of eliminating the random number in the second-party random number encryption intermediate parameter is as follows:

[0156] [[d]]2=[[d2]]-r2

[0157]

[0158] Wherein, [[d]]2 is the second-party encrypted intermediate parameter, [[d2]] is the second-party random number encrypted intermediate parameter, WX1 is the first-party local model prediction value, WX2 is the second-party local model prediction value, y is the sample label, and r2 is the preset second-party random number.

[0159] Step B42: Generate a second-party encrypted gradient based on the second-party encrypted intermediate parameters and the second-party local samples corresponding to the predicted values ​​of the second-party local model;

[0160] In this embodiment, a second-party encrypted gradient is generated based on the second-party encrypted intermediate parameters and the second-party local samples corresponding to the predicted values ​​of the second-party local model. Specifically, the second-party encrypted gradient is generated by calculating the product between the second-party encrypted intermediate parameters and the second-party local samples corresponding to the predicted values ​​of the second-party local model. The specific process for generating the second-party encrypted gradient is as follows:

[0161] [[g]]2=[[d]]2*X

[0162] Where [[g]]2 is the second-party encryption gradient, [[d]]2 is the second-party encryption intermediate parameter, and X is the second-party local sample.

[0163] Step B43: Based on the second-party encryption gradient, optimize the second-party local model to be trained to obtain the second-party federated model.

[0164] In this embodiment, the second-party local model to be trained is optimized based on the second-party encryption gradient to obtain the second-party federated model. Specifically, based on the second-party encryption gradient, the model parameters of the second-party local model to be trained are updated using a preset model parameter update method to obtain the second-party federated model. The preset model parameter update method includes gradient descent and gradient ascent, etc.

[0165] The step of optimizing the second-party local model to be trained based on the second-party encryption gradient to obtain the second-party federated model includes:

[0166] Step B431: Based on the second-party encrypted gradient, update the model parameters of the second-party local model to be trained, and determine whether the second-party local model to be trained after the model parameter update meets the preset training termination condition.

[0167] In this embodiment, it should be noted that the preset training end includes loss convergence and reaching the maximum number of iterations.

[0168] Step B432: If satisfied, the second-party local model to be trained with the updated model parameters is used as the second-party federated model.

[0169] If step B433 is not satisfied, then return to the step of obtaining the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.

[0170] In this embodiment, if the second-party local model to be trained after model parameter update meets the preset training termination condition, then the second-party local model to be trained after model parameter update is directly used as the second-party federated model. If the second-party local model to be trained after model parameter update does not meet the preset training termination condition, then the step of obtaining the second-party local model prediction value and the sample label corresponding to the second-party local model prediction value is returned.

[0171] Throughout the federated learning process in this application embodiment, the first-party random number encrypted value, the second-party random number encrypted value, the first-party random number encrypted intermediate parameter, and the second-party random number encrypted intermediate parameter exchanged between the first device and the second device are all in a homomorphic encryption state. While the second device can decrypt the first-party random number encrypted value, the privacy of the data is still guaranteed due to the mixing of random numbers. Similarly, while the first device can decrypt the second-party random number encrypted value, the privacy of the data is also guaranteed due to the mixing of random numbers. The first-party random number encrypted intermediate parameter held by the first device is encrypted using the public key in a preset second-party key pair. Unable to decrypt, the second device holds intermediate parameters encrypted with a second-party random number, which are encrypted using the public key in the preset first-party key pair. The second device cannot decrypt these parameters. Furthermore, throughout the federated learning process, the gradients and model parameters in the models of both the first and second devices are in a homomorphic encrypted state. Neither the first nor the second device can decrypt their own data, thus ensuring data privacy during the federated learning process. Since the data transmitted between the first and second devices is encrypted, neither device needs the other's assistance to decrypt the data during the federated learning process. Consequently, there is no decryption interface during the entire federated learning process, thus enhancing the security of the federated learning process.

[0172] This application provides a federated learning modeling optimization method. Compared to existing technologies where the label owner typically sends homomorphically encrypted labels to the unlabeled feature owner, and the feature owner calculates the homomorphically encrypted gradient and loss based on the labels, then sends these to the label owner, who decrypts them and feeds back the plaintext model gradient and loss to the feature owner for model updates, this application first obtains the second-party local model prediction value corresponding to the second-party local model to be trained and the sample labels corresponding to the second-party local model prediction value. Then, based on the second-party local model prediction value and the sample labels, a second-party random number encrypted value is generated by encrypting the second-party local model prediction value and the sample labels, and exchanged with a first device to obtain a first-party random number encrypted value. The first-party random number encrypted value is obtained by the first device encrypting the first-party local model prediction value corresponding to the first-party local model to be trained, so that the first device can obtain the second-party random number encrypted value. The system uses encrypted data and then generates encrypted intermediate parameters based on the first random number encrypted value. These intermediate parameters are exchanged with the first device to obtain encrypted intermediate parameters based on the second random number encrypted value. The second random number encrypted intermediate parameters are generated by the first device based on the second random number encrypted value. The first device uses these encrypted intermediate parameters to optimize the first local model to be trained, obtaining a first federated model. Then, based on the second random number encrypted intermediate parameters, the first device optimizes the second local model to be trained, obtaining a second federated model. Throughout the entire interaction process of building the federated model, all data exchanged between the first and second devices is encrypted. Therefore, this embodiment does not have a decryption interface during the federated model construction process. This overcomes the technical defect in the prior art where the feature owner can obtain the privacy data of the tag owner through the decryption interface during the entire federated learning process of model construction, leading to the leakage of sample tags, which are the privacy data of the tag owner. Therefore, the security of building models based on federated learning is improved.

[0173] Furthermore, referring to Figure 6 Based on the third embodiment of this application, in another embodiment of this application, the first-party federated model includes first-party encryption model parameters encrypted with the public key of a preset second-party key pair, and the second-party federated model includes second-party encryption model parameters encrypted with the public key of a preset first-party key pair.

[0174] After the step of encrypting intermediate parameters based on the second-party random number, optimizing the second-party local model to be trained, and obtaining the second-party federated model, the federated learning modeling optimization method further includes:

[0175] Step C10: Obtain the second-party sample to be predicted, and generate the output value of the second-party encrypted model based on the second-party sample to be predicted and the parameters of the second-party encrypted model.

[0176] In this embodiment, it should be noted that the second-party encryption model parameters are model parameters encrypted based on the public key in the preset first-party key pair.

[0177] A second-party sample to be predicted is obtained, and a second-party encrypted model output value is generated based on the second-party sample to be predicted and the second-party encrypted model parameters. Specifically, the second-party sample to be predicted is obtained, and the second-party encrypted model output value is obtained by calculating the product between the second-party sample to be predicted and the second-party encrypted model parameters.

[0178] Step C20: Send the output value of the second-party encryption model and the target sample ID of the second-party sample to be predicted to the first device, so that the first device can generate the output value of the first-party encryption model based on the first-party sample to be predicted corresponding to the target sample ID and the parameters of the first-party encryption model, and decrypt the output value of the second-party encryption model based on the private key in the preset first-party key pair to obtain the output value of the second-party model.

[0179] In this embodiment, the output value of the second-party encryption model and the target sample ID of the second-party predictable sample are sent to the first device. The first device generates the output value of the first-party encryption model based on the first-party predictable sample corresponding to the target sample ID and the parameters of the first-party encryption model. The first device then decrypts the output value of the second-party encryption model based on the private key in the preset first-party key pair to obtain the second-party model output value. Specifically, the output value of the second-party encryption model and the target sample ID of the second-party predictable sample are sent to the first device. The first device obtains the first-party predictable sample corresponding to the target sample ID, calculates the product between the first-party predictable sample and the parameters of the first-party encryption model to obtain the output value of the first-party encryption model, and decrypts the output value of the second-party encryption model based on the private key in the preset first-party key pair to obtain the second-party model output value. Then, the first device sends the output value of the second-party model and the output value of the first-party encryption model to the second device.

[0180] Step C30: Receive the first-party encryption model output value and the second-party model output value sent by the first device, and decrypt the first-party encryption model output value based on the private key in the preset second-party key pair to obtain the first-party model output value;

[0181] In this embodiment, it should be noted that since the parameters of the first-party encryption model are encrypted based on the public key in the preset second key pair, the output value of the first-party encryption model can be decrypted based on the private key in the preset second key pair to obtain the output value of the first-party model.

[0182] Step C40: Generate federated prediction results based on the output values ​​of the first-party model and the second-party model.

[0183] In this embodiment, a federated prediction result is generated based on the output values ​​of the first-party model and the second-party model. Specifically, the output values ​​of the first-party model and the second-party model are aggregated based on a preset aggregation method to obtain the federated prediction result. The preset aggregation method includes weighted summation and weighted average, etc. Figure 7 The diagram shows a flowchart of the second device generating a federated prediction result in an embodiment of this application. In this flowchart, A is the first device, B is the second device, x1, x2, x3, x4, and x5 are all features of the sample, y is the sample label, [[w]] in A are the parameters of the first-party encryption model, [[w]]*X in A are the output value of the first-party encryption model (i.e., [[w]]*X_a), [[w]] in B are the parameters of the second-party encryption model, [[w]]*X in B are the output value of the second-party encryption model (i.e., [[w]]*X_b), ID is the target sample ID, and the complete wx represents the federated prediction result, which is then converted into a prediction score.

[0184] This application provides a federated prediction method. Since the first and second federated models are constructed based on a federated learning process without a decryption interface, the model parameters in both the first and second devices are in a homomorphic encrypted state. This allows the acquisition of a second-party sample to be predicted. Based on the second-party sample and the second-party encrypted model parameters, a second-party encrypted model output value is generated. The second-party encrypted model output value and the target sample ID of the second-party sample to be predicted are then sent to the first device. The first device then generates a first-party encrypted model output based on the first-party sample to be predicted corresponding to the target sample ID and the first-party encrypted model parameters. The system receives the first-party encrypted model output value and the second-party encrypted model output value sent by the first device. Based on the private key in the preset first-party key pair, it decrypts the first-party encrypted model output value to obtain the second-party model output value. Then, it receives the first-party encrypted model output value and the second-party model output value sent by the first device, and decrypts the first-party encrypted model output value based on the private key in the preset second-party key pair to obtain the first-party model output value. Based on the first-party model output value and the second-party model output value, it generates a federated prediction result, thus achieving the purpose of federated prediction based on a federated model without a decryption interface, thereby achieving the purpose of federated prediction based on a more secure federated model and improving the security of federated prediction.

[0185] Reference Figure 8 , Figure 8 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiments of this application.

[0186] like Figure 8 As shown, the federated learning modeling optimization device may include: a processor 1001, such as a CPU, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between the processor 1001 and the memory 1005. The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0187] Optionally, the federated learning modeling optimization device may also include a rectangular user interface, a network interface, a camera, RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc. The rectangular user interface may include a display screen and an input submodule such as a keyboard; optionally, the rectangular user interface may also include standard wired or wireless interfaces. The network interface may optionally include standard wired or wireless interfaces (such as a Wi-Fi interface).

[0188] Those skilled in the art will understand that Figure 8 The structure of the federated learning modeling optimization device shown in the figure does not constitute a limitation on the federated learning modeling optimization device. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0189] like Figure 8 As shown, the memory 1005, as a computer storage medium, may include an operating system, a network communication module, and a federated learning modeling optimization program. The operating system is a program that manages and controls the hardware and software resources of the federated learning modeling optimization device, supporting the operation of the federated learning modeling optimization program and other software and / or programs. The network communication module is used to enable communication between the various components within the memory 1005, as well as communication with other hardware and software in the federated learning modeling optimization system.

[0190] exist Figure 8 In the federated learning modeling optimization device shown, the processor 1001 is used to execute the federated learning modeling optimization program stored in the memory 1005 to implement the steps of the federated learning modeling optimization method described above.

[0191] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0192] This application embodiment also provides a federated learning modeling optimization device, which is applied to a first device, and the federated learning modeling optimization device includes:

[0193] The acquisition module is used to acquire the first-party local model prediction value corresponding to the first-party local model to be trained.

[0194] The encrypted value exchange module is used to exchange encrypted values ​​with the second device based on the first random number encrypted value generated by random number encryption of the first local model prediction value, and to obtain the second random number encrypted value. The second random number encrypted value is obtained by the second device by random number encryption of the second local model prediction value and sample label corresponding to the second local model to be trained, so that the second device can obtain the first random number encrypted value.

[0195] The encrypted intermediate parameter exchange module is used to exchange encrypted intermediate parameters generated based on the encrypted value of the second random number with the second device to obtain encrypted intermediate parameters of the first random number. The encrypted intermediate parameters of the first random number are generated by the second device based on the encrypted value of the first random number, so that the second device can optimize the second local model to be trained based on the encrypted intermediate parameters of the second random number to obtain the second federated model.

[0196] The optimization module is used to encrypt intermediate parameters based on the first party's random number, optimize the first party's local model to be trained, and obtain the first party's federated model.

[0197] Optionally, the encrypted value exchange module is further configured to:

[0198] Based on the public key in the preset first-party random number and the preset second-party key pair, the first-party local model prediction value is encrypted with a random number to obtain the first-party random number encrypted value.

[0199] Based on the encrypted value of the first random number, the encrypted value is exchanged with the second device to obtain the encrypted value of the second random number.

[0200] Optionally, the first party local model prediction value is homomorphically encrypted based on the public key in the preset second party key pair to obtain the first party homomorphically encrypted model prediction value.

[0201] The predicted value of the first homomorphic encryption model is mixed with the preset first random number to generate the encrypted value of the first random number.

[0202] Optionally, the encrypted intermediate parameter exchange module is further configured to:

[0203] Based on the private and public keys in the preset first-party key pair, the encrypted value of the second-party random number, and the predicted value of the first-party local model, the intermediate parameters for the encryption of the second-party random number are generated.

[0204] Based on the second party's random number encrypted intermediate parameters, the first party's random number encrypted intermediate parameters are obtained by exchanging encrypted intermediate parameters with the second device.

[0205] Optionally, the encrypted intermediate parameter exchange module is further configured to:

[0206] Based on the private key in the preset first-party key pair, the encrypted value of the second-party random number is decrypted to obtain the mixed value of the second-party random number;

[0207] Based on the first local model prediction value and the second random number mixture value, generate intermediate parameters of the second random number mixture;

[0208] Based on the public key in the preset first-party key pair, the second-party random number mixing intermediate parameter is encrypted to obtain the second-party random number encryption intermediate parameter.

[0209] Optionally, the optimization module is further configured to:

[0210] Random number elimination is performed on the first party's encrypted intermediate parameters to obtain the first party's encrypted intermediate parameters;

[0211] A first-party encrypted gradient is generated based on the first-party encrypted intermediate parameters and the first-party local sample corresponding to the first-party local model prediction value.

[0212] Based on the first party's encryption gradient, the first party's local model to be trained is optimized to obtain the first party's federated model.

[0213] Optionally, the optimization module is further configured to:

[0214] Based on the first-party encrypted gradient, the model parameters of the first-party local model to be trained are updated, and it is determined whether the first-party local model to be trained after the model parameter update meets the preset training termination condition.

[0215] If the conditions are met, the first-party local model to be trained after the model parameters are updated will be used as the first-party federated model.

[0216] If the condition is not met, return to the step of obtaining the first-party local model prediction value corresponding to the first-party local model to be trained.

[0217] Optionally, the federated learning modeling optimization device is further used for:

[0218] Obtain a first-party sample to be predicted, and generate the output value of the first-party encrypted model based on the first-party sample to be predicted and the parameters of the first-party encrypted model;

[0219] The first party encryption model output value and the target sample ID of the first party to be predicted sample are sent to the second device, so that the second device can generate the second party encryption model output value based on the second party to be predicted sample corresponding to the target sample ID and the second party encryption model parameters, and decrypt the first party encryption model output value based on the private key in the preset second party key pair to obtain the first party model output value.

[0220] Receive the second-party encryption model output value and the first-party model output value sent by the second device, and decrypt the second-party encryption model output value based on the private key in the preset first-party key pair to obtain the second-party model output value;

[0221] Based on the output values ​​of the first-party model and the second-party model, federated prediction results are generated.

[0222] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0223] This application embodiment also provides a federated learning modeling optimization device, which is applied to a second device, and the federated learning modeling optimization device includes:

[0224] The acquisition module is used to acquire the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.

[0225] The encrypted value exchange module is used to exchange encrypted values ​​with the first device based on the second-party random number encrypted value generated by random number encryption of the second-party local model prediction value and the sample label, so as to obtain the first-party random number encrypted value. The first-party random number encrypted value is obtained by the first device by random number encryption of the first-party local model prediction value corresponding to the first-party local model to be trained, so as to allow the first device to obtain the second-party random number encrypted value.

[0226] An encrypted intermediate parameter exchange module is used to exchange encrypted intermediate parameters generated based on the encrypted value of the first random number with the first device to obtain encrypted intermediate parameters of the second random number. The encrypted intermediate parameters of the second random number are generated by the first device based on the encrypted value of the second random number, so that the first device can optimize the first local model to be trained based on the encrypted intermediate parameters of the first random number to obtain the first federated model.

[0227] The optimization module is used to encrypt intermediate parameters based on the second-party random number, optimize the second-party local model to be trained, and obtain the second-party federated model.

[0228] Optionally, the encrypted value exchange module is further configured to:

[0229] Based on the public key in the preset second-party random number and the preset first-party key pair, the second-party local model prediction value and the sample label are encrypted with random numbers to obtain the encrypted value of the second-party random number.

[0230] Based on the encrypted value of the second random number, the encrypted value is exchanged with the first device to obtain the encrypted value of the first random number.

[0231] Optionally, the encrypted value exchange module is further configured to:

[0232] Based on the public key in the preset first-party key pair, the second-party local model prediction value is homomorphically encrypted to obtain the second-party homomorphically encrypted model prediction value.

[0233] The predicted value of the second-party homomorphic encryption model, the sample label, and the preset second-party random number are mixed to generate the encrypted value of the second-party random number.

[0234] Optionally, the encrypted intermediate parameter exchange module is further configured to:

[0235] Based on the private and public keys in the preset second-party key pair, the encrypted value of the first-party random number, the predicted value of the second-party local model, and the sample label, the first-party random number encrypted intermediate parameters are generated.

[0236] Based on the first party's random number encrypted intermediate parameters, the second party's random number encrypted intermediate parameters are obtained by exchanging encrypted intermediate parameters with the first device.

[0237] Optionally, the encrypted intermediate parameter exchange module is further configured to:

[0238] Based on the private key in the preset second-party key pair, the encrypted value of the first-party random number is decrypted to obtain the mixed value of the first-party random number;

[0239] Based on the second-party local model prediction value, the first-party random number mixture value, and the sample label, generate the first-party random number mixture intermediate parameters;

[0240] Based on the public key in the preset first-party key pair, the first-party random number mixing intermediate parameter is encrypted to obtain the first-party random number encryption intermediate parameter.

[0241] Optionally, the optimization module is further configured to:

[0242] Random number elimination is performed on the second-party encrypted intermediate parameter to obtain the second-party encrypted intermediate parameter;

[0243] A second-party encrypted gradient is generated based on the second-party encrypted intermediate parameters and the second-party local samples corresponding to the second-party local model prediction values.

[0244] Based on the second-party encryption gradient, the second-party local model to be trained is optimized to obtain the second-party federated model.

[0245] Optionally, the optimization module is further configured to:

[0246] Based on the second-party encrypted gradient, the model parameters of the second-party local model to be trained are updated, and it is determined whether the second-party local model to be trained after the model parameter update meets the preset training termination condition.

[0247] If the conditions are met, the second-party local model to be trained after the model parameters are updated will be used as the second-party federated model.

[0248] If the conditions are not met, return to the step of obtaining the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.

[0249] Optionally, the federated learning modeling optimization device is further used for:

[0250] Obtain the second-party sample to be predicted, and generate the output value of the second-party encrypted model based on the second-party sample to be predicted and the parameters of the second-party encrypted model;

[0251] The output value of the second-party encryption model and the target sample ID of the second-party sample to be predicted are sent to the first device, so that the first device can generate the output value of the first-party encryption model based on the first-party sample to be predicted corresponding to the target sample ID and the parameters of the first-party encryption model, and decrypt the output value of the second-party encryption model based on the private key in the preset first-party key pair to obtain the output value of the second-party model.

[0252] The device receives the first-party encrypted model output value and the second-party model output value sent by the first device, and decrypts the first-party encrypted model output value based on the private key in the preset second-party key pair to obtain the first-party model output value.

[0253] Based on the output values ​​of the first-party model and the second-party model, federated prediction results are generated.

[0254] The specific implementation of the federated learning modeling optimization device in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0255] This application provides a medium that is a readable storage medium, and the readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the federated learning modeling optimization method described in any of the above claims.

[0256] The specific implementation of the readable storage medium in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0257] This application provides a computer program product, which includes one or more computer programs. The one or more computer programs can be executed by one or more processors to implement the steps of the federated learning modeling optimization method described above.

[0258] The specific implementation of the computer program product in this application is basically the same as the embodiments of the federated learning modeling optimization method described above, and will not be repeated here.

[0259] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. A federated learning modeling optimization method, characterized in that, The federated learning modeling optimization method is applied to the first device, and the federated learning modeling optimization method includes: Obtain the first-party local model prediction value corresponding to the first-party local model to be trained; Based on the first random number encrypted value generated by encrypting the first local model prediction value, the second random number encrypted value is exchanged with the second device to obtain the second random number encrypted value. The second random number encrypted value is obtained by the second device by encrypting the second local model prediction value and sample label corresponding to the second local model to be trained, so that the second device can obtain the first random number encrypted value. Based on the private key in the preset first-party key pair, the encrypted value of the second-party random number is decrypted to obtain a mixed value of the second-party random number; based on the predicted value of the first-party local model and the mixed value of the second-party random number, intermediate parameters of the second-party random number mixture are generated; based on the public key in the preset first-party key pair, the intermediate parameters of the second-party random number mixture are encrypted to obtain encrypted intermediate parameters of the second-party random number; based on the encrypted intermediate parameters of the second-party random number, encrypted intermediate parameters are exchanged with the second device to obtain encrypted intermediate parameters of the first-party random number, wherein the encrypted intermediate parameters of the first-party random number are generated by the second device based on the encrypted value of the first-party random number, so that the second device can optimize the second-party local model to be trained based on the encrypted intermediate parameters of the second-party random number to obtain the second-party federated model; Based on the intermediate parameters encrypted using the first-party random number, the first-party local model to be trained is optimized to obtain the first-party federated model.

2. The federated learning modeling optimization method as described in claim 1, characterized in that, The step of exchanging a first-party random number encrypted value, generated by encrypting the first-party local model prediction value with a second device to obtain a second-party random number encrypted value includes: Based on the public key in the preset first-party random number and the preset second-party key pair, the first-party local model prediction value is encrypted with a random number to obtain the first-party random number encrypted value. Based on the encrypted value of the first random number, the encrypted value is exchanged with the second device to obtain the encrypted value of the second random number.

3. The federated learning modeling optimization method as described in claim 2, characterized in that, The step of encrypting the first-party local model prediction value with a random number based on the public key of a preset first-party random number and a preset second-party key pair to obtain the encrypted first-party random number value includes: Based on the public key in the preset second-party key pair, the first-party local model prediction value is homomorphically encrypted to obtain the first-party homomorphically encrypted model prediction value. The predicted value of the first homomorphic encryption model is mixed with the preset first random number to generate the encrypted value of the first random number.

4. The federated learning modeling and optimization method as described in claim 1, characterized in that, The step of encrypting intermediate parameters based on the first party's random number, optimizing the first party's local model to be trained, and obtaining the first party's federated model includes: Random number elimination is performed on the first party's encrypted intermediate parameters to obtain the first party's encrypted intermediate parameters; A first-party encrypted gradient is generated based on the first-party encrypted intermediate parameters and the first-party local sample corresponding to the first-party local model prediction value. Based on the first party's encryption gradient, the first party's local model to be trained is optimized to obtain the first party's federated model.

5. The federated learning modeling and optimization method as described in claim 4, characterized in that, The step of optimizing the first-party local model to be trained based on the first-party encryption gradient to obtain the first-party federated model includes: Based on the first-party encrypted gradient, the model parameters of the first-party local model to be trained are updated, and it is determined whether the first-party local model to be trained after the model parameter update meets the preset training termination condition. If the conditions are met, the first-party local model to be trained after the model parameters are updated will be used as the first-party federated model. If the condition is not met, return to the step of obtaining the first-party local model prediction value corresponding to the first-party local model to be trained.

6. The federated learning modeling optimization method as described in claim 1, characterized in that, The first-party federated model includes first-party encryption model parameters encrypted with the public key of a preset second-party key pair, and the second-party federated model includes second-party encryption model parameters encrypted with the public key of a preset first-party key pair. After the step of encrypting intermediate parameters based on the first party's random number, optimizing the first party's local model to be trained, and obtaining the first party's federated model, the federated learning modeling optimization method further includes: Obtain a first-party sample to be predicted, and generate the output value of the first-party encrypted model based on the first-party sample to be predicted and the parameters of the first-party encrypted model; The first party encryption model output value and the target sample ID of the first party to be predicted sample are sent to the second device, so that the second device can generate the second party encryption model output value based on the second party to be predicted sample corresponding to the target sample ID and the second party encryption model parameters, and decrypt the first party encryption model output value based on the private key in the preset second party key pair to obtain the first party model output value. Receive the second-party encryption model output value and the first-party model output value sent by the second device, and decrypt the second-party encryption model output value based on the private key in the preset first-party key pair to obtain the second-party model output value; Based on the output values ​​of the first-party model and the second-party model, federated prediction results are generated.

7. A federated learning modeling optimization method, characterized in that, The federated learning modeling optimization method is applied to the second device, and the federated learning modeling optimization method includes: Obtain the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value; Based on the second-party random number encryption value generated by encrypting the second-party local model prediction value and the sample label, the encryption value is exchanged with the first device to obtain the first-party random number encryption value. The first-party random number encryption value is obtained by the first device by encrypting the first-party local model prediction value corresponding to the first-party local model to be trained, so that the first device can obtain the second-party random number encryption value. Based on the private key in the preset second-party key pair, the encrypted value of the first-party random number is decrypted to obtain a mixed value of the first-party random number; based on the predicted value of the second-party local model, the mixed value of the first-party random number, and the sample label, intermediate parameters of the first-party random number mixture are generated; based on the public key in the preset first-party key pair, the intermediate parameters of the first-party random number mixture are encrypted to obtain encrypted intermediate parameters of the first-party random number; based on the encrypted intermediate parameters of the first-party random number, encrypted intermediate parameters are exchanged with the first device to obtain encrypted intermediate parameters of the second-party random number, wherein the encrypted intermediate parameters of the second-party random number are generated by the first device based on the encrypted value of the second-party random number, so that the first device can optimize the first-party local model to be trained based on the encrypted intermediate parameters of the first-party random number to obtain the first-party federated model; Based on the intermediate parameters encrypted using the second-party random number, the second-party local model to be trained is optimized to obtain the second-party federated model.

8. The federated learning modeling optimization method as described in claim 7, characterized in that, The step of exchanging encrypted values ​​of a second-party random number generated by encrypting the second-party local model prediction value and the sample label with the first device to obtain the first-party random number encrypted value includes: Based on the public key in the preset second-party random number and the preset first-party key pair, the second-party local model prediction value and the sample label are encrypted with random numbers to obtain the encrypted value of the second-party random number. Based on the encrypted value of the second random number, the encrypted value is exchanged with the first device to obtain the encrypted value of the first random number.

9. The federated learning modeling optimization method as described in claim 8, characterized in that, The step of encrypting the second-party local model prediction value and the sample label with a random number based on a preset second-party random number and the public key in a preset first-party key pair to obtain the encrypted value of the second-party random number includes: Based on the public key in the preset first-party key pair, the second-party local model prediction value is homomorphically encrypted to obtain the second-party homomorphically encrypted model prediction value. The predicted value of the second-party homomorphic encryption model, the sample label, and the preset second-party random number are mixed to generate the encrypted value of the second-party random number.

10. The federated learning modeling optimization method as described in claim 7, characterized in that, The steps of encrypting intermediate parameters based on the second-party random number, optimizing the second-party local model to be trained, and obtaining the second-party federated model include: Random number elimination is performed on the second-party encrypted intermediate parameter to obtain the second-party encrypted intermediate parameter; A second-party encrypted gradient is generated based on the second-party encrypted intermediate parameters and the second-party local samples corresponding to the second-party local model prediction values. Based on the second-party encryption gradient, the second-party local model to be trained is optimized to obtain the second-party federated model.

11. The federated learning modeling optimization method as described in claim 10, characterized in that, The step of optimizing the second-party local model to be trained based on the second-party encryption gradient to obtain the second-party federated model includes: Based on the second-party encrypted gradient, the model parameters of the second-party local model to be trained are updated, and it is determined whether the second-party local model to be trained after the model parameter update meets the preset training termination condition. If the conditions are met, the second-party local model to be trained after the model parameters are updated will be used as the second-party federated model. If the conditions are not met, return to the step of obtaining the second-party local model prediction value corresponding to the second-party local model to be trained and the sample label corresponding to the second-party local model prediction value.

12. The federated learning modeling optimization method as described in claim 7, characterized in that, The first-party federated model includes first-party encryption model parameters encrypted with the public key of a preset second-party key pair, and the second-party federated model includes second-party encryption model parameters encrypted with the public key of a preset first-party key pair. After the step of encrypting intermediate parameters based on the second-party random number, optimizing the second-party local model to be trained, and obtaining the second-party federated model, the federated learning modeling optimization method further includes: Obtain the second-party sample to be predicted, and generate the output value of the second-party encrypted model based on the second-party sample to be predicted and the parameters of the second-party encrypted model; The output value of the second-party encryption model and the target sample ID of the second-party sample to be predicted are sent to the first device, so that the first device can generate the output value of the first-party encryption model based on the first-party sample to be predicted corresponding to the target sample ID and the parameters of the first-party encryption model, and decrypt the output value of the second-party encryption model based on the private key in the preset first-party key pair to obtain the output value of the second-party model. The device receives the first-party encrypted model output value and the second-party model output value sent by the first device, and decrypts the first-party encrypted model output value based on the private key in the preset second-party key pair to obtain the first-party model output value. Based on the output values ​​of the first-party model and the second-party model, federated prediction results are generated.

13. A federated learning modeling optimization device, characterized in that, The federated learning modeling optimization device includes: a memory, a processor, and a program stored in the memory for implementing the federated learning modeling optimization method. The memory is used to store programs that implement federated learning modeling optimization methods; The processor is configured to execute a program that implements the federated learning modeling optimization method to carry out the steps of the federated learning modeling optimization method as described in any one of claims 1 to 6 or 7 to 12.

14. A medium, said medium being a readable storage medium, characterized in that, The readable storage medium stores a program that implements the federated learning modeling optimization method, the program being executed by a processor to implement the steps of the federated learning modeling optimization method as described in any one of claims 1 to 6 or 7 to 12.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the federated learning modeling optimization method as described in any one of claims 1 to 6 or 7 to 12.

Images