Federal learning linear model training method and device, computer equipment and storage medium

Abstract

The embodiment of the invention provides a federal learning linear model training method and device, computer equipment, a storage medium and a computer program product, and relates to the technical field of data processing. The method comprises the following steps: receiving homomorphic encryption public keys of a plurality of data providers from a model training system; receiving provider intermediate parameter fragment ciphertexts sent by the data providers, and obtaining blind ciphertexts corresponding to the data providers according to the provider intermediate parameter fragment ciphertexts and the homomorphic encryption public keys; sending the blind ciphertext to each data provider; obtaining a gradient plaintext and a loss update value of a current model parameter update round of a data user according to the provider quadratic multiplication blind fragment; and according to the gradient plaintext and the loss updating value, carrying out iterative updating on the current mixed-direction federated linear model to obtain a target mixed-direction federated linear model corresponding to the current mixed-direction federated linear model. According to the method, the data security and the calculation efficiency are improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, apparatus, computer device, storage medium and computer program product for training a federated learning linear model. Background Art

[0002] Federated Learning (FL) is a distributed machine learning framework that allows multiple participants to collaborate on machine learning model training without sharing original data. This learning method pays special attention to data privacy protection and security, allowing all parties to collaborate without disclosing private data to other participants. In some scenarios, federated learning can be combined with linear models to implement federated learning linear model training and use. The core of horizontal federated learning is the encrypted aggregation of local model parameters by weighted averaging. Since model parameters generally correspond to features (one feature corresponds to one or more model parameters), vertical federated learning is achieved by exchanging intermediate calculation results (such as gradients) calculated using only the local data of each party. There may be multiple different methods for the vertical federation of each local machine learning model (depending on the decomposition, fitting approximation and optimization of different function formulas).

[0003] In the current federated learning linear model training, it is limited to one-way federated learning (vertical federated learning or horizontal federated learning). In one-way federated learning, due to the loss of a large amount of information of misaligned sample data after sample alignment, it will lead to the inability to model or difficulty in establishing an effective model. In addition, a hybrid federated learning linear model can be introduced. In the current hybrid federated learning linear model training, there are problems of high security risks and low data security. Summary of the invention

[0004] Based on this, it is necessary to provide a federated learning linear model training method, apparatus, computer equipment, storage medium and computer program product to address the above technical problems.

[0005] In a first aspect, the present application provides a method for training a linear model of federated learning, which is applied to a data user of a model training system. The method comprises:

[0006] Receiving homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user;

[0007] Receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0008] Sending the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user;

[0009] Obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0010] The current mixed-directional federated linear model is iteratively updated according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0011] In one of the embodiments, the blinded ciphertext of the aggregated parameter fragment ciphertext corresponding to each data provider is obtained based on the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider, including: encrypting the user intermediate parameter fragments using the homomorphic encryption public key to obtain the user intermediate parameter fragment ciphertext corresponding to the user intermediate parameter fragment; aggregating the user intermediate parameter fragment ciphertext with the provider intermediate parameter fragment ciphertext to obtain the aggregated parameter ciphertext and aggregated loss fragment ciphertext corresponding to each data provider; and obtaining the blinded ciphertext of the aggregated parameter fragment ciphertext based on the aggregated parameter ciphertext and the aggregated loss fragment ciphertext.

[0012] In one of the embodiments, obtaining the blinded ciphertext of the aggregation parameter fragment ciphertext based on the aggregation parameter ciphertext and the aggregation loss fragment ciphertext includes: obtaining the user gradient ciphertext corresponding to the data user based on the aggregation parameter ciphertext; obtaining the aggregation parameter fragment ciphertext based on the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext; and obtaining the blinded ciphertext based on the aggregation parameter fragment ciphertext.

[0013] In one of the embodiments, obtaining the blinded ciphertext based on the aggregation parameter fragment ciphertext includes: obtaining a random number and using the random number as a multiplication mask; blinding the aggregation parameter fragment ciphertext using the multiplication mask to obtain the blinded ciphertext corresponding to the aggregation parameter fragment ciphertext.

[0014] In one of the embodiments, obtaining the gradient plaintext and the loss update value of the current model parameter update round of the data user based on the provider's quadratic multiplication blinded fragments includes: obtaining the aggregated provider's quadratic multiplication blinded fragments corresponding to the provider's quadratic multiplication blinded fragments; obtaining the gradient plaintext and the loss update value of the current model parameter update round of the data user based on the aggregated provider's quadratic multiplication blinded fragments.

[0015] In one embodiment, the provider secondary multiplication blinded gradient fragments include provider secondary multiplication blinded gradient fragments, user gradient ciphertext blinded values ​​and loss fragment blinded values; the obtaining of aggregated provider secondary multiplication blinded fragments corresponding to the provider secondary multiplication blinded fragments includes: aggregating the provider secondary multiplication blinded gradient fragments of multiple data providers to obtain aggregated provider secondary multiplication blinded gradient fragments; aggregating the user gradient ciphertext blinded values ​​to obtain aggregated user gradient ciphertext blinded values; aggregating the loss fragment blinded values ​​to obtain aggregated loss fragment blinded values; obtaining the aggregated provider secondary multiplication blinded fragments according to the aggregated provider secondary multiplication blinded gradient fragments, the aggregated user gradient ciphertext blinded values ​​and the aggregated loss fragment blinded values.

[0016] In one of the embodiments, the obtaining of the gradient plaintext and the loss update value of the current model parameter update round of the data user according to the aggregated provider's secondary multiplication blinded fragments includes: derandomizing the aggregated provider's secondary multiplication blinded gradient fragments to obtain the provider gradient aggregation value corresponding to the aggregated provider's secondary multiplication blinded gradient fragments; derandomizing the aggregated user's gradient ciphertext blinded value to obtain the user's gradient aggregation value corresponding to the aggregated user's gradient ciphertext blinded value; derandomizing the aggregated loss fragment blinded value to obtain the loss aggregation value corresponding to the aggregated loss fragment blinded value; obtaining the gradient plaintext of the current model parameter update round according to the provider gradient aggregation value and the user gradient aggregation value; and obtaining the loss update value of the current model parameter update round according to the loss aggregation value.

[0017] In one of the embodiments, the gradient plaintext includes a user gradient plaintext and a provider gradient plaintext; obtaining the gradient plaintext of the current model parameter update round according to the provider gradient aggregation value and the user gradient aggregation value includes: obtaining the user gradient plaintext according to the user gradient aggregation value; sending the provider gradient aggregation value to each of the data providers; the data provider is used to perform secondary de-blinding on the provider gradient aggregation value to obtain the provider gradient plaintext.

[0018] In a second aspect, the present application provides a federated learning linear model training device, which is applied to a data user of a model training system. The device includes:

[0019] A first receiving module is used to receive homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key included in the homomorphic encryption key pair to the data user;

[0020] A second receiving module is used to receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0021] A sending module, used for sending the blinded ciphertext to each of the data providers; each of the data providers is used for obtaining the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user;

[0022] A calculation module, used to obtain the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0023] A training module is used to iteratively update the current mixed-directional federated linear model according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0024] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0025] Receiving homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user;

[0026] Receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0027] Sending the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user;

[0028] Obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0029] The current mixed-directional federated linear model is iteratively updated according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0030] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0031] Receiving homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user;

[0032] Receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0033] Sending the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user;

[0034] Obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0035] The current mixed-directional federated linear model is iteratively updated according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0036] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Receiving homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user;

[0038] Receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0039] Sending the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user;

[0040] Obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0041] The current mixed-directional federated linear model is iteratively updated according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0042] In the above-mentioned federated learning linear model training method, apparatus, computer equipment, storage medium and computer program product, first, homomorphic encryption public keys of multiple data providers from the model training system can be received; each data provider is used to generate a corresponding homomorphic encryption key pair, and the homomorphic encryption public key contained in the homomorphic encryption key pair is sent to the data user; next, the provider intermediate parameter fragment ciphertext sent by each data provider can be received, and according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider, the aggregate parameter fragment ciphertext corresponding to each data provider can be obtained. Blind ciphertext; further, the blind ciphertext can be sent to each data provider; each data provider is used to obtain the provider's quadratic multiplication blind fragments of each data provider according to the blind ciphertext, and send the provider's quadratic multiplication blind fragments to the data user; further, the gradient plaintext and loss update value of the current model parameter update round of the data user can be obtained according to the provider's quadratic multiplication blind fragments; finally, the current hybrid federated linear model can be iteratively updated according to the gradient plaintext and the loss update value to obtain the target hybrid federated linear model corresponding to the current hybrid federated linear model. In the method provided in the embodiment of the present application, horizontal federated learning and vertical federated learning can be combined to realize the training of the hybrid federated linear model of one-to-many isomorphic data providers, improve the feasibility of model training, and avoid the problem of sample data information loss, which can improve the data security in the data transmission and data processing process, and introduce multiple key homomorphic sharding encryption, multiplication mask post-decryption aggregation, random number sharing / mask sharing protocol and two-layer multiplication mask and other encryption and decryption methods, further improve data security and computational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A flowchart of a federated learning linear model training method provided in an embodiment of the present application;

[0045] Figure 2 A schematic diagram of a process for obtaining blinded ciphertext provided in an embodiment of the present application;

[0046] Figure 3 A schematic diagram of a process for obtaining a blinded ciphertext according to an aggregation parameter ciphertext and an aggregation loss fragment ciphertext provided in an embodiment of the present application;

[0047] Figure 4 A schematic diagram of a process for obtaining the gradient plaintext and loss update value of the current model parameter update round of a data user provided in an embodiment of the present application;

[0048] 5(a)-5(d) are schematic flow charts of a current mixed federated linear model process provided in an embodiment of the present application;

[0049] 6(a)-6(d) are flowchart diagrams of another current mixed federated linear model process provided by an embodiment of the present application;

[0050] Figure 7 A structural block diagram of a federated learning linear model training device provided in an embodiment of the present application;

[0051] Figure 8 An internal structure diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] In an exemplary embodiment, Figure 1 As shown, a method for training a linear model of federated learning is provided. This embodiment uses the method applied to the data user of the model training system as an example. It can be understood that the method can also be applied to a terminal, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] Step 102, receiving homomorphic encryption public keys from multiple data providers of the model training system.

[0055] Among them, the federated learning linear model training method provided in the embodiment of the present application can be applied to a model training system, which is used to train a mixed federated linear model. The model training system includes a data user and multiple data providers; the user's local data of the data user may include data identification, data label and data features; the provider's local data of the data provider may include data identification and data features. The user's local data may include multiple user sample data; the provider's local data may include multiple provider sample data. In this step, the user's data of the data user and the provider's data of the multiple data providers can be virtually fused to obtain an initial virtual fused data set corresponding to the mixed federated linear model. The virtual fused data set is a data set form formed by integrating data from different data providers and data users through specific technical means. It is not an actual physical fusion of the data of all parties, but a logical fusion of data through information records (for example, data identification ID, etc.) or serial numbers, so that it can be operated as a complete data set in subsequent model training or other tasks. Furthermore, the initial virtual fusion data set can be preprocessed to obtain a target virtual fusion data set corresponding to the hybrid federated linear model. In a possible implementation, the data preprocessing may include sample deduplication and feature alignment. The initial virtual fusion data set includes multiple virtual fusion data subsets, which may include a user virtual fusion data subset and multiple provider virtual fusion data subsets; the user virtual fusion data subset may include multiple user sample data; each provider virtual fusion data subset may include multiple provider sample data; each sample data in the multiple sample data includes a data identifier, a data label and a data feature. The sample deduplication may include: intersecting the multiple virtual fusion data subsets obtained by aligning the encrypted samples in pairs to obtain multiple sample intersections; for each intersection sample data in each sample intersection, confirming its belonging data subset, wherein the method of confirming each intersection sample data may include but is not limited to manual selection by the data user, automatic selection based on the data missing rate and random selection. The feature alignment may include obtaining the common features of multiple provider virtual fusion data subsets of the multiple data providers; storing the sample data corresponding to the common features as sample data for subsequent model training. After the above-mentioned hybrid encrypted sample alignment, sample deduplication and feature alignment (optional) operations, multiple virtual fusion data subsets are finally determined. These virtual fusion data sets integrate the relevant information of the data of all parties through a series of secure encryption, calculation and interaction operations without actually physically fusing the data of all parties. In the subsequent federated learning model training process, calculations can be performed based on these virtual fusion data sets to achieve collaborative use of data from all parties while protecting data privacy.

[0056] Next, each of the multiple data providers can configure its own exclusive homomorphic encryption key, which may include a homomorphic encryption public key and a homomorphic encryption private key. Homomorphic encryption is a special encryption technology that allows specific computing operations to be performed on encrypted data without decrypting the data. The result of the calculation after decryption is the same as the result of the same calculation directly on the original data, achieving the "computable but invisible" data. In homomorphic encryption, the public key is used to encrypt data; the private key is held by the data provider itself and is used to decrypt encrypted data. Only the party with the private key can restore the encrypted data to the original data, which ensures the ownership and privacy of the data. In the training of the hybrid federated learning linear model, each data provider generates its own homomorphic encryption public-private key pair. , and each data provider can send the generated homomorphic encryption public key to the corresponding data user, and the data user can receive the homomorphic encryption public keys from multiple data providers in the model training system. Each data provider is used to generate a corresponding homomorphic encryption key pair and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user.

[0057] Step 104, receiving the provider intermediate parameter fragment ciphertext sent by each data provider, and obtaining the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider based on the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider.

[0058] Among them, the data user can obtain the user's intermediate parameter fragment ciphertext based on the user's virtual fusion data subset; each of the multiple data providers can obtain the provider's intermediate parameter fragment ciphertext based on the provider's virtual fusion data subset. For the data user, the user's virtual fusion data subset may include multiple user sample data, and each of the multiple user sample data includes a sample number identifier and a sample feature component. The data user can calculate the user's intermediate parameter based on the user's virtual fusion data subset and the current model parameters, which can be seen in formula (1). The user's intermediate parameter is used to characterize the fitting of the current hybrid federated learning linear model to the user's local data. In addition, the data user can also calculate multiple loss fragments of the model. The loss function is a function that measures the difference between the model prediction value and the actual value. There is a one-to-one correspondence between the multiple loss fragments and the multiple data providers. Then, the data user can obtain the user's intermediate parameter fragment based on the user's intermediate parameter and the multiple loss fragments. For the data provider, the data provider can calculate the provider intermediate parameters based on the virtual fusion data subset and the current model parameters of the provider, as shown in formula (2). Next, each data provider can use its own configured homomorphic encryption public key to encrypt the provider intermediate parameters, obtain the provider intermediate parameter fragment ciphertext of each data provider, and send the provider intermediate parameter fragment ciphertext to the data user. The current model parameters are the current model parameters of the current hybrid federated learning linear model.

[0059]

[0060] in, It is the intermediate parameter of the user; is the current model parameter of the user; For the The characteristic components of the user sample data.

[0061]

[0062] in, It is the provider's intermediate parameter; is the current model parameters of the provider; For the The characteristic components of the sample data of each provider.

[0063] Next, the data user can obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider based on the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider. Specifically, it may include: the provider virtual fusion data subset of each data provider may include multiple provider sample data. For each provider sample data of the multiple provider sample data, for example, the i-th provider sample data of the j-th data provider, the data user can use the homomorphic encryption public key of the j-th data provider to encrypt the i-th user intermediate parameter fragment of the data provider to obtain the i-th user intermediate parameter fragment ciphertext. The loss fragment contained in the i-th user intermediate parameter fragment corresponds to the j-th data provider. Furthermore, the i-th user intermediate parameter fragment ciphertext can be aggregated with the corresponding i-th provider intermediate parameter fragment ciphertext to obtain the aggregate parameter ciphertext corresponding to the i-th provider sample data. , and the aggregated loss fragment ciphertext of the jth data provider , to obtain the aggregation parameter ciphertext corresponding to each provider sample data, and the aggregation loss fragment ciphertext of each data provider, the aggregation parameter fragment ciphertext may include the aggregation parameter ciphertext and the aggregation loss fragment ciphertext. Further, the data user can obtain the user gradient ciphertext of the data user according to the aggregation parameter ciphertext . Furthermore, the data user can obtain a random number and use the random number as a multiplication mask, and use the multiplication mask to blind the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext to obtain the blinded ciphertext of the aggregation parameter fragment ciphertext corresponding to each data provider.

[0064] Step 106: Send the blinded ciphertext to each data provider.

[0065] Among them, each data provider is used to obtain the provider's secondary multiplication blinded fragments of each data provider according to the blinded ciphertext, and send the provider's secondary multiplication blinded fragments to the data user, which may specifically include: the multiple data providers can use sharing technology, such as key sharing technology and random number sharing scheme, to obtain a same random number as a second-layer multiplication mask Each data provider uses its own homomorphic encryption private key to decrypt the corresponding blinded ciphertext received, and calculates the blinded gradient fragments of each data provider , using the two-layer multiplication mask The provider's blinded gradient fragment is blinded by a second multiplication to obtain the provider's second blinded gradient fragment In addition, each provider can also use their own homomorphic encryption private key to encrypt the gradient ciphertext of the user. and the corresponding loss fragment ciphertext , and obtain the blinded value of the user's gradient ciphertext and the blinded value of the loss fragment ciphertext. Further, each data provider can obtain the blinded value of the gradient fragment according to the provider's secondary blinding. , the user's gradient ciphertext blinding value of the user's gradient ciphertext and the loss-fragment blinding value of the aggregated loss-fragment ciphertext, obtain the corresponding provider's quadratic multiplication blinded fragment, and send the provider's quadratic multiplication blinded fragment to the data user.

[0066] Step 108, based on the provider's quadratic multiplication blinded fragments, obtain the gradient plaintext and loss update value of the data user's current model parameter update round.

[0067] Among them, the data user receives the provider's secondary multiplication blinded fragments sent by each provider, and can respectively aggregate the provider's secondary blinded gradient fragments, the user's gradient ciphertext blinded values ​​and the loss fragment blinded values ​​contained in the provider's secondary multiplication blinded fragments to obtain the aggregated provider's secondary multiplication blinded gradient fragments, the aggregated user's gradient ciphertext blinded values ​​and the aggregated loss fragment blinded values. Next, the data user can respectively derandomize the aggregated provider's secondary multiplication blinded gradient fragments, the aggregated user's gradient ciphertext blinded values ​​and the aggregated loss fragment blinded values ​​to obtain the provider gradient aggregate value corresponding to the aggregated provider's secondary multiplication blinded gradient fragments. , the user gradient aggregation value corresponding to the blinded value of the aggregated user gradient ciphertext, and the loss aggregation value corresponding to the blinded value of the aggregated loss fragment.

[0068] Next, the data user can use the gradient aggregation value of the provider after one layer of deblinding to each data provider; then, each data provider can aggregate the gradient value of the provider Perform secondary deblinding to obtain the provider gradient plaintext of the current model parameter update round of each data provider; the data user can obtain the user gradient plaintext of the current model parameter update round according to the user gradient aggregation value; and obtain the loss update value of the current model parameter update round according to the loss aggregation value. The current model parameter update round can be the current iteration round of the current hybrid federated linear model in the iterative update; the gradient plaintext can include the user gradient plaintext and the provider gradient plaintext.

[0069] Step 110, iteratively update the current hybrid federated linear model according to the gradient plaintext and the loss update value to obtain a target hybrid federated linear model corresponding to the current hybrid federated linear model.

[0070] Among them, the parameters of the current hybrid federated linear model can be updated according to the gradient plaintext and the loss update value, and then, steps 102 to 110 can be executed repeatedly until the iteration stop condition is met. In one possible implementation, the iteration stop condition can be to compare the current loss update value of the current model parameter update round with the historical loss update value of the previous model parameter update round to obtain the loss change value of the current loss update value and the historical loss update value. If the loss change value is less than the loss change threshold, it indicates that the current hybrid federated linear model has converged and the current hybrid federated linear model meets the iteration stop condition; if the loss change value is greater than or equal to the loss change threshold, it indicates that the current hybrid federated linear model has not converged, the current hybrid federated linear model does not meet the iteration stop condition, and the next round of iterative update continues.

[0071] In the method of this embodiment, first, homomorphic encryption public keys of multiple data providers from the model training system can be received; each data provider is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user; next, the provider intermediate parameter fragment ciphertext sent by each data provider can be received, and according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider, the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider is obtained; further, the blinded ciphertext can be sent to each data provider; each data provider is used to obtain the provider quadratic multiplication blinded fragment of each data provider according to the blinded ciphertext, and send the provider quadratic multiplication blinded fragment to the data user; further, the gradient plaintext and loss update value of the current model parameter update round of the data user can be obtained according to the provider quadratic multiplication blinded fragment; finally, the current hybrid federated linear model can be iteratively updated according to the gradient plaintext and the loss update value to obtain the target hybrid federated linear model corresponding to the current hybrid federated linear model. In the method provided in the embodiment of the present application, horizontal federated learning and vertical federated learning can be combined to realize the training of hybrid federated linear models of multiple isomorphic data providers, improve the feasibility of model training, and avoid the problem of information loss in sample data. It can improve data security during data transmission and data processing. In addition, encryption and decryption methods such as multi-key homomorphic sharding encryption, multiplication mask post-decryption aggregation, random number sharing / mask sharing protocol and two-layer multiplication mask are introduced to further improve data security and computing efficiency.

[0072] In an exemplary embodiment, Figure 2As shown, in step 104, obtaining the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider may include steps 202 to 206. Among them:

[0073] Step 202: encrypt the user intermediate parameter fragments using the homomorphic encryption public key to obtain the user intermediate parameter fragment ciphertext corresponding to the user intermediate parameter fragments.

[0074] Step 204: Aggregate the user's intermediate parameter fragment ciphertext and the provider's intermediate parameter fragment ciphertext to obtain the aggregated parameter ciphertext and aggregated loss fragment ciphertext corresponding to each data provider.

[0075] Step 206, obtaining a blinded ciphertext of the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext and the aggregation loss fragment ciphertext.

[0076] Among them, for the data user, the user virtual fusion data subset may include multiple user sample data, and each of the multiple user sample data includes a sample number identifier and a sample feature component. The data user can calculate the user intermediate parameter based on the user virtual fusion data subset and the current model parameters, which can be seen in formula (1). The user intermediate parameter is used to characterize the fitting of the current hybrid federated learning linear model to the user's local data. In addition, the data user can also calculate multiple loss fragments of the model. The loss function is a function that measures the difference between the model prediction value and the actual value. There is a one-to-one correspondence between the multiple loss fragments and the multiple data providers. Then, the data user can obtain the user intermediate parameter fragment based on the user intermediate parameter and the multiple loss fragments. For the data provider, the data provider can calculate the provider intermediate parameters based on the virtual fusion data subset and the current model parameters of the provider, as shown in formula (2). Next, each data provider can use its own configured homomorphic encryption public key to encrypt the provider intermediate parameters, obtain the provider intermediate parameter fragment ciphertext of each data provider, and send the provider intermediate parameter fragment ciphertext to the data user. The current model parameters are the current model parameters of the current hybrid federated learning linear model.

[0077] Next, the data user can obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider based on the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider. Specifically, it may include: the provider virtual fusion data subset of each data provider may include multiple provider sample data. For each provider sample data of the multiple provider sample data, for example, the i-th provider sample data of the j-th data provider, the data user can use the homomorphic encryption public key of the j-th data provider to encrypt the i-th user intermediate parameter fragment of the data provider to obtain the i-th user intermediate parameter fragment ciphertext. The loss fragment contained in the i-th user intermediate parameter fragment corresponds to the j-th data provider. Furthermore, the i-th user intermediate parameter fragment ciphertext can be aggregated with the corresponding i-th provider intermediate parameter fragment ciphertext to obtain the aggregate parameter ciphertext corresponding to the i-th provider sample data. , and the aggregated loss fragment ciphertext of the jth data provider , to obtain the aggregation parameter ciphertext corresponding to each provider sample data, and the aggregation loss fragment ciphertext of each data provider, the aggregation parameter fragment ciphertext may include the aggregation parameter ciphertext and the aggregation loss fragment ciphertext.

[0078] In an exemplary embodiment, Figure 3 As shown, step 206 may include steps 302 to 306. Among them:

[0079] Step 302, obtaining the user gradient ciphertext corresponding to the data user according to the aggregation parameter ciphertext.

[0080] Step 304, obtaining the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext.

[0081] Step 306, obtaining a blinded ciphertext according to the aggregated parameter fragment ciphertext.

[0082] In an exemplary embodiment, step 306 may include: obtaining a random number and using the random number as a multiplication mask; blinding the aggregation parameter fragment ciphertext using the multiplication mask to obtain a blinded ciphertext corresponding to the aggregation parameter fragment ciphertext.

[0083] Among them, the data user can obtain the user gradient ciphertext of the data user according to the aggregation parameter ciphertext . Furthermore, the data user can obtain a random number and use the random number as a multiplication mask, and use the multiplication mask to blind the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext to obtain the blinded ciphertext of the aggregation parameter fragment ciphertext corresponding to each data provider.

[0084] In the method of this embodiment, encryption and decryption methods such as multi-key homomorphic sharding encryption, multiplication mask and decryption aggregation, random number sharing / mask sharing protocol and two-layer multiplication mask are introduced to further improve data security and computing efficiency.

[0085] In an exemplary embodiment, Figure 4 As shown, step 108 may include steps 402 to 404. Among them:

[0086] Step 402: Obtain aggregated provider quadratic multiplication blinded fragments corresponding to provider quadratic multiplication blinded fragments.

[0087] Among them, the secondary multiplication blinded fragments include provider secondary blinded gradient fragments, user gradient ciphertext blinded values ​​and loss fragment blinded values; the provider secondary blinded gradient fragments, user gradient ciphertext blinded values ​​and loss fragment blinded values ​​can be aggregated respectively to obtain aggregated provider secondary multiplication blinded gradient fragments, aggregated user gradient ciphertext blinded values ​​and aggregated loss fragment blinded values; the aggregated provider secondary multiplication blinded fragments can include the aggregated provider secondary multiplication blinded gradient fragments, the aggregated user gradient ciphertext blinded values ​​and the aggregated loss fragment blinded values.

[0088] Step 404, based on the aggregation provider's quadratic multiplication blinded fragments, obtain the data user's current model parameter update round's gradient plaintext and loss update value.

[0089] In an exemplary embodiment, step 402 may include:

[0090] Aggregate the provider quadratic multiplication blinded gradient fragments of multiple data providers to obtain aggregated provider quadratic multiplication blinded gradient fragments; aggregate the user gradient ciphertext blinded values ​​to obtain the aggregated user gradient ciphertext blinded value; aggregate the loss fragment blinded values ​​to obtain the aggregated loss fragment blinded value; obtain the aggregated provider quadratic multiplication blinded fragments according to the aggregated provider quadratic multiplication blinded gradient fragments, the aggregated user gradient ciphertext blinded value and the aggregated loss fragment blinded value.

[0091] In an exemplary embodiment, step 404 may include:

[0092] Derandomize the aggregated provider quadratic multiplication blinded gradient fragments to obtain the provider gradient aggregation value corresponding to the aggregated provider quadratic multiplication blinded gradient fragments; derandomize the aggregated user gradient ciphertext blinded value to obtain the user gradient aggregation value corresponding to the aggregated user gradient ciphertext blinded value; derandomize the aggregated loss fragment blinded value to obtain the loss aggregation value corresponding to the aggregated loss fragment blinded value; obtain the gradient plaintext of the current model parameter update round based on the provider gradient aggregation value and the user gradient aggregation value; obtain the loss update value of the current model parameter update round based on the loss aggregation value.

[0093] Among them, the data user can derandomize the aggregate provider's secondary multiplication blinded gradient fragment, the aggregate user's gradient ciphertext blinded value and the aggregate loss fragment blinded value respectively to obtain the provider gradient aggregation value corresponding to the aggregate provider's secondary multiplication blinded gradient fragment , the user gradient aggregation value corresponding to the blinded value of the aggregated user gradient ciphertext, and the loss aggregation value corresponding to the blinded value of the aggregated loss fragment.

[0094] In an exemplary embodiment, the step of obtaining the gradient plaintext of the current model parameter update round according to the gradient aggregation value of the provider and the gradient aggregation value of the user may include:

[0095] According to the user's gradient aggregation value, the user's gradient plaintext is obtained; the provider's gradient aggregation value is sent to each data provider; the data provider is used to perform secondary de-blinding on the provider's gradient aggregation value to obtain the provider's gradient plaintext.

[0096] Among them, the data user can use the gradient aggregation value of the provider after one layer of deblinding to each data provider; then, each data provider can aggregate the gradient value of the provider Perform secondary deblinding to obtain the provider gradient plaintext of the current model parameter update round of each data provider; the data user can obtain the user gradient plaintext of the current model parameter update round according to the user gradient aggregation value; and obtain the loss update value of the current model parameter update round according to the loss aggregation value. The current model parameter update round can be the current iteration round of the current hybrid federated linear model in the iterative update; the gradient plaintext can include the user gradient plaintext and the provider gradient plaintext.

[0097] In the method of this embodiment, multiple key homomorphic shard encryption + multiplication mask post-decryption aggregation + random number sharing + two-layer multiplication mask are used to replace the secret aggregation of the same public key encryption calculation results, thereby solving the two main problems of the previous solution: 1) The roles of multiple data providers are not equal, and there is a master-slave relationship. The core / master data provider holds the private key for decryption. The probability that other data providers cannot accept this solution is very high, so it is difficult to apply it. 2) Since the generation and management of the homomorphic encryption key used to protect the relevant data or intermediate calculation results of other subordinate data providers is the core / master data provider, assuming that the master data provider and the data user collude, the data user can decrypt all ciphertexts transmitted in all communications and infer the original data of each sample, which poses a greater security risk, thereby improving the data security of data transmission and data processing.

[0098] In an exemplary embodiment, as shown in FIG. 5( a ) to FIG. 5 ( d ), a method for training a federated learning linear model is provided. The method may include:

[0099] In this embodiment, in each iterative update of the current mixed-directional federated linear model, the calculation of the intermediate variables of the current mixed-directional federated linear model can refer to the following formula:

[0100]

[0101] in, Provider intermediate parameters; is the current model parameters of the provider; is the characteristic component of the sample data of the i-th provider; It is the intermediate parameter of the user; is the current model parameter of the user; is the characteristic component of the sample data of the i-th user; L is the loss function; To provide the gradient; is the user gradient. Since there are multiple data providers (host parties), assuming that the data user (Guest party) and the jth data provider (Host party) (jth data provider (Host party)), after hybrid encrypted sample alignment and deduplication, the virtual fusion dataset used for final training is , then use the subscript j to indicate that all calculation results are in The fragments on the sample set. And, it is agreed to use To mark In homomorphic encryption public key (Right now The ciphertext generated under the homomorphic encryption public key generated by the data provider (Host). In this embodiment, the overall training process steps of the current mixed federated linear model are shown in Figures 5 (a)-5 (d).

[0102] In another exemplary embodiment, as shown in FIG6(a)-6(d), a federated learning linear model training method is provided, and the method may include:

[0103] In this embodiment, in each iterative update of the current mixed-directional federated linear model, the calculation of the intermediate variables of the current mixed-directional federated linear model can refer to the following formula:

[0104]

[0105] in, Provider intermediate parameters; is the current model parameters of the provider; is the characteristic component of the sample data of the i-th provider; It is the intermediate parameter of the user; is the current model parameter of the user; is the characteristic component of the sample data of the i-th user; L is the loss function; To provide the gradient; is the user gradient. Since there are multiple data providers (host parties), assuming that the data user (Guest party) and the jth data provider (Host party) (jth data provider (Host party)), after hybrid encrypted sample alignment and deduplication, the virtual fusion dataset used for final training is , then use the subscript j to indicate that all calculation results are in The fragments on the sample set. And, it is agreed to use To mark In homomorphic encryption public key (Right now The ciphertext generated under the homomorphic encryption public key generated by the data provider (Host). In this embodiment, the overall training process steps of the current mixed federated linear model are shown in Figures 6 (a)-6 (d).

[0106] In the method of this embodiment, first, homomorphic encryption public keys of multiple data providers from the model training system can be received; each data provider is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user; next, the provider intermediate parameter fragment ciphertext sent by each data provider can be received, and according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider, the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each data provider is obtained; further, the blinded ciphertext can be sent to each data provider; each data provider is used to obtain the provider quadratic multiplication blinded fragment of each data provider according to the blinded ciphertext, and send the provider quadratic multiplication blinded fragment to the data user; further, the gradient plaintext and loss update value of the current model parameter update round of the data user can be obtained according to the provider quadratic multiplication blinded fragment; finally, the current hybrid federated linear model can be iteratively updated according to the gradient plaintext and the loss update value to obtain the target hybrid federated linear model corresponding to the current hybrid federated linear model. In the method provided in the embodiment of the present application, horizontal federated learning and vertical federated learning can be combined to realize the training of a hybrid federated linear model of one-to-many homogeneous data providers, improve the feasibility of model training, and avoid the problem of information loss in sample data. It can improve data security during data transmission and data processing. In addition, encryption and decryption methods such as multi-key homomorphic sharding encryption, multiplication mask post-decryption aggregation, random number sharing / mask sharing protocol and two-layer multiplication mask are introduced to further improve data security and computing efficiency.

[0107] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0108] Based on the same inventive concept, the embodiment of the present application also provides a federated learning linear model training device for implementing the above-mentioned federated learning linear model training method. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above-mentioned method, so the specific limitations in one or more federated learning linear model training device embodiments provided below can refer to the above-mentioned limitations on the federated learning linear model training method, which will not be repeated here.

[0109] In one embodiment, Figure 7 As shown, a federated learning linear model training device is provided, including: a first receiving module 702, a second receiving module 704, a sending module 706, a computing module 708 and a training module 710, wherein:

[0110] The first receiving module 702 is used to receive homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key included in the homomorphic encryption key pair to the data user;

[0111] The second receiving module 704 is used to receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers;

[0112] A sending module 706 is used to send the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and send the provider's secondary multiplication blinded fragments to the data user;

[0113] A calculation module 708 is used to obtain the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider;

[0114] The training module 710 is used to iteratively update the current mixed-directional federated linear model according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

[0115] In one embodiment, the second receiving module 704 is also used to: encrypt the user intermediate parameter fragments using the homomorphic encryption public key to obtain the user intermediate parameter fragment ciphertext corresponding to the user intermediate parameter fragment; aggregate the user intermediate parameter fragment ciphertext with the provider intermediate parameter fragment ciphertext to obtain the aggregate parameter ciphertext and aggregate loss fragment ciphertext corresponding to each data provider; and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext based on the aggregate parameter ciphertext and the aggregate loss fragment ciphertext.

[0116] In one embodiment, the second receiving module 704 is also used to: obtain the blinded ciphertext of the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext and the aggregation loss fragment ciphertext, including: obtaining the user gradient ciphertext corresponding to the data user according to the aggregation parameter ciphertext; obtaining the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext; and obtaining the blinded ciphertext according to the aggregation parameter fragment ciphertext.

[0117] In one embodiment, the second receiving module 704 is further used to: obtain a random number and use the random number as a multiplication mask; use the multiplication mask to blind the aggregation parameter fragment ciphertext to obtain the blinded ciphertext corresponding to the aggregation parameter fragment ciphertext.

[0118] In one embodiment, the calculation module 708 is also used to: obtain the aggregated provider quadratic multiplication blinded fragments corresponding to the provider quadratic multiplication blinded fragments; and obtain the gradient plaintext and the loss update value of the current model parameter update round of the data user based on the aggregated provider quadratic multiplication blinded fragments.

[0119] In one embodiment, the provider's secondary multiplication blinded gradient fragments include provider's secondary multiplication blinded gradient fragments, user's gradient ciphertext blinded values ​​and loss fragment blinded values; the calculation module 708 is also used to: aggregate the provider's secondary multiplication blinded gradient fragments of multiple data providers to obtain aggregated provider's secondary multiplication blinded gradient fragments; aggregate the user's gradient ciphertext blinded values ​​to obtain aggregated user's gradient ciphertext blinded values; aggregate the loss fragment blinded values ​​to obtain aggregated loss fragment blinded values; obtain the aggregated provider's secondary multiplication blinded fragments according to the aggregated provider's secondary multiplication blinded gradient fragments, the aggregated user's gradient ciphertext blinded values ​​and the aggregated loss fragment blinded values.

[0120] In one embodiment, the calculation module 708 is also used to: derandomize the aggregated provider quadratic multiplication blinded gradient fragments to obtain the provider gradient aggregation value corresponding to the aggregated provider quadratic multiplication blinded gradient fragments; derandomize the aggregated user gradient ciphertext blinded value to obtain the user gradient aggregation value corresponding to the aggregated user gradient ciphertext blinded value; derandomize the aggregated loss fragment blinded value to obtain the loss aggregation value corresponding to the aggregated loss fragment blinded value; obtain the gradient plaintext of the current model parameter update round based on the provider gradient aggregation value and the user gradient aggregation value; and obtain the loss update value of the current model parameter update round based on the loss aggregation value.

[0121] In one embodiment, the gradient plaintext includes user gradient plaintext and provider gradient plaintext; the calculation module 708 is also used to: obtain the user gradient plaintext according to the user gradient aggregation value; send the provider gradient aggregation value to each of the data providers; the data provider is used to perform secondary de-blinding on the provider gradient aggregation value to obtain the provider gradient plaintext.

[0122] Each module in the above-mentioned federated learning linear model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0123] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the training of a federated learning linear model. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for training a federated learning linear model is implemented.

[0124] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0125] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0129] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0130] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for training a linear model using federated learning, applied to a data user of a model training system, characterized in that: The method comprises: Receiving homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key contained in the homomorphic encryption key pair to the data user; Receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers; Sending the blinded ciphertext to each of the data providers; each of the data providers is used to obtain the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and send the provider's secondary multiplication blinded fragments to the data user; Obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider; The current mixed-directional federated linear model is iteratively updated according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

2. The method according to claim 1, characterized in that The step of obtaining the blinded ciphertext of the aggregated parameter fragment ciphertext corresponding to each data provider according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each data provider includes: Encrypting the user intermediate parameter fragments by using the homomorphic encryption public key to obtain the user intermediate parameter fragment ciphertext corresponding to the user intermediate parameter fragments; Aggregating the user's intermediate parameter fragment ciphertext and the provider's intermediate parameter fragment ciphertext to obtain the aggregated parameter ciphertext and aggregated loss fragment ciphertext corresponding to each of the data providers; The blinded ciphertext of the aggregation parameter fragment ciphertext is obtained according to the aggregation parameter ciphertext and the aggregation loss fragment ciphertext.

3. The method according to claim 2, characterized in that The step of obtaining the blinded ciphertext of the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext and the aggregation loss fragment ciphertext comprises: Obtaining a user gradient ciphertext corresponding to the data user according to the aggregation parameter ciphertext; Obtaining the aggregation parameter fragment ciphertext according to the aggregation parameter ciphertext, the aggregation loss fragment ciphertext and the user gradient ciphertext; The blinded ciphertext is obtained according to the aggregation parameter fragment ciphertext.

4. The method according to claim 3, characterized in that The step of obtaining the blinded ciphertext according to the aggregated parameter fragment ciphertext includes: Obtain a random number, and use the random number as a multiplication mask; The aggregation parameter fragment ciphertext is blinded by using the multiplication mask to obtain the blinded ciphertext corresponding to the aggregation parameter fragment ciphertext.

5. The method according to any one of claims 1 to 4, characterized in that: The step of obtaining the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider includes: Obtaining an aggregated provider quadratic multiplication blinded fragment corresponding to the provider quadratic multiplication blinded fragment; According to the quadratic multiplication blinded fragments of the aggregation provider, the gradient plaintext and the loss update value of the current model parameter update round of the data user are obtained.

6. The method according to claim 5, characterized in that The provider's quadratic multiplication blinded gradient fragments include the provider's quadratic multiplication blinded gradient fragments, the user's gradient ciphertext blinding value and the loss fragment blinding value; The obtaining the aggregated provider secondary multiplication blinded fragment corresponding to the provider secondary multiplication blinded fragment includes: Aggregating the provider quadratic multiplication blinded gradient fragments of a plurality of the data providers to obtain aggregated provider quadratic multiplication blinded gradient fragments; Aggregating the user gradient ciphertext blinded values ​​to obtain an aggregated user gradient ciphertext blinded value; Aggregating the loss fragment blinding values ​​to obtain an aggregated loss fragment blinding value; The aggregation provider's quadratic multiplication blinded fragment is obtained according to the aggregation provider's quadratic multiplication blinded gradient fragment, the aggregation user's gradient ciphertext blinded value and the aggregation loss fragment blinded value.

7. The method according to claim 6, characterized in that The obtaining, according to the secondary multiplication blinding fragments of the aggregation provider, the gradient plaintext and the loss update value of the current model parameter update round of the data user comprises: De-randomizing the aggregated provider quadratic multiplication blinded gradient fragments to obtain the provider gradient aggregation value corresponding to the aggregated provider quadratic multiplication blinded gradient fragments; De-randomizing the aggregated user gradient ciphertext blinded value to obtain a user gradient aggregate value corresponding to the aggregated user gradient ciphertext blinded value; De-randomizing the aggregate loss fragment blinding value to obtain a loss aggregation value corresponding to the aggregate loss fragment blinding value; Obtaining the gradient plaintext of the current model parameter update round according to the provider gradient aggregate value and the user gradient aggregate value; According to the loss aggregation value, the loss update value of the current model parameter update round is obtained.

8. The method according to claim 7, characterized in that The gradient plaintext includes a user gradient plaintext and a provider gradient plaintext; The obtaining the gradient plaintext of the current model parameter update round according to the provider gradient aggregate value and the user gradient aggregate value includes: Obtaining the user gradient plaintext according to the user gradient aggregation value; The provider gradient aggregation value is sent to each of the data providers; the data provider is used to perform secondary de-blinding on the provider gradient aggregation value to obtain the provider gradient plaintext.

9. A federated learning linear model training device, applied to a data user of a model training system, characterized in that: The device comprises: A first receiving module is used to receive homomorphic encryption public keys from multiple data providers of the model training system; each of the data providers is used to generate a corresponding homomorphic encryption key pair, and send the homomorphic encryption public key included in the homomorphic encryption key pair to the data user; A second receiving module is used to receive the provider intermediate parameter fragment ciphertext sent by each of the data providers, and obtain the blinded ciphertext of the aggregate parameter fragment ciphertext corresponding to each of the data providers according to the provider intermediate parameter fragment ciphertext and the homomorphic encryption public key corresponding to each of the data providers; A sending module, used for sending the blinded ciphertext to each of the data providers; each of the data providers is used for obtaining the provider's secondary multiplication blinded fragments of each of the data providers according to the blinded ciphertext, and sending the provider's secondary multiplication blinded fragments to the data user; A calculation module, used to obtain the gradient plaintext and loss update value of the current model parameter update round of the data user according to the secondary multiplication blinded fragments of the provider; A training module is used to iteratively update the current mixed-directional federated linear model according to the gradient plaintext and the loss update value to obtain a target mixed-directional federated linear model corresponding to the current mixed-directional federated linear model.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images