Federated learning model training method and system

Abstract

The present invention provides a method and system for training a federated learning model, the method comprising: creating a blockchain network for training a federated learning model; receiving local training model parameters uploaded by an organization user during each round of training of the federated learning model, the organization user being a blockchain user who trains the federated learning model locally using local user data; sending the local training model parameters uploaded by all organization users to worker users, who jointly train the federated learning model using all local training model parameters; updating the federated learning model using the joint training model parameters uploaded by the worker users who successfully created the block; and sending the joint training model parameters to the organization user to update the local training model parameters of the organization user. In the present invention, by introducing other participating users other than the data provider to participate in the training of the federated learning model, the user range is expanded.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of business support technology, and in particular to a federated learning model training method and system. Background Art

[0002] Federated Learning is an emerging AI basic technology that conducts efficient machine learning among multiple parties or computing nodes under the premise of ensuring information security during big data exchange, protecting terminal data and personal data privacy, and ensuring legal compliance. Currently, some federated learning models are trained through federated learning.

[0003] However, the current method of training federated learning models through federated learning has the following problems: the participants in federated learning are limited to data providers, resulting in a small user range and low training efficiency. Summary of the invention

[0004] The embodiments of the present invention provide a federated learning model training method and system, which are used to solve the problems of small user range and low training efficiency in the current method of training a federated learning model through federated learning.

[0005] In order to solve the above-mentioned technical problems, the present invention is achieved as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for training a federated learning model, including:

[0007] Create a blockchain network for training federated learning models;

[0008] During each round of training of the federated learning model, local training model parameters uploaded by organizational users are received. The organizational users are blockchain users who train the federated learning model locally using local user data.

[0009] Sending the local training model parameters uploaded by all the organization users to the worker users, and the worker users jointly train the federated learning model using the local training model parameters uploaded by all the organization users;

[0010] Updating the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block;

[0011] The joint training model parameters are sent to the organization user to update the local training model parameters of the organization user.

[0012] Optionally, the local training model parameters uploaded by the organization's users are encrypted using a homomorphic encryption algorithm.

[0013] Optionally, the organizational user uses a customized key to homomorphically encrypt local training model parameters.

[0014] Optionally, in each round of training of the federated learning model, the blockchain node of the blockchain network receives the local training model parameters uploaded by the organization user, including:

[0015] When receiving the local training model parameters uploaded by the organization user, determine whether the organization user has uploaded an encrypted certificate at the same time, where the encrypted certificate is issued by any N endorsing nodes of the blockchain network based on zero-knowledge proof;

[0016] If the organization user does not send the encrypted proof, the organization user is determined to be a malicious participant.

[0017] Optionally, the method further includes:

[0018] Before training the federated learning model, additional samples uploaded by organizational users are received to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are the outputs obtained by the organizational users training the federated learning model by adding the input samples to their own local user data;

[0019] Receive local training model parameters uploaded by organizational users;

[0020] Any N endorsing nodes of the blockchain network randomly extract K samples from the additional sample library, and perform accuracy judgment on the local training model parameters according to the K samples;

[0021] If the average accuracy of N endorsement nodes is higher than the preset threshold, an encrypted certificate is issued to the organization user.

[0022] Optionally, the method further includes:

[0023] After the blockchain network is created, each user is given initial assets;

[0024] After obtaining a converged federated learning model, asset settlement is performed for each user based on the transactions that occur in each round.

[0025] Optionally, based on the transactions occurring in each round, before the asset settlement for each user, the following is also included:

[0026] After the user joins the blockchain network, a smart contract set is deployed on the user's blockchain node. If the user is an organization user, the smart contract set includes at least one of the following: a status query contract, a model update contract, and a model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract, and a model download contract;

[0027] When the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

[0028] When the model update contract is called by the organization user, the local training model parameters of the organization user are uploaded to the blockchain network to complete a model parameter update transaction;

[0029] When the model download contract is called by the worker user, all local training model parameters uploaded by the organization users are downloaded from the blockchain network. When the model download contract is called by the organization user, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction.

[0030] When the model processing contract is called by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

[0031] In a second aspect, an embodiment of the present invention provides a federated learning model training system, including:

[0032] Create a module for creating a blockchain network for training federated learning models;

[0033] A first receiving module is used to receive local training model parameters uploaded by an organization user during each round of training of the federated learning model, where the organization user is a blockchain user who trains the federated learning model locally using local user data;

[0034] A first sending module is used to send the local training model parameters uploaded by all the organization users to the worker users, and the worker users use the local training model parameters uploaded by all the organization users to jointly train the federated learning model;

[0035] An updating module, configured to update the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block;

[0036] The second sending module is used to send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

[0037] Optionally, the local training model parameters uploaded by the organization's users are encrypted using a homomorphic encryption algorithm.

[0038] Optionally, the organizational user uses a customized key to homomorphically encrypt local training model parameters.

[0039] Optionally, the system further includes:

[0040] The encryption module is used to determine whether the organization user has uploaded an encryption certificate at the same time when receiving the local training model parameters uploaded by the organization user. The encryption certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof; if the organization user does not send the encryption certificate, the organization user is determined to be a malicious participant.

[0041] Optionally, the system further includes:

[0042] A first receiving module is used to receive additional samples sent by organizational users before training the federated learning model to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are outputs obtained by the organizational users adding the input samples to their own local user data to train the federated learning model;

[0043] The second receiving module is used to receive local training model parameters uploaded by organizational users;

[0044] A discrimination module, used for randomly extracting K samples from the additional sample library, and performing accuracy discrimination on the parameters of the local training model according to the K samples;

[0045] The certification module is used to issue an encrypted certificate to the organization user if the average accuracy of the N endorsement nodes is higher than a preset threshold.

[0046] Optionally, the system further includes:

[0047] The initial asset granting module is used to grant initial assets to each user after the blockchain network is created;

[0048] The settlement module is used to settle assets for each user based on the transactions that occurred in each round after the converged federated learning model is obtained.

[0049] Optionally, the system further includes:

[0050] A contract deployment module, used to deploy a smart contract set on the user's blockchain node after the user joins the blockchain network. If the user is an organization user, the smart contract set includes at least one of the following: a status query contract, a model update contract, and a model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract, and a model download contract;

[0051] When the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

[0052] When the model update contract is called by the organization user, the local training model parameters of the organization user are uploaded to the blockchain network to complete a model parameter update transaction;

[0053] When the model download contract is called by the worker user, all local training model parameters uploaded by the organization users are downloaded from the blockchain network. When the model download contract is called by the organization user, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction.

[0054] When the model processing contract is called by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

[0055] In a third aspect, an embodiment of the present invention provides a computing device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the federated learning model training method of the first aspect.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the federated learning model training method of the first aspect are implemented.

[0057] In the embodiments of the present invention, blockchain and federated learning are combined. By introducing other participating users besides data providers, users who do not have sufficient user data or do not want to share user data also have the right to participate in federated learning, thereby expanding the user range and effectively improving the efficiency of model training. In addition, rewards can be obtained by generating blocks through model training. These rewards can be used to offset model usage fees, etc., which is superior to existing technical solutions in terms of user range and incentive mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0059] Figure 1 A schematic diagram of a process flow of a federated learning model training method according to an embodiment of the present invention;

[0060] Figure 2 A schematic diagram of the structure of a federated learning model training system according to an embodiment of the present invention;

[0061] Figure 3 A schematic diagram of the structure of a computing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0063] Blockchain technology is a joint innovative application of four technologies: smart contracts, peer-to-peer networks, encryption algorithms, and consensus mechanisms. Blockchain is essentially a linked list storage structure based on timestamps, also known as a distributed ledger or decentralized database, because the nodes of the blockchain form a peer-to-peer network, showing decentralized characteristics. The blockchain determines the writer based on the consensus mechanism, while ensuring the consistency of data among nodes in the entire network; it ensures data security and reliability based on encryption algorithms, and uses smart contracts deployed on nodes for automated transaction processing. In summary, blockchain has the characteristics of decentralization, collective maintenance, cross-validation, openness and transparency, non-tamperability, and data traceability. It is applied to various application scenarios that require new database support, multi-subject business, lack of mutual trust among subjects, and no trusted third party. Typical examples include digital finance, food and drug traceability, and judicial evidence storage.

[0064] As a decentralized, data-encrypted, and tamper-proof distributed shared database, blockchain can provide data confidentiality for data exchange in federated learning to protect user privacy and ensure data security among all participants. It can also ensure data consistency among multiple participants for model training. The value-driven incentive mechanism of blockchain can also increase the enthusiasm of all participants to provide data and update network model parameters.

[0065] In the present invention, blockchain and federated learning are combined to train a federated learning model, wherein the federated learning model may be a business recommendation model, etc.

[0066] Please refer to Figure 1 , an embodiment of the present invention provides a federated learning model training method, comprising:

[0067] Step 11: Create a blockchain network for training the federated learning model;

[0068] In the embodiment of the present invention, the participants of the blockchain network are collectively referred to as blockchain users. Blockchain users can be divided into organizational users and worker users according to their contribution methods.

[0069] (a) Organizational users refer to business vendors that own user data and are willing to use user data to participate in federated learning. Organizations contribute by sharing samples and participating in the training of federated learning models. Each time an organization submits updated model parameters, it can receive corresponding rewards.

[0070] (b) Worker users refer to users who do not have sufficient user data, or have user data but are unwilling to use their own user data to participate in federated learning, but still want to use the federated learning model, such as small-scale business manufacturers. The contribution of worker users includes training the federated learning model and updating model parameters, and producing new blocks. For example, if multiple worker users compete to complete the work of a block, the first worker user who completes the block and passes the consensus confirmation can be rewarded.

[0071] In the embodiment of the present invention, multiple service providers in the same field jointly discuss a deep learning model and use it as a federated learning model. Considering that the services provided by multiple service providers belong to the same field, the feature fields of their data often have a high degree of overlap, while the user group overlaps a low degree. Therefore, a sample-based federated learning method is used to extract part of the data with overlapping feature fields and non-overlapping user samples as shared samples.

[0072] When providing services, service providers build local data collection modules into software (such as e-commerce, news and other apps) or hardware (such as home theater video on demand, cloud gaming services and other devices) based on actual business needs. The modules are used to collect user data and upload it to the manufacturer's server to form the manufacturer's local user database.

[0073] Step 12: During each round of training of the federated learning model, receiving local training model parameters uploaded by the organizational user, wherein the organizational user is a blockchain user who trains the federated learning model locally using local user data;

[0074] Organization users can use local user data to train federated learning models locally without uploading local user data, thereby protecting the privacy of user data and avoiding user data leakage.

[0075] Step 13: Send the local training model parameters uploaded by all the organization users to the worker users, and the worker users use the local training model parameters uploaded by all the organization users to jointly train the federated learning model;

[0076] Step 14: Update the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block;

[0077] Step 15: Send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

[0078] In the embodiments of the present invention, blockchain and federated learning are combined. By introducing other participating users besides data providers, users who do not have sufficient user data or do not want to share user data also have the right to participate in federated learning, thereby expanding the user range and effectively improving the efficiency of model training. In addition, rewards can be obtained by generating blocks through model training. These rewards can be used to offset model usage fees, etc., which is superior to existing technical solutions in terms of user range and incentive mechanism.

[0079] The reward mechanism of the embodiment of the present invention is described below.

[0080] In the embodiment of the present invention, optionally, the method further includes:

[0081] After the blockchain network is created, initial assets are granted to each user; the participants include organization users and worker users, and the initial assets are used to reflect the cost (monetary cost or data cost) paid by the user for participating in federated learning.

[0082] After obtaining a converged federated learning model, asset settlement is performed for each user based on the transactions that occur in each round.

[0083] In the embodiment of the present invention, optionally, before performing asset settlement for each user according to the transactions occurring in each round, the following is further included:

[0084] After the user joins the blockchain network, a smart contract set S is deployed on the user's blockchain node. If the user is an organizational user, the smart contract set S includes at least one of the following: Status query contract S 1 , model update contract S 2 and model download contract S 3 ; If the user is a worker user, the smart contract set includes at least one of the following: Status query contract S1 , Model Processing Contract S 4 and model download contract S 3 ; It should be noted that, in addition to this contract, the smart contract set S should be scalable and other smart contracts should be added according to actual business needs.

[0085] Among them, the status query contract S 1 When called by a user, a query transaction is submitted to query the world status on all distributed ledgers owned by the participant, that is, to query the current remaining available assets of the user; that is, the participant can use the status query contract S 1 Submit a query transaction.

[0086] The model updates the contract S 2 When called by the organization user, the organization user's local training model parameters are uploaded to the blockchain network to complete a model parameter update transaction; that is, the organization user can update the contract S by calling the model 2 Upload the encrypted local training model parameters to the blockchain. In each iteration of federated learning, all organizational users (organization users) need to complete the model parameter upload so that worker users can use all model parameters for joint training.

[0087] The model download contract S 3 When called by the worker user, all local training model parameters uploaded by the organization user are downloaded from the blockchain network. The model download contract S 3 When called by the organization user, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction; that is, the participating party calls the model download contract S 3 , you can submit a model parameter download transaction once.

[0088] The model processing contract S 4 When called by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction. After completing the joint training and successfully creating a block, the worker user can call the model processing contract S 4 , use the joint training model parameters to update the parameters of the federated learning model.

[0089] A complete round of federated learning iteration consists of the following steps:

[0090] (a) Organize users to use local user data to train the federated learning model and obtain local training model parameters. Call the model update contract S 2Generate a transaction, encrypt the local training model parameters and upload them to the blockchain, that is, all users exchange encrypted local training model parameters. For a transaction initiated by multiple users, each user needs to pay a transaction fee based on the amount of data they own. Generally speaking, the more data a party has, the less fees they pay.

[0091] (b) Worker users collect the model update contract S proposed by all organization users 2 Transaction, call model to download contract S 3 Download all encrypted model parameters, use all model parameters to complete the joint training of the federated learning model, and the worker user who successfully creates the block calls the model processing contract S 4 Complete the model parameter update.

[0092] (c) Organize users to call the model to download contract S 3 Download the joint training model parameters, decrypt them and update the local training model parameters.

[0093] At this point, a complete round of federated learning iteration is completed. At this point, a block in the blockchain is successfully created, the worker user who created the block is rewarded, and the organization user who participated in the training completes an update of the local model. At this point, each organization user calculates whether the loss function meets the requirements. If not, the next round of iteration is initiated, and the above three steps (a), (b), and (c) are repeated until the loss function converges.

[0094] When all users complete multiple rounds of federated learning iterations and obtain a converged federated learning model, actual asset settlement is performed based on the changes in user assets caused by transactions in each round of iteration.

[0095] Although the organization user trains the federated learning model locally, the model parameters obtained through training contain some features of the local user data. If a malicious party obtains the plaintext model parameters and the predetermined federated learning model, the features of the local user data can be inferred from them, resulting in data leakage. Therefore, in an embodiment of the present invention, in order to avoid leakage of data feature information caused by leakage of model parameters, the local training model parameters uploaded by the organization user are optionally encrypted using a homomorphic encryption algorithm. Further optionally, the organization user uses a custom key to homomorphically encrypt the local training model parameters.

[0096] The characteristic of homomorphic encryption is that the homomorphic encrypted data is processed to obtain an output, and the output is decrypted, and the result is the same as the output obtained by processing the unencrypted original data in the same way. Therefore, as a worker user who actually conducts joint training, even if the gradient is not decrypted, the model training task can be completed. That is, the worker user cannot access any unencrypted data throughout the process, avoiding the leakage of local model data feature information.

[0097] In the disclosed embodiment, the model parameters uploaded by the organization users are encrypted by a homomorphic encryption algorithm, which prevents information leakage caused by plain text model parameters. The data that the worker users contact when training the model are all encrypted data, and the worker users are completely unable to decrypt them, thus solving the problem of mutual trust between the organization users and the worker users.

[0098] In the process of federated learning, there are multiple participants, among which there may be malicious participants. Malicious participants can obtain joint training model parameters at no or little cost by maliciously uploading erroneous or forged model parameters, thereby reducing the accuracy of the joint training model.

[0099] In order to avoid attacks by malicious participants, in the embodiment of the present invention, in each round of training of the federated learning model, the blockchain node of the blockchain network receives the local training model parameters uploaded by the organization user, including:

[0100] When receiving the local training model parameters uploaded by the organization user, determine whether the organization user has uploaded an encrypted certificate at the same time, where the encrypted certificate is issued by any N endorsing nodes of the blockchain network based on zero-knowledge proof;

[0101] If the organization user does not send the encrypted proof, the organization user is determined to be a malicious participant.

[0102] In the embodiment of the present invention, optionally, the method further includes:

[0103] Before training the federated learning model, additional samples uploaded by organizational users are received to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are the outputs obtained by the organizational users training the federated learning model by adding the input samples to their own local user data;

[0104] Receive local training model parameters uploaded by organizational users;

[0105] Any N endorsing nodes of the blockchain network randomly extract K samples from the additional sample library, and perform accuracy judgment on the local training model parameters according to the K samples;

[0106] If the average accuracy of N endorsement nodes is higher than the preset threshold, an encrypted certificate is issued to the organization user.

[0107] Wherein, N is a positive integer greater than 1, and K is a positive integer greater than 1.

[0108] That is to say, before the start of federated learning, the organization users use a random algorithm to generate additional samples, and all organization users jointly generate an additional sample library. The data in the additional sample library are all automatically generated and have nothing to do with any organization user's own user data. Each user needs to add the additional sample library to its own user database as a supplementary training set for model training. In each round of federated learning iteration, in the step where the organization user uploads the encrypted model parameters, each organization user needs to submit an encrypted certificate at the same time. The encrypted certificate is issued by any N endorsement nodes based on zero-knowledge proof. The specific method of issuing is: the endorsement node randomly extracts K samples from the additional sample library and judges the encrypted model parameters uploaded by the organization user. If the average accuracy of the N endorsement nodes is higher than the threshold λ, it is considered that the encrypted model parameters uploaded by the organization user are correct. That is, the input sample in the sample is input into the federated learning model composed of the encrypted model parameters to obtain the output sample, and the average accuracy of the output samples of the N endorsement nodes is judged.

[0109] In the embodiment of the present invention, an encryption proof mechanism is proposed based on the zero-knowledge proof concept. This mechanism can verify the correctness of the encrypted model parameters without obtaining the model parameters uploaded by the organization's users, preventing malicious participants from uploading wrong parameters or forged parameters, and solving the problem in the prior art that it is difficult to trace and punish malicious attacks that federated learning partners may face.

[0110] Please refer to Figure 2 The embodiment of the present invention further provides a federated learning model training system 20, including:

[0111] Creating module 21, for creating a blockchain network for training a federated learning model;

[0112] A first receiving module 22 is used to receive local training model parameters uploaded by an organization user during each round of training of the federated learning model, where the organization user is a blockchain user who trains the federated learning model locally using local user data;

[0113] A first sending module 23 is used to send the local training model parameters uploaded by all the organization users to the worker users, and the worker users use the local training model parameters uploaded by all the organization users to jointly train the federated learning model;

[0114] An updating module 24, configured to update the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block;

[0115] The second sending module 25 is used to send the joint training model parameters to the organization user to update the local training model parameters of the organization user.

[0116] Optionally, the local training model parameters uploaded by the organization's users are encrypted using a homomorphic encryption algorithm.

[0117] Optionally, the organizational user uses a customized key to homomorphically encrypt local training model parameters.

[0118] Optionally, also include:

[0119] The encryption module is used to determine whether the organization user has uploaded an encryption certificate at the same time when receiving the local training model parameters uploaded by the organization user. The encryption certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof; if the organization user does not send the encryption certificate, the organization user is determined to be a malicious participant.

[0120] Optionally, the system further includes:

[0121] A first receiving module is used to receive additional samples sent by organizational users before training the federated learning model to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are outputs obtained by the organizational users adding the input samples to their own local user data to train the federated learning model;

[0122] The second receiving module is used to receive local training model parameters uploaded by organizational users;

[0123] A discrimination module, used for randomly extracting K samples from the additional sample library, and performing accuracy discrimination on the parameters of the local training model according to the K samples;

[0124] The certification module is used to issue an encrypted certificate to the organization user if the average accuracy of the N endorsement nodes is higher than a preset threshold.

[0125] Optionally, the system further includes:

[0126] The initial asset granting module is used to grant initial assets to each user after the blockchain network is created;

[0127] The settlement module is used to settle assets for each user based on the transactions that occurred in each round after the converged federated learning model is obtained.

[0128] Optionally, the system further includes:

[0129] A contract deployment module, used to deploy a smart contract set on the user's blockchain node after the user joins the blockchain network. If the user is an organization user, the smart contract set includes at least one of the following: a status query contract, a model update contract, and a model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract, and a model download contract;

[0130] When the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user;

[0131] When the model update contract is called by the organization user, the local training model parameters of the organization user are uploaded to the blockchain network to complete a model parameter update transaction;

[0132] When the model download contract is called by the worker user, all local training model parameters uploaded by the organization users are downloaded from the blockchain network. When the model download contract is called by the organization user, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction.

[0133] When the model processing contract is called by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

[0134] In summary, the solution of the present invention mainly consists of the following parts:

[0135] A federated learning network. Multiple businesses in the same business domain (such as cloud services, video or communications, etc.) provide their own collected user data and conduct deep learning model training locally. The model parameters of the intermediate model obtained through training are encrypted and uploaded to a blockchain network composed of all manufacturers for joint training. The model parameters after joint training are downloaded for local model update. This iterative process results in a federated learning model for the business domain, such as a business recommendation model. Each manufacturer can use this model to make more accurate business recommendations to users. During the entire training process, all user privacy data is stored locally in the business manufacturer, avoiding the leakage of user privacy data due to shared samples.

[0136] A blockchain network is used to ensure that the data uploaded by all parties involved in federated learning cannot be tampered with and is traceable. When malicious participants try to use incorrect parameter updates to destroy the overall model effect, the cross-validation mechanism of the blockchain network can be used to quickly locate the malicious participants and avoid incorrect updates. In addition, the inherent incentive mechanism of the blockchain network allows participants with more contributions to obtain additional income compensation.

[0137] A cryptographic proof mechanism based on homomorphic encryption and zero-knowledge proof is used to ensure that the local data characteristics of each participant are not leaked and to avoid attacks by malicious participants.

[0138] A federated learning model training system is used to realize all functions involved in this technical solution, including local data collection, local modeling, model parameter addition and multi-party joint training of the final model, data upload to the blockchain, blockchain contribution incentives, and tracing of malicious attacks that federated learning partners may face.

[0139] In one embodiment, the federated learning model training system involved in this solution includes: a user information management module, a local model training module, a blockchain operation module, a settlement module and an encryption module.

[0140] The user information management module is used for basic information management such as user registration, information recording and information modification. In addition, this module also has the function of classifying users according to their asset proof (monetary assets or data assets) and limiting the user's functional permissions according to the user's category.

[0141] The local model training module is used to complete the training, updating and maintenance of the user's local model. Organization users call this module to use the user data in the local database for preliminary model training, and use the federated learning model parameters after the worker user completes the joint training to update the local model; the worker user calls this module to perform joint training on all the encrypted models collected.

[0142] The blockchain operation module is used to provide users with functions such as node joining or exiting the blockchain, initiating asset transactions, uploading or downloading model parameters, submitting encryption proofs and data traceability, and submitting user operation results to the blockchain.

[0143] The encryption module is used to provide users with the function of homomorphic encryption based on customized keys and the encryption correctness audit function based on zero-knowledge proof. In the step of uploading encrypted model parameters, the function of presenting encryption proof is completed by this module.

[0144] Please refer to Figure 3An embodiment of the present invention further provides a computing device 30, including a processor 31, a memory 32, and a computer program stored in the memory 32 and executable on the processor 31. When the computer program is executed by the processor 31, each process of the above-mentioned federated learning model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0145] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned federated learning model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. The computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0146] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0147] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a computing device, etc.) to execute the methods described in each embodiment of the present invention.

[0148] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A method for training a federated learning model, characterized in that: include: Create a blockchain network for training federated learning models; In each round of training of the federated learning model, local training model parameters uploaded by organizational users are received. The organizational users are blockchain users who use local user data to train the federated learning model locally. The blockchain users are divided into organizational users and worker users according to their contribution methods. The contribution methods of organizational users include: sharing local training model parameters and participating in the training of the federated learning model. The contribution methods of worker users include: training the federated learning model and updating model parameters, and producing new blocks. Sending the local training model parameters uploaded by all the organization users to the worker users, and the worker users jointly train the federated learning model using the local training model parameters uploaded by all the organization users; Updating the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block; Sending the joint training model parameters to the organization user to update the local training model parameters of the organization user; In each round of training of the federated learning model, the blockchain nodes of the blockchain network receive the local training model parameters uploaded by the organization users, including: When receiving the local training model parameters uploaded by the organization user, determine whether the organization user has uploaded an encrypted certificate at the same time, where the encrypted certificate is issued by any N endorsing nodes of the blockchain network based on zero-knowledge proof; If the organization user does not send the encrypted proof, the organization user is determined to be a malicious participant; Before training the federated learning model, additional samples uploaded by organizational users are received to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are the outputs obtained by the organizational users training the federated learning model by adding the input samples to their own local user data; Receive local training model parameters uploaded by organizational users; Any N endorsing nodes of the blockchain network randomly extract K samples from the additional sample library, and perform accuracy judgment on the local training model parameters according to the K samples; If the average accuracy of N endorsement nodes is higher than the preset threshold, an encrypted certificate is issued to the organization user.

2. The method according to claim 1, characterized in that The local training model parameters uploaded by the organization's users are encrypted using a homomorphic encryption algorithm.

3. The method according to claim 2, characterized in that The organizational user uses a customized key to homomorphically encrypt the local training model parameters.

4. The method according to claim 1, characterized in that Also includes: After the blockchain network is created, each user is given initial assets; After obtaining a converged federated learning model, asset settlement is performed for each user based on the transactions that occur in each round.

5. The method according to claim 4, characterized in that According to the transactions in each round, before the asset settlement for each user, the following is also included: After the user joins the blockchain network, a smart contract set is deployed on the user's blockchain node. If the user is an organization user, the smart contract set includes at least one of the following: a status query contract, a model update contract, and a model download contract; if the user is a worker user, the smart contract set includes at least one of the following: a status query contract, a model processing contract, and a model download contract; When the status query contract is called by the user, a query transaction is submitted to query the current remaining available assets of the user; When the model update contract is called by the organization user, the local training model parameters of the organization user are uploaded to the blockchain network to complete a model parameter update transaction; When the model download contract is called by the worker user, all local training model parameters uploaded by the organization users are downloaded from the blockchain network to complete a model parameter download transaction; when the model download contract is called by the organization user, the joint training model parameters uploaded by the worker user are downloaded from the blockchain network to complete a model parameter download transaction; When the model processing contract is called by the worker user, the joint training model parameters are uploaded to the blockchain network to complete a processing model parameter transaction.

6. A federated learning model training system, characterized in that: include: Create a module for creating a blockchain network for training federated learning models; The first receiving module is used to receive local training model parameters uploaded by organizational users during each round of training of the federated learning model. The organizational users are blockchain users who use local user data to train the federated learning model locally. The blockchain users are divided into organizational users and worker users according to their contribution methods. The contribution methods of the organizational users include: sharing samples and participating in the training of the federated learning model. The contribution methods of the worker users include: training the federated learning model and updating model parameters, and producing new blocks. A first sending module is used to send the local training model parameters uploaded by all the organization users to the worker users, and the worker users use the local training model parameters uploaded by all the organization users to jointly train the federated learning model; An updating module, configured to update the federated learning model using the joint training model parameters uploaded by the worker user who successfully created the block; A second sending module, used to send the joint training model parameters to the organization user to update the local training model parameters of the organization user; An encryption module is used to determine whether the organization user has uploaded an encryption certificate at the same time when receiving the local training model parameters uploaded by the organization user, and the encryption certificate is issued by any N endorsement nodes of the blockchain network based on zero-knowledge proof; if the organization user does not send the encryption certificate, the organization user is determined to be a malicious participant; A first receiving module is used to receive additional samples sent by organizational users before training the federated learning model to form an additional sample library, wherein the additional samples include multiple pairs of input samples and output samples, wherein the input samples are generated by the organizational users using a random algorithm, and the output samples are outputs obtained by the organizational users adding the input samples to their own local user data to train the federated learning model; The second receiving module is used to receive local training model parameters uploaded by organizational users; A discrimination module, used for randomly extracting K samples from the additional sample library, and performing accuracy discrimination on the parameters of the local training model according to the K samples; The certification module is used to issue an encrypted certificate to the organization user if the average accuracy of the N endorsement nodes is higher than a preset threshold.

7. A computing device, characterized in that include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the steps of the federated learning model training method as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the federated learning model training method as described in any one of claims 1 to 5.

Images