Verifiable federal learning method and device, electronic equipment, storage medium and program product

Abstract

The invention provides a verifiable federated learning method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: a trusted third-party key generation center generates a public parameter, and transmits the public parameter to a plurality of clients and a server; the client generates a mask, generates a secret key based on a public parameter, encrypts a local model parameter of the client based on the secret key to obtain a ciphertext, generates a related proof of the ciphertext and the secret key, makes a commitment for the secret key by using irreversible gradient transformation and broadcasts the commitment, and sends the ciphertext and the secret key to the server; the server verifies the ciphertext and the secret key, if verification passes, the ciphertext and the secret key are received, an aggregation result, an aggregation signature and auxiliary information are generated based on the ciphertext and the secret key, and the aggregation result, the aggregation signature and the auxiliary information are sent to the client; and the client verifies the aggregation result based on the aggregation signature and the auxiliary information, if the verification is passed, the aggregation result is received, and a local model of the client is updated based on the aggregation result.

Description

Technical Field

[0001] The present disclosure relates to the technical field of federated learning, and in particular to a verifiable federated learning method, device, electronic device, storage medium, and program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the disclosure that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] With the development of multi-data technology, the amount of data and information generated and collected in social activities has increased dramatically, and federated learning has gradually become a popular distributed machine learning paradigm. It enables devices and institutions to participate in model training without sharing their own original data, alleviates the data privacy issues caused by collecting different data sets, and effectively disintegrates the situation of data silos. However, the model gradients shared in federated learning will also leak the private information of participants, which makes the model vulnerable to attacks, such as model inversion attacks and member reasoning attacks. In order to resist privacy leakage attacks in federated learning, existing works have proposed various technologies, such as differential privacy-secure multi-party computation, differential privacy, homomorphic encryption, etc., to ensure the security of gradient information. However, most of them are based on the common assumption that the aggregator is honest but curious, and it will not deviate from the pre-arranged operations, but try to obtain some private information. In fact, the server may be malicious, and it can arbitrarily deviate from the FL protocol (Federated Learning, referred to as FL) by deliberately manipulating the training process to gain benefits. In addition to inferring customer privacy, it will also threaten the correctness of the aggregation results and weaken the availability of the training model. Summary of the invention

[0004] In view of this, the purpose of the present disclosure is to propose a verifiable federated learning method, device, electronic device, storage medium and program product, which can at least solve one of the technical problems in the related art to a certain extent.

[0005] Based on the above purpose, the first aspect of the exemplary embodiment of the present disclosure provides a verifiable federated learning method, which is implemented by a trusted third-party key generation center, multiple clients and a server, and the method includes:

[0006] The trusted third-party key generation center generates public parameters and sends the public parameters to the multiple clients and the server;

[0007] The client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server;

[0008] The server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature and the auxiliary information to the client;

[0009] The client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

[0010] Based on the same inventive concept, the second aspect of the exemplary embodiment of the present disclosure provides a verifiable federated learning device, which is implemented by a trusted third-party key generation center, multiple clients and a server, and the device includes:

[0011] A public parameter generation module, configured to generate public parameters for the trusted third-party key generation center, and send the public parameters to the multiple clients and the server;

[0012] The client upload module is configured such that the client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server;

[0013] A server download module is configured to verify the ciphertext and the key by the server, and if the verification is successful, receive the ciphertext and the key, generate an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and send the aggregation result, the aggregation signature and the auxiliary information to the client;

[0014] The client update module is configured so that the client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

[0015] Based on the same inventive concept, a third aspect of the exemplary embodiment of the present disclosure provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in the first aspect is implemented.

[0016] Based on the same inventive concept, a fourth aspect of the exemplary embodiments of the present disclosure provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect.

[0017] Based on the same inventive concept, a fifth aspect of the exemplary embodiments of the present disclosure provides a computer program product, including computer program instructions. When the computer program instructions are executed on a computer, the computer executes the method described in the first aspect.

[0018] As can be seen from the above, the verifiable federated learning method, device, electronic device, storage medium and program product provided by the embodiments of the present disclosure include: the trusted third-party key generation center generates public parameters and sends the public parameters to the multiple clients and the server; the client generates a mask through non-interactive key negotiation based on the public keys of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant certificates for the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives the commitments of other clients, and sends the ciphertext and the key to the server; the server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregate result, an aggregate signature and auxiliary information based on the ciphertext and the key, and sends the aggregate result, the aggregate signature and the auxiliary information to the client; the client verifies the aggregate result based on the aggregate signature and the auxiliary information, receives the aggregate result if the verification is successful, and updates the local model of the client based on the aggregate result. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of an application scenario of verifiable federated learning provided for an exemplary embodiment of the present disclosure;

[0021] Figure 2 A flowchart of verifiable federated learning provided for an exemplary embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of a structure of a verifiable federation device provided for an exemplary embodiment of the present disclosure;

[0023] Figure 4 A schematic structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] It is understandable that before using the technical solutions disclosed in the embodiments of this application, the type, scope of use, usage scenarios, etc. of the personal information involved in this application should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0025] For example, in response to receiving an active request from a user, auxiliary information is sent to the user to clearly inform the auxiliary information user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server or storage medium that performs the operation of the technical solution of the present application based on the auxiliary information.

[0026] As an optional but non-limiting implementation, in response to receiving an active request from the user, the auxiliary information may be sent to the user in the form of a pop-up window, in which the auxiliary information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of the present application. Other methods that meet the relevant laws and regulations may also be applied to the implementation method of the present application.

[0028] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0029] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, the principles and spirit of the present disclosure will be described with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0030] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0031] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should be understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. The article "one" or "a" before an element does not exclude the existence of multiple such elements.

[0032] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0033] As described in the background technology, with the development of multi-data technology, the amount of data and information generated and collected in social activities has increased dramatically, and federated learning has gradually become a popular distributed machine learning paradigm. It enables equipment organizations to participate in model training without sharing their own original data, alleviates the data privacy issues caused by collecting different data sets, and effectively disintegrates the situation of data silos. However, the model gradients shared in federated learning will also leak the private information of participants, which makes the model vulnerable to attacks, such as model inversion attacks and member reasoning attacks. In order to resist privacy leakage attacks in federated learning, existing works have proposed various technologies, such as differential privacy-secure multi-party computing, differential privacy, homomorphic encryption, etc., to ensure the security of gradient information. However, most of them are based on the common assumption that the aggregator is honest but curious, and it will not deviate from the pre-arranged operations, but try to obtain some private information. In fact, the server may be malicious, and it can arbitrarily deviate from the FL protocol (Federated Learning, referred to as FL) by deliberately manipulating the training process to gain benefits. In addition to inferring customer privacy, it also threatens the correctness of the aggregation results and weakens the availability of the training model.

[0034] In response to the above problems, most existing verifiable federated learning solutions only focus on the correctness of the data sent by the server, and cannot verify the information uploaded by the client. (1) Attackers may impersonate clients to upload wrong information, affecting the aggregation process and destroying the model aggregation results; (2) There may also be lazy clients who only use the global model to update their local models and refuse to provide valuable local information to the global model. (3) Secondly, malicious clients may collude with malicious servers to make other clients believe that the aggregation results have been manipulated, and ultimately bypass the verification mechanism.

[0035] In order to solve the above problems, the present disclosure provides a verifiable federated learning scheme, which specifically includes: the trusted third-party key generation center generates public parameters and sends the public parameters to the multiple clients and the server; the client generates a mask through non-interactive key negotiation based on the public keys of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives the commitments of other clients, and sends the ciphertext and the key to the server; the server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregate result, an aggregate signature and auxiliary information based on the ciphertext and the key, and sends the aggregate result, the aggregate signature and the auxiliary information to the client; the client verifies the aggregate result based on the aggregate signature and the auxiliary information, receives the aggregate result if the verification is successful, and updates the local model of the client based on the aggregate result.

[0036] Specific:

[0037] (1) Perform security protection and verification on the data during the training process. Use function encryption methods to design a more secure model aggregation method to achieve secure aggregation of model parameters; use zero-knowledge proof ideas and aggregate signature algorithms to perform security verification on the information transmitted between the client and the server to achieve verifiability of secure aggregation.

[0038] (2) Implement anti-collusion attack verification. Considering that there may be collusion between the server and the user to bypass the verification mechanism, the anti-collusion effect is achieved by combining local aggregate signatures and lightweight commitments. Gradient ciphertext is used to generate the aggregate signature of the aggregation result, and the decryption key is used for local verification. In order to achieve the purpose of anti-collusion verification, lightweight commitments are used to ensure the credibility of the verification value.

[0039] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0040] refer to Figure 1 , which is a schematic diagram of an application scenario of the verifiable federated learning method provided by the exemplary embodiment of the present disclosure.

[0041] In this application scenario, there are several client devices 101 (such as client 1, client 2 and client n), a server (also referred to as an aggregator) 102 and a trusted third-party key generation center (KGC) 103. The client devices 101, the server 102 and the trusted third-party key generation center 103 can be connected through a wired or wireless communication network to achieve data interaction.

[0042] The client device 101 may be an electronic device with data transmission and multimedia input / output functions close to the user, including but not limited to a desktop computer, a mobile phone, a mobile computer, a tablet computer, a media player, a smart wearable device, a personal digital assistant (PDA) or other electronic devices capable of implementing the above functions. The electronic device may include a processor and a display screen with a touch input function, the display screen is used to present a graphical user interface, the graphical user interface can display an application interface, and the processor is used to process application data, generate a graphical user interface, and control the display of the graphical user interface on the display screen.

[0043] The server 102 and the trusted third-party key generation center 103 can both be independent physical servers, or a server cluster or distributed system composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0044] In some exemplary embodiments, the verifiable federated learning method may be executed on the client device 101 or the server 102 .

[0045] The trusted third-party key generation center 103 generates public parameters and sends the public parameters to multiple clients 101 and servers 102;

[0046] The client 101 generates a mask through non-interactive key negotiation based on the public key of other clients 101, generates a key based on the public parameters, encrypts the local model parameters of the client 101 based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives the commitment of other clients 101, and sends the ciphertext and the key to the server 102;

[0047] The server 102 verifies the ciphertext and the key. If the verification is successful, the server 102 receives the ciphertext and the key, generates an aggregation result, an aggregation signature, and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature, and the auxiliary information to the client 101.

[0048] The client 101 verifies the aggregation result based on the aggregation signature and the auxiliary information. If the verification is successful, the client 101 receives the aggregation result and updates the local model of the client 101 based on the aggregation result.

[0049] Combine the following Figure 1 The application scenario is used to describe the verifiable federated learning method according to the exemplary embodiment of the present disclosure. It should be noted that the above application scenario is only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0050] refer to Figure 2 , which is a flowchart of a verifiable federated learning method provided by an exemplary embodiment of the present disclosure.

[0051] A verifiable federated learning method is implemented by a trusted third-party key generation center, multiple clients and a server, and the method includes the following steps:

[0052] Step S210: The trusted third-party key generation center generates public parameters, and sends the public parameters to the multiple clients and the server.

[0053] In the specific implementation, the trusted third-party key generation center (Key Generation Center, KGC) generates public parameters pm = (G, p, g, w0, H, l0, l'0, pp NIZK ,U). Where G is a multiplicative cyclic group of order p, g is the generator of G; w0 is the initialization model parameter; H represents the secure hash function; l0,l'0 is a pair of labels; pp NIZK It is the public parameter generated by calling the function NIZK.Setup in the zero-knowledge proof system; U is a singular square matrix. KGC then distributes the public parameter pm to all clients and servers.

[0054] Step S220: the client generates a mask through non-interactive key negotiation based on the public keys of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server.

[0055] In this exemplary embodiment, the client generates a mask through non-interactive key negotiation based on the public key of other clients, including:

[0056] Any one of the multiple clients generates a pair of public key and private key, and sends the public key to other clients of the multiple clients;

[0057] Generate a pair of seeds between any two clients among the multiple clients to obtain target parameters;

[0058] Each client of the plurality of clients obtains the mask based on the target parameter.

[0059] In the specific implementation, based on 1≤i≤I, I represents the number of clients, the i-th client generates a pair of public and private keys (npk i ,nsk i ), and send the public key set to other clients through the aggregator. A pair of seeds nk is generated between any two clients i and j i,j , and obtain a set of parameters {nk i,1 ,nk i,2 ,…,nk i,I}, a total of I-1. Each client uses the parameter {nk i,1 ,nk i,2 ,…,nk i,I}Calculated Where PRG is a pseudo-random algorithm.

[0060] In this exemplary embodiment, the step of generating a key based on the public parameters and encrypting the local model parameters of the client based on the key to obtain a ciphertext includes:

[0061] The client generates a private key based on the public parameter, generates a secret key based on the private key, and blinds the secret key based on the mask to obtain a partial decryption key;

[0062] The client encrypts the local model parameters of the client based on the key to obtain a gradient ciphertext.

[0063] In this exemplary embodiment, the generating of the related proof of the ciphertext and the key includes:

[0064] The client generates a correlation certificate between the ciphertext and the key using zero-knowledge proof.

[0065] In the specific implementation, the client encrypts the local model parameters and generates a signature. Based on 1≤i≤I, I represents the number of clients, each client i uses the public parameter pm to encrypt the KeyGen algorithm according to the verifiable function to generate (pk i ,vk i ,ek i ,sk i ). To improve efficiency, each gradient w with d dimensions i and key sk i Divided into groups, each containing M gradient elements. If the number of elements in the last group is less than M, add 0 padding to the remaining elements. Client i uses the private key sk i,[k] =(sk i,,[k],1 ,sk i,[k],2 …sk i,[k],M ) Calculate the key sk′ y,i,[k] =y i ·sk i,[k] ,in And use mask i Blind the key to get the partial decryption key Client i runs the verifiable function encryption Encrypt algorithm to obtain the gradient ciphertext

[0066] In the specific implementation, each client i calls the NIZK.Prove and NIZK.Prove algorithms of the zero-knowledge proof system to generate relevant proofs of partial decryption keys and ciphertexts; each client i uses the irreversible gradient transformation U to transform sk i Make a commitment Then, before uploading the ciphertext key information to the server, each client i broadcasts And receive commitments from other clients to prevent honest clients from accepting forged results.

[0067] Step S230: The server verifies the ciphertext and the key. If the verification is successful, the server receives the ciphertext and the key, generates an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature and the auxiliary information to the client.

[0068] In this exemplary embodiment, the server verifies the ciphertext and the key, and if the verification is successful, receives the ciphertext and the key, generates an aggregation result, an aggregation signature, and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature, and the auxiliary information to the client, including:

[0069] The server verifies the partial decryption key and the gradient ciphertext, and receives the partial decryption key and the gradient ciphertext if the verification is successful;

[0070] The server performs ciphertext aggregation on the partial decryption key and the gradient ciphertext to obtain an aggregation result;

[0071] The server obtains an aggregate signature based on the gradient ciphertext;

[0072] The server obtains auxiliary information based on the partial decryption key;

[0073] The server sends the aggregation result, the aggregation signature and the auxiliary information to the client.

[0074] In the specific implementation, the server receives the encrypted data. Before the ciphertext is aggregated, in order to ensure the correctness of the information uploaded by the client and prevent attackers from pretending to be clients and uploading wrong data, which will affect the aggregate decryption, the server verifies the ciphertext C uploaded by each client i through the VerifyCT and VerifyDK algorithms of the zero-knowledge proof system. l,i and key Dk y,i If the verification is successful, the server will accept the information uploaded by client i. i Upload information C l,i The gradient ciphertext ct in l,i and Dk y,i Partial decryption key dk in y,i Perform ciphertext aggregation and obtain the aggregation result according to the Decrypt algorithm encrypted by the verifiable function The [w] form represents g w .

[0075] In the specific implementation, the aggregation result is generated for verification. The server uploads the ciphertext C according to the client. l,i The gradient ciphertext ct in l,i Calculate the aggregate signature in The server then uses the partial decryption key dk y As input, an auxiliary information is calculated for each client i The server will sign Auxiliary information hint corresponding to client i iSent to the corresponding client i for verification.

[0076] Step S240: The client verifies the aggregation result based on the aggregation signature and the auxiliary information. If the verification is successful, the client receives the aggregation result and updates the local model of the client based on the aggregation result.

[0077] In this exemplary embodiment, the client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result, including:

[0078] The client determines whether the aggregate signature, the auxiliary information, and the aggregate result satisfy a preset condition;

[0079] If satisfied, the client verifies the auxiliary information through the commitment and the key;

[0080] If the verification is successful, the client receives the aggregation result and updates the local model of the client based on the aggregation result.

[0081] In specific implementation, each client i receives the aggregated result and signature. And auxiliary information hint i Verify the correctness of the aggregation results. First, determine Is it equal to [w]+[u l ](hint i +dk y,i ), where w is the gradient aggregation result sent by the server and received by the client. Client i uses the commitment and its own key dk y,i Hint for auxiliary information i Verify and judge U·(hint i +dk y,i ) is equivalent to ∑ i C i Using the above two equations, the client can detect malicious behavior of the server. If the two equations are true, the aggregation result is accepted and the current round of federated learning training process ends; otherwise, it is rejected.

[0082] Through the above exemplary embodiments, bidirectional verification of the transmitted data is achieved. The existing verifiable federated learning mainly targets the problem of incorrect aggregation results being transmitted from the server, without considering that malicious clients may upload forged information. Therefore, this scheme combines verifiable multi-client function encryption and local aggregate signature schemes to achieve security verification of data transmitted between clients and servers, effectively resisting Byzantine attacks, and erroneous updates injected by malicious clients or servers can be effectively detected. Bidirectional verification is performed on data transmitted between servers and clients. Verification of the server's aggregation results and user-uploaded information effectively prevents malicious servers and attackers from impersonating participants to forge information, disrupting the model training process, and damaging the training effect.

[0083] It achieves anti-collusion attack. In order to prevent the server and the client from colluding to bypass the aggregation result verification mechanism, the user combines the aggregate signature, auxiliary information and lightweight commitment to verify the aggregation result, thus achieving anti-collusion verification. While the client generates ciphertext to protect the privacy and security of local information, it uses the ciphertext to generate the aggregate signature and uses the decryption key as auxiliary information for verification. Lightweight commitment is used to ensure the credibility of the auxiliary information, effectively resisting the collusion between the server and the client. It achieves anti-collusion attack using local aggregate signature and lightweight commitment. Regardless of whether the server and the client collude, the aggregation result forged by the server cannot successfully bypass the verification mechanism, preventing the server and malicious clients from manipulating the aggregation results to pass the verification and deceive ordinary users.

[0084] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0085] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a verifiable federated learning device.

[0087] refer to Figure 3, a verifiable federated learning device is implemented by a trusted third-party key generation center, multiple clients and servers, and the device includes:

[0088] A public parameter generation module 910 is configured to generate public parameters for the trusted third-party key generation center and send the public parameters to the multiple clients and the server;

[0089] The client upload module 920 is configured such that the client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server;

[0090] The server download module 930 is configured to verify the ciphertext and the key by the server, and if the verification is successful, receive the ciphertext and the key, generate an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and send the aggregation result, the aggregation signature and the auxiliary information to the client;

[0091] The client update module 940 is configured so that the client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

[0092] In some exemplary embodiments, the client upload module 920 is specifically configured to:

[0093] Any one of the multiple clients generates a pair of public key and private key, and sends the public key to other clients of the multiple clients;

[0094] Generate a pair of seeds between any two clients among the multiple clients to obtain target parameters;

[0095] Each client of the plurality of clients obtains the mask based on the target parameter.

[0096] In some exemplary embodiments, the client upload module 920 is specifically configured to:

[0097] The client generates a private key based on the public parameter, generates a secret key based on the private key, and blinds the secret key based on the mask to obtain a partial decryption key;

[0098] The client encrypts the local model parameters of the client based on the key to obtain a gradient ciphertext.

[0099] In some exemplary embodiments, the client upload module 920 is specifically configured to:

[0100] The client generates a correlation certificate between the ciphertext and the key using zero-knowledge proof.

[0101] In some exemplary embodiments, the server download module 930 is specifically configured as follows:

[0102] The server verifies the partial decryption key and the gradient ciphertext, and receives the partial decryption key and the gradient ciphertext if the verification is successful;

[0103] The server performs ciphertext aggregation on the partial decryption key and the gradient ciphertext to obtain an aggregation result;

[0104] The server obtains an aggregate signature based on the gradient ciphertext;

[0105] The server obtains auxiliary information based on the partial decryption key;

[0106] The server sends the aggregation result, the aggregation signature and the auxiliary information to the client.

[0107] In some exemplary embodiments, the client update module 940 is specifically configured to:

[0108] The client determines whether the aggregate signature, the auxiliary information, and the aggregate result satisfy a preset condition;

[0109] If satisfied, the client verifies the auxiliary information through the commitment and the key;

[0110] If the verification is successful, the client receives the aggregation result and updates the local model of the client based on the aggregation result.

[0111] For the convenience of description, the above device is described by dividing it into various modules according to its functions. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0112] The device of the above embodiment is used to implement the corresponding verifiable federated learning method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0113] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the verifiable federated learning method described in any of the above embodiments is implemented.

[0114] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0115] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0116] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0117] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0118] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0119] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0120] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0121] The electronic device of the above embodiment is used to implement the corresponding verifiable federated learning method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0122] The memory 1020 stores machine-readable instructions executable by the processor 1010. When the electronic device is running, the processor 1010 communicates with the memory 1020 via the bus 1030, so that the processor 1010 executes the following instructions when running:

[0123] The trusted third-party key generation center generates public parameters and sends the public parameters to the multiple clients and the server;

[0124] The client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server;

[0125] The server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature and the auxiliary information to the client;

[0126] The client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

[0127] In a possible implementation, in the instructions executed by the processor 1010, the client generates a mask through non-interactive key negotiation based on the public key of another client, including:

[0128] Any one of the multiple clients generates a pair of public key and private key, and sends the public key to other clients of the multiple clients;

[0129] Generate a pair of seeds between any two clients among the multiple clients to obtain target parameters;

[0130] Each client of the plurality of clients obtains the mask based on the target parameter.

[0131] In a possible implementation manner, in the instructions executed by the processor 1010, generating a key based on the public parameter, and encrypting the local model parameter of the client based on the key to obtain a ciphertext includes:

[0132] The client generates a private key based on the public parameter, generates a secret key based on the private key, and blinds the secret key based on the mask to obtain a partial decryption key;

[0133] The client encrypts the local model parameters of the client based on the key to obtain a gradient ciphertext.

[0134] In a possible implementation, in the instructions executed by the processor 1010, the generating of the related proof of the ciphertext and the key includes:

[0135] The client generates a correlation certificate between the ciphertext and the key using zero-knowledge proof.

[0136] In a possible implementation, in the instructions executed by the processor 1010, the server verifies the ciphertext and the key, and if the verification is successful, receives the ciphertext and the key, generates an aggregation result, an aggregation signature, and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature, and the auxiliary information to the client, including:

[0137] The server verifies the partial decryption key and the gradient ciphertext, and receives the partial decryption key and the gradient ciphertext if the verification is successful;

[0138] The server performs ciphertext aggregation on the partial decryption key and the gradient ciphertext to obtain an aggregation result;

[0139] The server obtains an aggregate signature based on the gradient ciphertext;

[0140] The server obtains auxiliary information based on the partial decryption key;

[0141] The server sends the aggregation result, the aggregation signature and the auxiliary information to the client.

[0142] In a possible implementation, in the instructions executed by the processor 1010, the client verifies the aggregation result based on the aggregate signature and the auxiliary information, receives the aggregation result if the verification passes, and updates the local model of the client based on the aggregation result, including:

[0143] The client determines whether the aggregate signature, the auxiliary information, and the aggregate result satisfy a preset condition;

[0144] If satisfied, the client verifies the auxiliary information through the commitment and the key;

[0145] If the verification is successful, the client receives the aggregation result and updates the local model of the client based on the aggregation result.

[0146] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the verifiable federated learning method described in any of the above embodiments.

[0147] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0148] The above-mentioned non-transitory computer-readable storage medium can be any available medium or data storage device that can be accessed by a computer, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0149] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the verifiable federated learning method described in any embodiment in the above exemplary method part, and have the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0150] Based on the same inventive concept, corresponding to the verifiable federated learning method described in any of the above embodiments, the present disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor executes the verifiable federated learning method. Corresponding to the execution subject corresponding to each step in each embodiment of the verifiable federated learning method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0151] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the verifiable federated learning method described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0152] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." In addition, in some embodiments, the present disclosure may also be implemented in the form of a computer program product in one or more computer-readable media, which contains computer-readable program code.

[0153] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive examples) of computer-readable storage media may include, for example: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0154] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0155] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0156] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified in the boxes in the flowchart and / or block diagram.

[0158] These computer program instructions may also be stored in a computer-readable medium that enables a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable medium produce a product that includes an instruction device that implements the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0159] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable device to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0160] In addition, although the operations of the disclosed method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0161] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0162] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0163] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0164] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0165] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0166] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

[0167] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims. The scope of the attached claims conforms to the broadest interpretation, thereby including all such modifications and equivalent structures and functions.

Claims

1. A verifiable federated learning method, characterized in that: The method is implemented by a trusted third-party key generation center, multiple clients and servers, and includes: The trusted third-party key generation center generates public parameters and sends the public parameters to the multiple clients and the server; The client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server; The server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature and the auxiliary information to the client; The client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

2. The method according to claim 1, characterized in that The client generates a mask through non-interactive key negotiation based on the public key of the other client, including: Any one of the multiple clients generates a pair of public key and private key, and sends the public key to other clients of the multiple clients; Generate a pair of seeds between any two clients among the multiple clients to obtain target parameters; Each client of the plurality of clients obtains the mask based on the target parameter.

3. The method according to claim 1, characterized in that The step of generating a key based on the public parameter and encrypting the local model parameter of the client based on the key to obtain a ciphertext includes: The client generates a private key based on the public parameter, generates a secret key based on the private key, and blinds the secret key based on the mask to obtain a partial decryption key; The client encrypts the local model parameters of the client based on the key to obtain a gradient ciphertext.

4. The method according to claim 1, characterized in that: The generating of the ciphertext and the key-related proof includes: The client generates a correlation certificate between the ciphertext and the key using zero-knowledge proof.

5. The method according to claim 3, characterized in that: The server verifies the ciphertext and the key, receives the ciphertext and the key if the verification is successful, generates an aggregation result, an aggregation signature, and auxiliary information based on the ciphertext and the key, and sends the aggregation result, the aggregation signature, and the auxiliary information to the client, including: The server verifies the partial decryption key and the gradient ciphertext, and receives the partial decryption key and the gradient ciphertext if the verification is successful; The server performs ciphertext aggregation on the partial decryption key and the gradient ciphertext to obtain an aggregation result; The server obtains an aggregate signature based on the gradient ciphertext; The server obtains auxiliary information based on the partial decryption key; The server sends the aggregation result, the aggregation signature and the auxiliary information to the client.

6. The method according to claim 1, characterized in that The client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result, including: The client determines whether the aggregate signature, the auxiliary information, and the aggregate result satisfy a preset condition; If satisfied, the client verifies the auxiliary information through the commitment and the key; If the verification is successful, the client receives the aggregation result and updates the local model of the client based on the aggregation result.

7. A verifiable federated learning device, characterized in that: The device is implemented by a trusted third-party key generation center, multiple clients and servers, and includes: A public parameter generation module, configured to generate public parameters for the trusted third-party key generation center, and send the public parameters to the multiple clients and the server; The client upload module is configured such that the client generates a mask through non-interactive key negotiation based on the public key of other clients, generates a key based on the public parameters, encrypts the local model parameters of the client based on the key to obtain a ciphertext, generates relevant proofs of the ciphertext and the key, makes a commitment to the key using an irreversible gradient transformation, broadcasts the commitment and receives commitments from other clients, and sends the ciphertext and the key to the server; A server download module is configured to verify the ciphertext and the key by the server, and if the verification is successful, receive the ciphertext and the key, generate an aggregation result, an aggregation signature and auxiliary information based on the ciphertext and the key, and send the aggregation result, the aggregation signature and the auxiliary information to the client; The client update module is configured so that the client verifies the aggregation result based on the aggregation signature and the auxiliary information, receives the aggregation result if the verification is successful, and updates the local model of the client based on the aggregation result.

8. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer program instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images