A safe and efficient model-verifiable aggregation method

Abstract

The present invention relates to a safe and efficient model verifiable aggregation method, which belongs to the field of artificial intelligence machine learning and comprises the following steps: S1: a trusted institution initializes a global model and generates a key; S2: local training is performed; S3: a server aggregates model ciphertexts uploaded by participants according to Top-k index values, and returns the aggregated result to all participants; S4: each participant first verifies whether the verification code broadcast by other participants is valid according to the signature, and aggregates the verification code with its own verification code if the verification is passed; then the participant uses the private key to decrypt the aggregated result returned by the server to obtain the plaintext of the aggregated model; then the correctness of the aggregated result is verified by the verification code. If the verification fails, the participant will reject the model returned by the server and exit the training; otherwise, the local model is updated and a new round of training is started until the model meets the accuracy requirement.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence machine learning and relates to a safe and efficient model-verifiable aggregation method. Background Art

[0002] To protect the interests and privacy of data providers, the industry has proposed federated learning, a distributed machine learning model where "data remains static, model remains dynamic." In this framework, a parameter aggregation server aggregates the model parameters provided by each participant to generate a global model. However, as a third party, the trustworthiness of parameter servers is questionable. For example, a malicious server could reconstruct a participant's original data through inference attacks based on the parameters uploaded by the participant, thereby stealing the participant's sensitive information. Furthermore, if motivated by illicit interests or under attack, the server might return an erroneous aggregated model, misleading training results.

[0003] There are already some academic achievements related to secure model aggregation. For example, Xu G, Li H, and Liu S proposed the VerifyNet scheme in "Verifynet: Secure and verifiable federated learning" [in IEEE Transactions on Information Forensics and Security, pp. 911-926, 2019]. This scheme uses homomorphic hash functions and Shamir secret sharing to achieve secure gradient aggregation and verification. However, the verification overhead of this scheme is linear in the dimension of the gradient vector, which cannot effectively meet the needs of secure model aggregation in resource-constrained scenarios. Guo X, Liu Z, and Li J proposed a lightweight verifiable aggregation scheme in "VeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning" [in Transactions on Information Forensics and Security, pp. 1736-1751, 2020]. By adopting commitment techniques and linear homomorphic hash functions, it significantly reduces the system's communication overhead. However, the performance of this scheme degrades as the number of exiting clients increases. Fu A, Zhang X, and Xiong N in "VFL: A verifiable federated learning with privacy-preserving for big data in industrial IoT" [in IEEE Transactions on Industrial Informatics, pp 3316-3326, 2020] achieved verifiable aggregation of the model through blinding technology and Lagrange interpolation. The overhead of the verification mechanism is independent of the number of clients. However, this method does not support the situation where the client is offline and cannot meet the needs of practical applications.

[0004] Although research on model privacy protection and verifiable aggregation has achieved promising results, it still faces many challenges in IoT scenarios. First, IoT devices typically have limited computing power, making complex verification schemes difficult to implement on them. Second, during federated learning, the network bandwidth of IoT devices is often limited. As the number of network parameters increases, communication costs and latency also increase, severely impacting the efficiency of federated learning. Furthermore, IoT devices may drop out of training tasks due to network interruptions or device failures. Therefore, the proposed solution needs to support dynamic participation of IoT devices in training. Summary of the Invention

[0005] In light of this, the present invention aims to provide a secure and efficient model-verifiable aggregation method. This method compresses local models using a Top-k sparse approach to reduce device communication overhead, protects the privacy of uploaded model parameters through homomorphic encryption, and ultimately implements a lightweight verification method to verify model correctness. This method is robust to device disconnections, eliminating the need for additional operations when a device logs out.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A safe and efficient model-verifiable aggregation method includes the following steps:

[0008] S1: The trusted authority initializes the global model and generates keys;

[0009] S2: Participants use private data to train a model to obtain a local model, and use the Top-k sparse method to compress the model, retaining k important parameters. They then calculate the verification code of the local model and broadcast the verification code and its signature to all other participants. The compressed model is then encrypted using the Paillier encryption algorithm. Finally, the model ciphertext and Top-k index values are uploaded to the server for aggregation.

[0010] S3: The server aggregates the model ciphertexts uploaded by participants based on the Top-k index values and returns the aggregation results to all participants;

[0011] S4: Each participant first verifies the validity of the verification code broadcast by other participants based on the signature. If the verification is successful, it aggregates it with its own verification code. The participant then uses the private key to decrypt the aggregated result returned by the server to obtain the plaintext of the aggregated model. The correctness of the aggregated result is then verified using the verification code. If the verification fails, the participant will reject the model returned by the server and exit training. Otherwise, the local model will be updated and a new round of training will begin until the model meets the accuracy requirements.

[0012] Furthermore, the trusted institution in step S1 initializes the global model and generates a key, specifically including:

[0013] S11: Model initialization: The trusted institution initializes the model parameters M and learning rate α based on the neural network model architecture agreed upon by all participants;

[0014] S12: Key initialization: The trusted institution generates a public-private key pair (pk, sk) for the system based on the Paillier encryption algorithm, and secretly sends the private key to all participants; then generates a different key pair (pk i ,sk i ).

[0015] Furthermore, step S2 specifically includes the following steps:

[0016] S21: Participants train the local model g according to the SGD algorithm i ;

[0017] S22: Local model g is sparsely distributed using the Top-k model method. i Compression

[0018] S23: Participants calculate the verification code of the compressed model according to the verification code generation rules:

[0019]

[0020] Among them, k represents the index value of the model dimension;

[0021] Then use your own private key sk i Sign the verification code:

[0022] sg i =sign(MAC i ,sk i )

[0023] Among them, sign represents the signature algorithm, whose input is the message authentication code MAC i and private key, the output is a signature of length L;

[0024] Then (MAC i ,sg i ) broadcast to all other participants;

[0025] S24: Participants use the system public key to calculate the ciphertext of the compressed model according to the Paillier encryption algorithm

[0026] S25: Data upload: Participants will And the index value of the Top-k model is uploaded to the server for aggregation.

[0027] Furthermore, the global aggregation described in step S3 specifically includes:

[0028] S31: The aggregation server aggregates the global model based on the index values uploaded by the participants:

[0029]

[0030] Among them, P1 represents the set of participants who uploaded the model in the current round;

[0031] S32: The aggregation server returns the aggregated global model.

[0032] Furthermore, step S4 specifically includes the following steps:

[0033] S41: Each participant verifies the received verification code:

[0034] verify{MAC j ,sg j ,pk j}

[0035] Among them, verify() represents the verification algorithm, whose input is participant P j Model Validation Code MAC j 、Signature sg j And the public key pk j , the verification algorithm outputs True when the signature is valid, otherwise it outputs False;

[0036] If the verification is successful, the verification code is aggregated with its own verification code:

[0037]

[0038] S42: Participants use the system private key sk to decrypt the result returned by the server according to the principle of Paillier encryption algorithm, thereby obtaining the plaintext of the aggregation model

[0039] S43: Participants verify the correctness of the aggregation model and confirm MAC agg Whether If the verification results are the same, it means that the server has aggregated the data correctly and no tampering has occurred during transmission;

[0040] S44: After verification, the participant accepts the results returned by the server and fills in the values of the missing dimensions, and then starts a new round of training.

[0041] The beneficial effects of this invention are: First, while protecting the privacy of participants, the correctness of the aggregation results can be verified, thereby improving the credibility and stability of the model. Second, this method can cope with the situation where participants drop out of training, and the system has strong robustness. In addition, this method compresses the size of the local model through the Top-k algorithm, has the characteristics of lightweight communication overhead, and can be adapted to resource-constrained training scenarios, thereby improving practicality and scalability.

[0042] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0044] Figure 1 This is a system architecture diagram of the present invention;

[0045] Figure 2 Flowchart of a safe and efficient model-verifiable aggregation method designed for the present invention. DETAILED DESCRIPTION

[0046] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0047] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0048] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0049] like Figure 1 The following is a secure and efficient model verifiable aggregation framework, which specifically includes the following entities:

[0050] Trusted institution: The trusted institution is responsible for generating and distributing the necessary parameters and keys for the entire training.

[0051] Aggregation Server: The aggregation server is responsible for aggregating the uploaded ciphertext and distributing the results to participants in each round. As a third-party aggregation server, it is untrustworthy. It may attempt to steal participants' private information through the received data and may even forge aggregation results to influence model updates.

[0052] Participants: Participants have independent local data and use it to train local models. They then upload this data to the server for aggregation, aiming to obtain a global model with better performance and greater generalization capabilities. Due to privacy and security considerations, participants encrypt their local models before uploading them and generate corresponding model verification codes to verify the correctness of the results returned by the server.

[0053] Figure 2 The safe and efficient model-verifiable aggregation method process of the present invention is shown, which includes the following steps:

[0054] 1) In the initialization phase, the trusted organization generates and distributes the necessary parameters and keys for the entire training. The specific steps are as follows:

[0055] Down:

[0056] (1) Parameter initialization. The trusted institution initializes the model parameters M, learning rate α, and number of training rounds T based on the neural network model architecture agreed upon by all participants.

[0057] (2) Key initialization. Given a security parameter The trusted authority executes the Paillier key generation algorithm to generate the system's public and private key pairs The private key sk is secretly distributed to all participants, and the public key pk is made public to all entities. In addition, the trusted institution also provides each participant P i Generate their respective key pairs (pk i ,sk i ).

[0058] 2) Local training phase, the specific steps are as follows:

[0059] (1) Compute the local model. Participant P i According to the global model, the local model is trained using private data to obtain the model g i This process is usually based on the stochastic model descent algorithm, which is a common method for finding optimal models in machine learning.

[0060] (2) Compression model. Participant P i Train to get the local model g i Then, it is compressed by the Top-k model sparse method That is, the first k dimensions with the largest absolute values in the original model are selected for uploading, thereby reducing redundant transmission of the model and lowering communication overhead.

[0061] (3) Calculate the verification code. Participants calculate the verification code of the compressed model as follows:

[0062]

[0063] Among them, k represents the index value of the model dimension. Then, sign it with your own private key

[0064] sg i =sign(MAC i ,sk i )

[0065] Among them, sign represents the signature algorithm, whose input is the message authentication code MAC i and private key, the output is a signature of length L. Then (MAC i ,sg i ) is broadcast to all other participants. The communication overhead for verification in the present invention is independent of the dimension of the model and the number of participants, and is only related to R and the length L of the signature.

[0066] (4) Encrypted model. Participants encrypt the local model using the system public key according to the Paillier encryption algorithm.

[0067] (5) Data upload. Participants upload the model ciphertext and Top-k index values to the server for aggregation.

[0068] 3) Global aggregation stage, the specific steps are as follows:

[0069] (1) Model aggregation. The aggregation server aggregates the global model based on the index values uploaded by the participants:

[0070]

[0071] Among them, P1 represents the set of participants who uploaded the model in the current round.

[0072] (2) Model distribution: The aggregation server returns the aggregated global model.

[0073] 4) Model update phase, the specific steps are as follows:

[0074] (1) Signature verification. Participant P i Verify the received verification code:

[0075] verify(MAC j ,sg j ,pk j )

[0076] Among them, verify() represents the verification algorithm, whose input is participant P j Model Validation Code MAC j 、Signature sg j And the public key pk j , the verification algorithm outputs True when the signature is valid, otherwise it outputs False.

[0077] If the verification is successful, the verification code will be aggregated with its own verification code:

[0078]

[0079] (2) Model decryption. Participants use the system private key sk to decrypt the result returned by the server according to the principle of Paillier encryption algorithm, thereby obtaining the plaintext of the aggregated model.

[0080] (3) Correctness verification. Participants verify the correctness of the aggregation model:

[0081]

[0082] Participants verify the correctness of the aggregation results based on the above equation. If the equation is true, it means that the server has aggregated the data correctly and the model has not been tampered with during transmission.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. A safe and efficient model-verifiable aggregation method, characterized by: The following steps are involved: S1: The trusted authority initializes the global model and generates keys; S2: Participants use private data to train a model to obtain a local model, and use the Top-k sparse method to compress the model, retaining k important parameters. They then calculate the verification code of the local model and broadcast the verification code and its signature to all other participants. The compressed model is then encrypted using the Paillier encryption algorithm. Finally, the model ciphertext and Top-k index values are uploaded to the server for aggregation. S3: The server aggregates the model ciphertexts uploaded by participants based on the Top-k index values and returns the aggregation results to all participants; S4: Each participant first verifies the validity of the verification code broadcast by other participants based on the signature. If the verification is successful, it aggregates it with its own verification code. The participant then uses the private key to decrypt the aggregated result returned by the server to obtain the plaintext of the aggregated model. The correctness of the aggregated result is then verified using the verification code. If the verification fails, the participant will reject the model returned by the server and exit training. Otherwise, the local model will be updated and a new round of training will begin until the model meets the accuracy requirements.

2. The safe and efficient model-verifiable aggregation method according to claim 1, characterized in that: The trusted institution in step S1 initializes the global model and generates a key, specifically including: S11: Model initialization: The trusted institution initializes the model parameters M and learning rate α based on the neural network model architecture agreed upon by all participants; S12: Key initialization: The trusted institution generates a public-private key pair (pk, sk) for the system based on the Paillier encryption algorithm, and secretly sends the private key to all participants; then generates a different key pair (pk i ,sk i ).

3. The safe and efficient model-verifiable aggregation method according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21: Participants train the local model g according to the SGD algorithm i ; S22: Local model g is sparsely distributed using the Top-k model method. i Compression S23: Participants calculate the verification code of the compressed model according to the verification code generation rules: Among them, k represents the index value of the model dimension; Then use your own private key sk i Sign the verification code: sg i =sign(MAC i ,sk i ) Among them, sign represents the signature algorithm, whose input is the message authentication code MAC i and private key, the output is a signature of length L; Then (MAC i ,sg i ) broadcast to all other participants; S24: Participants use the system public key to calculate the ciphertext of the compressed model according to the Paillier encryption algorithm S25: Data upload: Participants will And the index value of the Top-k model is uploaded to the server for aggregation.

4. The safe and efficient model-verifiable aggregation method according to claim 3, characterized in that: The global aggregation described in step S3 specifically includes: S31: The aggregation server aggregates the global model based on the index values uploaded by the participants: Among them, P1 represents the set of participants who uploaded the model in the current round; S32: The aggregation server returns the aggregated global model.

5. The safe and efficient model-verifiable aggregation method according to claim 4, characterized in that: The step S4 specifically includes the following steps: S41: Each participant verifies the received verification code: verify{MAC j ,sg j ,pk j } Among them, verify() represents the verification algorithm, whose input is participant P j Model Validation Code MAC j 、Signature sg j And the public key pk j , the verification algorithm outputs True when the signature is valid, otherwise it outputs False; If the verification is successful, the verification code is aggregated with its own verification code: S42: Participants use the system private key sk to decrypt the result returned by the server according to the principle of Paillier encryption algorithm, thereby obtaining the plaintext of the aggregation model S43: Participants verify the correctness of the aggregation model and confirm MAC agg Whether If the verification results are the same, it means that the server has aggregated the data correctly and no tampering has occurred during transmission; S44: After verification, the participant accepts the results returned by the server and fills in the values of the missing dimensions, and then starts a new round of training.

Images