A distributed federated learning aggregation method, terminal and readable storage medium

Abstract

The present invention relates to the field of machine learning technology, and discloses a distributed federated learning aggregation method, terminal and readable storage medium, wherein the method comprises: obtaining local model parameters of this round of iteration, dividing the local model parameters into multiple secret shares corresponding to the participants, sending each of the secret shares to the corresponding participants, and receiving the secret shares sent by the participants; performing aggregation locally according to the secret shares to obtain a local aggregation vector, and sharing the local aggregation vector with a manager server; receiving a global aggregation model updated by the manager server, and updating the local model according to the global aggregation model. The present invention divides the local model parameters by secret division, so that each participant can only obtain the divided local model parameters, and the manager server can only obtain the final aggregation calculation result, but cannot obtain the intermediate local model parameters, thereby effectively defending against reasoning attacks.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a distributed federated learning aggregation method, a terminal and a readable storage medium. Background Art

[0002] In traditional centralized machine learning, data is usually collected on a central server for model training. However, this method has the risk of privacy leakage and the problem of data centralization. To this end, the existing technology proposes an emerging machine learning method, namely Federated Learning (FL). Federated Learning trains the model on the local device and sends the updated information of the model to the central server for integration without sharing the original data, thereby achieving collaborative learning under the premise of protecting data privacy. However, the current situation shows that although federated learning improves data privacy and security, it is still vulnerable to reasoning attacks and poisoning attacks.

[0003] Among them, an inference attack refers to an attacker trying to infer information about training data from the model. In a federated learning scenario, even if the data is not directly shared, the attacker can still infer some sensitive information by analyzing model updates (such as weight updates); a poisoning attack refers to an attacker deliberately modifying its local data or model updates, so that the performance of the centralized model degrades or tends to make wrong decisions.

[0004] In order to deal with poisoning attacks, existing technologies have proposed a variety of aggregation algorithms that rely on robustness, such as using median or mean clipping techniques to aggregate model updates, which can reduce the impact of abnormal updates on the global model; or anomaly detection technology, that is, performing anomaly detection before model updates to identify and exclude updates that may be submitted by malicious parties.

[0005] However, these methods require verification and calculation of the model parameters provided by the participants, which results in the model parameters being made public in plain text. Attackers can infer sensitive information by analyzing the model parameters.

[0006] Therefore, a technical solution is needed to simultaneously address these two attacks, ensuring the robustness and reliability of the model while protecting data privacy. Summary of the invention

[0007] The purpose of the present invention is to provide a distributed federated learning aggregation method, terminal and readable storage medium, which aims to solve the problem that the existing aggregation algorithm relying on robustness relies on plaintext shared parameters for calculation and lacks privacy protection, resulting in the inability to resist reasoning attacks.

[0008] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0009] The present invention provides a distributed federated learning aggregation method, which includes:

[0010] Obtaining the local model parameters of this round of iteration, dividing the local model parameters into a plurality of secret shares corresponding to the participants, sending each of the secret shares to the corresponding participants, and receiving the secret shares sent by the participants;

[0011] Aggregating locally according to the secret share to obtain a local aggregation vector, and sharing the local aggregation vector with the manager server;

[0012] Receive the global aggregation model updated by the manager server, and update the local model according to the global aggregation model.

[0013] Furthermore, dividing the local model parameters into a plurality of secret shares corresponding to the participants specifically includes:

[0014] Get threshold value , according to the threshold value Establish Polynomial of degree;

[0015] according to The secret share corresponding to each participant is calculated based on the local model parameters using a polynomial.

[0016] Further, the locally aggregating according to the secret share to obtain a local aggregation vector specifically includes:

[0017] Calculating using a robust aggregation algorithm according to the secret share to obtain a local robust aggregation parameter;

[0018] Get the local robustness aggregation parameters transmitted by the participants, according to More than 100 local robust aggregation parameters are used to restore the global robust aggregation parameters;

[0019] Aggregation is performed locally according to the global robustness aggregation parameter and the secret share to obtain a local aggregation vector.

[0020] Furthermore, the distributed federated learning aggregation method further comprises: verifying the participants to obtain verified honest participants;

[0021] Submit the verified honest party to the manager server.

[0022] Furthermore, the verification of the participant specifically includes:

[0023] Verify the shared consistency of the secret shares of all participants;

[0024] The computational effectiveness of all participants in calculating local robustness aggregation parameters is verified.

[0025] Furthermore, the shared consistency verification of the secret shares of all participants specifically includes:

[0026] Get Public Polynomial ;

[0027] Get the local client number j in the system to satisfy the security share of shared consistency regulation The corresponding party is the honest party, where: For the i-th participant The secret share sent to itself;

[0028] The security share that satisfies the shared consistency regulation satisfy: The value of is The value of the polynomial at the variable value j;

[0029] in, for The value of the variable is j.

[0030] Furthermore, the verification of the computational validity of all participants in calculating the local robustness aggregation parameters specifically includes:

[0031] Get the multiplication triples of all participants in computing the local robustness aggregation parameters;

[0032] All multiplication triplets are verified, and the parties that pass the verification are the honest parties.

[0033] Furthermore, if the number of the honest participants is greater than a security threshold, aggregation is performed locally according to the secret share to obtain a local aggregation vector;

[0034] If the number of honest participants is less than the safety threshold, the distributed federated learning aggregation method is terminated.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, which includes: a memory, a processor, and a distributed federated learning aggregation program stored on the memory and executable on the processor, and when the distributed federated learning aggregation program is executed by the processor, the terminal is controlled to implement the steps of the distributed federated learning aggregation method as described above.

[0036] In addition, to achieve the above-mentioned purpose, the present invention also provides a readable storage medium, which stores a distributed federated learning aggregation program, and when the distributed federated learning aggregation program is executed by a processor, it implements the steps of the distributed federated learning aggregation method as described above.

[0037] The present invention adopts the above technical solution to achieve the following effects:

[0038] The present invention solves the privacy problem of existing aggregation algorithms by splitting the local model parameters in a secret splitting manner. In the process of federated learning, each participant can only obtain the split local model parameters, and the manager server can only obtain the final aggregation calculation results, but cannot obtain the intermediate local model parameters. This effectively solves the problem of lack of privacy protection in existing aggregation algorithms that rely on robustness, thereby preventing inference attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flowchart of the steps of the distributed federated learning aggregation method in a preferred embodiment of the present invention;

[0040] Figure 2 It is a system framework diagram of a distributed federated learning aggregation method in a preferred embodiment of the present invention;

[0041] Figure 3 is a protocol flow chart of any two parties in a preferred embodiment of the present invention;

[0042] Figure 4 This is a graph showing the single-round aggregation time results of the present invention against different attacks on the Fashion-MNIST dataset;

[0043] Figure 5 This is a graph showing the single-round aggregation time results of the present invention against different attacks on the CIFAR10 dataset:

[0044] Figure 6 It is the time cost diagram of zero-knowledge proof generation and verification of the present invention under 1K model parameters;

[0045] Figure 7 It is the time cost diagram of zero-knowledge proof generation and verification under 100 clients of the present invention;

[0046] Figure 8 It is a communication overhead diagram of zero-knowledge proof generation and verification under 1K model parameters of the present invention;

[0047] Fig. 9 It is the communication overhead diagram of zero-knowledge proof generation and verification under 100 clients of the present invention;

[0048] Fig.10It is the comparison result of the convergence and accuracy test of the aggregation model of the present invention and the defenseless one under malicious attack based on the Fashion-MNIST dataset;

[0049] Fig.11 It is a schematic diagram of the operating environment of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail with reference to the accompanying drawings and examples. It should be understood that the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention. All other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0051] Embodiment 1

[0052] The first embodiment of the present application is a distributed federated learning aggregation method. The traditional federated learning aggregation algorithm can be described as follows: in the T+1 round aggregation process, the aggregation model The aggregation formula is:

[0053] ;

[0054] Where n is the number of participants participating in the T+1 round of aggregation process, which is a subset of the total number of participants N. is the i-th participant The local model obtained in round T update, is the i-th participant The local objective function is used to describe How to meet the participants Dataset . The general form of can be written as:

[0055] ;

[0056] in, is the i-th participant The number of data samples, J function is a loss function, which is used to measure the given model parameters The prediction error under and are the qth sample and qth label respectively.

[0057] The global objective function of the aggregation process is to minimize the loss function In this embodiment, the Stochastic Gradient Descent (SGD) algorithm is used as the benchmark method, and the global objective function is described as:

[0058] ;

[0059] Where M represents the model, represents the i-th participant The loss function is represents the i-th participant The local data set size is , each participant has its own objective function and obtains its own model. The aggregate model is the average of the models of each participant.

[0060] It can be seen that the federated learning aggregation algorithm obtains the final model through multiple rounds of aggregation, that is, multiple iterative training. Therefore, the distributed federated learning aggregation method of the present invention includes multiple training iterations. Figure 1 and Figure 2 , each training iteration includes the steps:

[0061] S1. Obtain the local model parameters of this round of iteration, divide the local model parameters into multiple secret shares corresponding to the participants, send each of the secret shares to the corresponding participants, and receive the secret shares sent by the participants;

[0062] In each round of aggregation process of the distributed federated learning aggregation method of the present invention, a server as a manager selects a subset of participants in each training iteration for aggregation and broadcasts the global model. During each round of iterative training aggregation, the client of each participant receives the list of participants distributed by the manager server, communicates with each participant in the list of participants, shares model parameters with each participant and verifies them.

[0063] Please refer to Figure 2 In this embodiment, the manager server and the client of the participant run in a semi-synchronous network with a bidirectional ring topology between the participants, and the participant-manager server connection adopts a star topology. In this embodiment, in order to improve efficiency, a Chord-based point-to-point network protocol is adopted, which simplifies the process of locating and broadcasting secret shares between the participants. Each participant maintains a fingerprint table (Finger Table, FT), and the number of entries in the fingerprint table corresponds to the bit length of the hash key , for the i-th participant The client of the kth party stores the address of the successor node in its FT. The address is Determine, where mod represents modular operation. The fingerprint table enables the participants to quickly identify the address of the next participant in the ring to achieve efficient message routing. In terms of security, the participants communicate directly with each other through authenticated channels that prevent interception or tampering. These channels are protected by protocols such as TLS (Transport Layer Security).

[0064] Given that it is difficult to verify whether the privacy-preserving computing process is executed correctly, in this embodiment, the client of each participant uses threshold secret sharing as the basic multi-party secure computing building block, and ensures the normal execution of the privacy-preserving computing process through zero-knowledge proof. Threshold secret sharing divides its own secret parameters into n secret shares based on the Shamir secret sharing method, and sets a threshold t. After the division, any t secret shares out of the n secret shares can be used to reveal the secret, and reconstructed through the Lagrange interpolation method to reconstruct the complete secret parameters, but less than t secret shares will not leak any information about the secret parameters.

[0065] Right now ;

[0066] in, represents the i-th participant For the jth participant The secret share established, where xi represents the i-th participant The secret parameter, represents the division based on Shamir's secret sharing method, n is the number of participants, which is also the number of secret shares divided by Shamir's secret sharing method, and t is the threshold of Shamir's secret sharing method.

[0067] Specifically, each participant has its own Polynomial ,in, represents the t-degree polynomial of the first participant, represents the i-th participant Polynomial of degree, Indicates the nth participant Polynomial of degree.

[0068] Each participant divides the secret parameter according to its own polynomial to obtain a secret share. For example, the secret parameter xi of the i-th participant is the secret share of the j-th participant. The secret shares established are:

[0069] ;

[0070] in, represents the i-th participant The polynomial of degree j is the value of variable j. is the i-th participant Parameters of the degree polynomial.

[0071] Please refer to Figure 3 In this embodiment, the system has an overlay network O for public information. At the beginning, the i-th participant With the jth participant Agreed public and private keys , where K is the safety parameter, Indicates Key Agreement (key negotiation, the i-th participant The public key Upload to the overlay network.

[0072] Then use the private key to After encryption, upload, the jth participant Get encrypted Then, use the private key to obtain , completing the exchange of secret shares.

[0073] S2. Aggregate locally according to the secret share to obtain a local aggregation vector, and share the local aggregation vector with the manager server.

[0074] In this embodiment, the manager server records and monitors any computational deviations or misconduct of the participants by maintaining a list V. The maintenance list V initially includes all participants. After the zero-knowledge proof of the computational process of each participant is completed, the manager server collects the shares that have passed a certain threshold verification and rebuilds the global model in the form of secret sharing. The manager server performs verifiable secret sharing (VSS) and counts the number of valid participants that meet all the requirements for computational integrity verification. Participants that are detected as failing the verification or exiting the protocol will be removed from the maintenance list V.

[0075] In this embodiment, it is assumed that the participants and the manager server are malicious adversaries. This assumption can provide strong security protection. Specific malicious behaviors may include: the participants and the manager server will try to steal the original data behind the secret parameters of the shared model; the participants and the manager server may not comply with the framework agreement in order to obtain the original data; the participants will try to destroy the aggregation results; and the participants will try to destroy the integrity of the intermediate aggregation results.

[0076] Zero-knowledge proof refers to a two-party zero-knowledge proof in which the prover is convinced that a specific statement made by the verifier is true without revealing any additional information. In this embodiment, an aggregate predicate is defined is a statement that a proving party wants to prove to a verifying party. In particular, this embodiment defines a zero-knowledge proof scheme involving multiple verifiers as: , where the symbol L represents the input is a true conditional language, PK stands for Proof of Knowledge, and x stands for testimony, i.e., the input secret parameter.

[0077] As mentioned above, it is difficult to verify whether the privacy-preserving computation process is executed correctly. The client of a malicious participant may distribute incorrect shares. In order to resolve this inconsistency, this embodiment needs to verify whether the secrets shared by each participant are normal, that is, the aggregation predicate Including shared consistency proof, in this embodiment, through a public Polynomial , for the jth participant , if the following formula is located in If the secret shares of all messages are consistent, then everyone has distributed the correct secret shares:

[0078] ;

[0079] in, express Polynomial The value of the variable j.

[0080] It can be seen that, given the polynomial addition property, all When the values ​​of the polynomials are added at the same point (j), the result of the addition is still a value at the same point (j). Through this method, it can be determined whether the received secret shares meet the consistency. After that, through the elimination method, it can be determined which secret shares do not meet the consistency and the participants who do not meet the consistency can be excluded.

[0081] In addition, in order to prevent malicious participants from making wrong calculations during the calculation process, or from making calculation errors due to other problems, in this embodiment, the aggregation algorithm is converted into a digital circuit, and the calculation of the digital circuit is verified using a multiplication triple, that is, the aggregation predicate Including calculation validity. Specifically, take the i-th participant The multiplication triplet calculated by the h-th multiplication For example, H represents the total number of multiplication calculations. and represents the i-th participant The two multiplication elements calculated by the h-th multiplication of represents the i-th participant The result of the h-th multiplication of , the i-th participant Generate proof components: , and upload it after encrypting it with the public key, and obtain the jth participant Proof of , then check The second-degree arithmetic relation of .

[0082] Specifically, the i-th participant Calculate first The value of The value of is downgraded to obtain the downgraded value. ,Right now , and then and To verify, that is:

[0083] ;

[0084] in, That is, the i-th participant For the jth participant The verification result is If it is 0, it means that the verification is passed. is a random number used to prevent the issued proof result This causes intermediate values ​​to be observed to further enhance privacy.

[0085] In addition, in order to speed up the calculation speed of verification, this embodiment defines:

[0086] ; where v represents the element of the G function and g represents the number of G function elements

[0087] definition:

[0088] ;

[0089] ;

[0090] Thus, it is proved , which is equivalent to the proof ; To avoid dishonest sharing of statements by parties and First, generate a set of auxiliary polynomial functions , so that yes The e-th input of yes The protocol then generates two polynomials q(1) and q(2), where , , so only need to check The value of .

[0091] In this embodiment, according to the aggregation predicate Participants that do not meet the requirements of shared consistency and computational validity are excluded, and only participants that meet the requirements of shared consistency and computational validity are retained to participate in subsequent aggregation operations.

[0092] It is worth noting that the verification in this embodiment has termination safety. If the number of participants who pass the verification is greater than the safety threshold, subsequent aggregation calculations are performed. If the number of participants who pass the verification is less than the safety threshold, subsequent aggregation calculations are not performed.

[0093] Specifically, in a synchronous environment, a message sent at time T is guaranteed to be correctly received at time T+∆. This means that an honest party will inevitably receive messages from other nt-1 honest parties after a finite delay. The present invention adopts Shamir's secret sharing, and any t correct message parts are sufficient to reconstruct the original message. Therefore, this requires nt≥t, resulting in n≥2t. This determines the fault tolerance to be (n-1) / 2. In addition, the system of this embodiment integrates Reed-Solomon error correction codes, so that secret sharing can withstand p errors and e erasures, provided that , where (nt-1) represents the number of honest participants. This makes the system robust to correcting up to (n-1) / 3 errors. Similarly, when the error rate reaches a threshold, the system can handle up to p=(n-1) / 2 erasures (i.e., the number of participants who exited the current aggregation round). Therefore, the number of malicious participants is the maximum malicious adversary threshold of the framework of the present invention against malicious participants, and (nm) is the security threshold of this embodiment.

[0094] In this embodiment, the aggregation algorithm may be a relationship-based aggregation algorithm, such as SignGuard, or a similarity-based aggregation algorithm, such as Multi-Krum algorithm.

[0095] In the relation-based aggregation algorithm, the input of the converted digital circuit is the kth participant The secret share shared by the i-th participant accepted , , and the secret share shared by the jth participant , ;

[0096] The output is and The order of magnitude .

[0097] Specifically, Perform local operations in parallel , get the size order relationship , and share it.

[0098] In the secret recovery phase, more than t non and The participants reconstruct the secret parameters of the i-th participant through the Lagrange interpolation method and the jth party's secret parameter Between The results are broadcast to all participants and Size relationship .

[0099] Finally, according to the specific aggregation algorithm, the i-th participant According to the relationship Compute local aggregation vector , and Share it with the manager server, which reconstructs the complete global aggregation vector based on the aggregation vectors of each participant through Lagrange interpolation method , according to the global aggregation vector Update Aggregate Model .

[0100] Similarly, the similarity-based algorithm takes as input the kth participant The secret share shared by the i-th participant accepted , , and the secret share shared by the jth participant , ;

[0101] The output is and Similarity ;

[0102] Specifically, Perform local operations in parallel , and share it;

[0103] In the secret recovery phase, more than t non and The participants reconstruct the similarity between xi and xj through Lagrange interpolation method ;

[0104] Finally, according to the specific aggregation algorithm, the i-th participant Calculate the local aggregation vector based on the similarity of each participant , and Share it with the manager server, which reconstructs the complete global aggregation vector based on the aggregation vectors of each participant through Lagrange interpolation method , according to the global aggregation vector Update Aggregate Model .

[0105] S3. Receive the global aggregation model updated by the manager server, and update the local model according to the global aggregation model.

[0106] After that, it can be used according to the local model. Specifically, the local model can be an image classification model, which performs classification tasks locally to identify and classify targets. It can also be a large end-to-end model, such as an autonomous driving model for cars. Similarly, it can also be other types of models.

[0107] The present invention verifies the effectiveness of the adopted method, which implements the code under the PyTorch deep learning framework and benchmarks the computation and communication costs of different entities in the protocol. The underlying hardware resources used by all participants are deployed on a server, which includes an Intel Xeon-6230 CPU @2.10GHz, a CentOS 7.5 operating system, a 10 Gbps data switch, and a 1TB hard disk. Local training is equipped with four Nvidia 3090 GPU cards and 90GB of memory. This framework uses Python version 3.8.10 and C programming language for code implementation. The ciphertext module design only uses CPU resources. The security parameters selected by this method for secret sharing and polynomial arithmetic operations all follow 128-bit security. This method applies an 8-bit probabilistic quantization method to ensure calculation accuracy while reducing the amount of parameter transmission communication.

[0108] Application Datasets: We test the ciphertext aggregation predicate on two real-world image classification task datasets. Fashion-MNIST: A grayscale dataset of 60,000 fashion item images suitable for clothing and accessories recognition. CIFAR-10: A color dataset of 60,000 images covering 10 categories for general object classification tasks. In terms of evaluating the accuracy of machine learning models, we consider the effectiveness of different defense strategies on two popular artificial intelligence (AI) application datasets. For Fashion-MNIST, we use the convolutional neural network LeNet-5 with 60K learnable parameters. For CIFAR-10, we use resnet-20 with 273K learnable parameters. Our framework evaluates two aggregation methods against poisoning attacks on global aggregation. 1) Multi-Krum: A classic aggregation rule that selects model updates based on their mutual proximity and excludes the farthest updates to ensure the robustness of aggregation. 2) SignGuard: A recently proposed aggregation rule that effectively uses the sign of the gradient instead of its actual value for model updates, providing compression advantages and robustness to small adversarial perturbations. At the same time, the present invention evaluates three Byzantine attack methods. 1) ALIE (A Little Is Enough): The attacker manipulates a small amount of data or model parameters to skew the global model update. This perturbation is usually small, but strategically selected to maximize the impact. 2) MinMax: The attacker tries to maximize the model's loss on its own data while minimizing the loss on other client data. 3) Noise: The attacker adds a specific noise vector to the model update. Given a true update ∇F, the contaminated update becomes ∇F+noise, where the noise is designed for adversarial purposes.

[0109] Please refer to the results Figure 4-Figure 10 ,in, Figure 4 This is a graph showing the single-round aggregation time results of the present invention against different attacks on the Fashion-MNIST dataset; Figure 5 This is a graph showing the single-round aggregation time results of the present invention against different attacks on the CIFAR10 dataset: Figure 6 It is the time cost diagram of zero-knowledge proof generation and verification of the present invention under 1K model parameters; Figure 7 It is the time cost diagram of zero-knowledge proof generation and verification under 100 clients of the present invention; Figure 8 It is a communication overhead diagram of zero-knowledge proof generation and verification under 1K model parameters of the present invention; Fig. 9 It is the communication overhead diagram of zero-knowledge proof generation and verification under 100 clients of the present invention; Fig.10 This is a graph comparing the convergence and accuracy test results of the aggregation model of the present invention and the defenseless model under malicious attacks based on the Fashion-MNIST dataset.

[0110] Embodiment 2

[0111] See also Fig.11 Based on the above method, the present invention also provides a terminal, which includes a processor 10, a memory 20 and a display 30. Fig.11 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0112] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a distributed federated learning aggregation program 40 is stored on the memory 20, and the distributed federated learning aggregation program 40 can be executed by the processor 10.

[0113] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing related programs of the distributed federated learning aggregation method.

[0114] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components of the terminal communicate with each other via a system bus.

[0115] In one embodiment, when the processor 10 executes the distributed federated learning aggregation program 40 in the memory 20, the steps of the distributed federated learning aggregation method described above are implemented.

[0116] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the terminal. Furthermore, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 can also be used to temporarily store data that has been output or is to be output. In one embodiment, a distributed federated learning aggregation program is stored on the memory 20, and the distributed federated learning aggregation program can be executed by the processor 10, thereby realizing the distributed federated learning aggregation method in the present application.

[0117] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the distributed federated learning aggregation method.

[0118] Embodiment 3

[0119] This embodiment provides a storage medium, wherein the readable storage medium stores a distributed federated learning aggregation program, and when the distributed federated learning aggregation program is executed by a processor, the steps of the distributed federated learning aggregation method described above are implemented.

[0120] In summary, the present invention solves the privacy problem of the existing aggregation algorithm by splitting the local model parameters in a secret splitting manner. In the process of federated learning, each participant can only obtain the split local model parameters, and the manager server can only obtain the final aggregation calculation results, but cannot obtain the intermediate local model parameters. It effectively solves the problem of lack of privacy protection in the existing aggregation algorithm relying on robustness, thereby preventing reasoning attacks.

[0121] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or terminal including the element.

[0122] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The storage medium can be a memory, a disk, an optical disk, etc.

[0123] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A distributed federated learning aggregation method, characterized in that: The distributed federated learning aggregation method includes: No. Participants Obtain the local model parameters of this iteration, and divide the local model parameters into multiple secret shares corresponding to the participants, represents the number of participants, Participants For the Participants The divided secret shares are , sending each of the secret shares to the corresponding participants, and receiving the secret shares sent by the participants as The secret shares divided among the participants ; No. Participants According to each The secret shares divided among the participants The robust aggregation algorithm is used for calculation to obtain the local robust aggregation parameters; Get the local robustness aggregation parameters transmitted by the participants, according to More than 100 local robust aggregation parameters are used to restore the global robust aggregation parameters; Aggregate locally according to the global robustness aggregation parameter and the secret share to obtain a local aggregation vector, and share the local aggregation vector with the manager server; in, is the threshold of the secret share; Receive a global aggregation model updated by a manager server, and update a local model according to the global aggregation model; The distributed federated learning aggregation method further includes: verifying the participants to obtain verified honest participants; Submit the verified honest participants to the manager server; if the number of the honest participants is greater than the security threshold, perform aggregation locally according to the secret share to obtain a local aggregation vector; If the number of the honest participants is less than the safety threshold, the distributed federated learning aggregation method is terminated; The safety threshold is not less than .

2. A distributed federated learning aggregation method according to claim 1, characterized in that: The said Participants The local model parameters of this iteration are obtained, and the local model parameters are divided into a plurality of secret shares corresponding to the participants, specifically including: No. Participants Get threshold value , according to the threshold value Establish Polynomial of degree; according to The secret share corresponding to each participant is calculated based on the local model parameters using a polynomial.

3. A distributed federated learning aggregation method according to claim 1, characterized in that: The verification of the participant specifically includes: Verify the shared consistency of the secret shares of all participants; The computational effectiveness of all participants in calculating local robustness aggregation parameters is verified.

4. A distributed federated learning aggregation method according to claim 1, characterized in that: The shared consistency verification of the secret shares of all participants specifically includes: Get Public Polynomial ; Get the local client ID in the system , which satisfies the security share of shared consistency regulation The corresponding party is the honest party; The security share that satisfies the shared consistency regulation satisfy: The value of is The polynomial is in the variable value The value of in, for When the variable value The value of .

5. A distributed federated learning aggregation method according to claim 1, characterized in that: The verification of the computational validity of all participants in calculating the local robustness aggregation parameters specifically includes: Get the multiplication triples of all participants in computing the local robustness aggregation parameters; All multiplication triplets are verified, and the parties that pass the verification are the honest parties.

6. A terminal, characterized in that: The terminal includes: a memory, a processor, and a distributed federated learning aggregation program stored in the memory and executable on the processor. When the distributed federated learning aggregation program is executed by the processor, the terminal is controlled to implement the steps of the distributed federated learning aggregation method as described in any one of claims 1 to 5.

7. A readable storage medium, characterized in that: The readable storage medium stores a distributed federated learning aggregation program, and when the distributed federated learning aggregation program is executed by a processor, the steps of the distributed federated learning aggregation method according to any one of claims 1 to 5 are implemented.

Images